Project

General

Profile

Bug #12295

Smart proxy stops listening on HTTPS port after log rotate

Added by Bryan Kearney over 3 years ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Category:
Core
Target version:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1273785
Description of problem:
The foreman-proxy stops listening on port 9090 after the logrotate was performed:

[crash] root@li-lc-1017:/etc/logrotate.d# grep '04:28' /var/log/cron
Oct 21 04:28:01 li-lc-1017 anacron31376: Job `cron.daily' started
Oct 21 04:28:01 li-lc-1017 run-parts(/etc/cron.daily)[1263]: starting logrotate
Oct 21 04:28:14 li-lc-1017 run-parts(/etc/cron.daily)[1382]: finished logrotate
Oct 21 04:28:14 li-lc-1017 run-parts(/etc/cron.daily)[1263]: starting makewhatis.cron
Oct 21 04:28:18 li-lc-1017 run-parts(/etc/cron.daily)[1555]: finished makewhatis.cron
Oct 21 04:28:18 li-lc-1017 run-parts(/etc/cron.daily)[1263]: starting mlocate.cron

[crash] root@li-lc-1017:/etc/logrotate.d# ls l /var/log/foreman-proxy/
total 4
-rw-r--r-
. 1 foreman-proxy foreman-proxy 0 Oct 19 12:22 access.log
rw-r--r-. 1 foreman-proxy foreman-proxy 0 Oct 21 04:28 proxy.log
rw-r--r-. 1 foreman-proxy foreman-proxy 1366 Oct 21 04:28 proxy.log-20151021.gz

[crash] root@li-lc-1017:/var/log/foreman-proxy# ps aux | grep proxy
root 922 0.0 0.0 103308 832 pts/0 S+ 08:24 0:00 grep proxy
493 1378 0.2 0.1 162960 54032 ? SNl 04:28 0:30 ruby /usr/share/foreman-proxy/bin/smart-proxy

[crash] root@li-lc-1017:/etc/logrotate.d# lsof -P -n | grep 'foreman-proxy.*TCP'
ruby 1378 foreman-proxy 5u IPv4 10120344 0t0 TCP *:8000 (LISTEN)

Checking the logrotate script is does a 'service foreman-proxy condrestart'.
I think that the startup might be too quickly after the stopping so the port 9090 is not released yet for re-use.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Force logrotate logrotate -f /etc/logrotate.conf
2.
3.

Actual results:
Foreman-proxy is not listening the port 9090

Expected results:
Foreman-proxy is listening again on the port 9090

Additional info:


Related issues

Related to Foreman Remote Execution - Bug #14314: Tests are failing due to logging change in smart proxyClosed2016-03-22
Related to Discovery - Bug #14315: Discovery should no longer use Proxy::Log directlyClosed2016-03-22
Related to Smart Proxy - Bug #14755: Error during startup, terminating. unsupported signal SIGUSR1 (windows)Closed2016-04-21
Related to foreman-tasks - Bug #19224: smart_proxy_dynflow_core.log not reopened during logrotateClosed2017-04-07

Associated revisions

Revision 7cc71a10 (diff)
Added by Dmitri Dolguikh about 3 years ago

Fixes #12295 - Added support for log rotation

SIGUSR1 now can be used to re-open log file when file-based logger is used.

History

#1 Updated by Dominic Cleal over 3 years ago

  • Status changed from New to Feedback
  • Assignee deleted (Ohad Levy)

Assuming this is Foreman 1.7, please try again with 1.10.0-RC1, the startup code has been refactored since. I can't reproduce any restart failure with condrestart alone.

#2 Updated by Dmitri Dolguikh about 3 years ago

Apparently the issue is still there. I don't think adding a delay between launching http and https apps is a great approach (slows down startup, 5 seconds may not be sufficient time to for the process to release the socket). Updating 'sigterm' handler to stop http and https apps and then exit smart-proxy would be my preferred approach.

#3 Updated by Dmitri Dolguikh about 3 years ago

  • Status changed from Feedback to New

#4 Updated by Dominic Cleal about 3 years ago

  • Subject changed from Capsule stops listening on 9090 after log rotate to Smart proxy stops listening on HTTPS port after log rotate
  • Category set to Core
  • Priority changed from High to Normal

Or provide another signal handler just for rolling logs, avoiding the need to restart at all.

#5 Updated by Dmitri Dolguikh about 3 years ago

I like the idea of using a dedicated signal for log rolling even better.

#6 Updated by Dmitri Dolguikh about 3 years ago

  • Status changed from New to Assigned
  • Assignee set to Dmitri Dolguikh

#7 Updated by The Foreman Bot about 3 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/smart-proxy/pull/394 added

#8 Updated by The Foreman Bot about 3 years ago

  • Pull request https://github.com/theforeman/foreman-packaging/pull/1058 added

#9 Updated by The Foreman Bot about 3 years ago

  • Pull request https://github.com/theforeman/foreman-packaging/pull/1060 added

#10 Updated by Anonymous about 3 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#11 Updated by Dominic Cleal about 3 years ago

  • Legacy Backlogs Release (now unused) set to 136

#12 Updated by Stephen Benjamin about 3 years ago

  • Related to Bug #14314: Tests are failing due to logging change in smart proxy added

#13 Updated by Stephen Benjamin about 3 years ago

  • Related to Bug #14315: Discovery should no longer use Proxy::Log directly added

#14 Updated by Dominic Cleal about 3 years ago

  • Related to Bug #14755: Error during startup, terminating. unsupported signal SIGUSR1 (windows) added

#15 Updated by Ivan Necas about 2 years ago

  • Related to Bug #19224: smart_proxy_dynflow_core.log not reopened during logrotate added

Also available in: Atom PDF