Project

General

Profile

Bug #21295

Slow start of rails environment when starting passenger or foreman-tasks

Added by Adam Ruzicka over 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Rails
Target version:
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1493046

Description of problem:
In satellite 6.3, the time for everything to be ready to respond
to http requests or handle tasks is significantly longer than in 6.2.
This leads to potential initial timeouts when trying to register
a host after the installation finishes or after restarting of services.

Steps to Reproduce (a):
1. systemctl restart httpd
2. wait until the /users/login is available

Actual results:
It takes about 2 minutes to be ready

Expected results:
On similar setup on satellite 6.2, it takes under a minute

Steps to Reproduce (b):
1. systemctl restart foreman-tasks
2. watch /var/log/foreman/dynflow_executor.output for containing 'Everything ready for world'

Actual results:
It takes about 3.5 minutes to be ready

Expected results:
On similar setup on satellite 6.2, it takes under a minute

Additional info:

I've narrowed it down to /opt/rh/rh-ror42/root/usr/share/gems/gems/railties-4.2.6/lib/rails/application/routes_reloader.rb , that runs twice during the boot time and seems to cause the significant slow down at startup. It seems that
something changed between rails 4.1 and 4.2 to cause the delays.

The fix should make sure we don't start reloading the routes twice at the initialization, as that's what causes the long time to start.

Setting the severity on high, as it has impact not only on response times after the installation finishes, but also when the passenger scales up the processes.


Related issues

Related to Katello - Bug #22743: Update failed at migrate_foreman: "can't modify frozen Array"Closed2018-03-01

Associated revisions

Revision c2e58b1b (diff)
Added by Ivan Necas over 2 years ago

Fixes #21295 - load the routes before we load the controllers

After upgrade from Rails 4.1 to Rails 4.2 there was significant
drop on rails startup in production. I have tracked it down to
several changes in Rails, but it might also be affected by the
fact the we use the url_helpers on more places than older Foreman
versions.

The reason of the slowness is the `define_method` calls on a module for
every named route we have. The `define_method` is more expensive
the more classes include the module that gets the new method.

The solution of this problem is to load the routes before we
load the controllers (including the eager loading) so that the modules
get defined new methods before the controllers include it in their code.

The original after_finalize loading is still here, but should not get
run again, as it callse `execute_if_updated` and nothing new usually
gets updated there.

To test, one can try in devel setup:

time echo exit | bundle exec rails c production

or in production:

service httpd restart; time curl -k https://localhost

In devel setup, I got from 30s to 10s, in production with Katello
and other plugins (it gets more and more visible with more plugins
around), I got from 1m 15s to 15s.

Please not `foreman-rake console` is not affected, as the rake
environment turns off the eager loading.

History

#1 Updated by Adam Ruzicka over 2 years ago

  • Category set to Foreman plugin
  • Assignee set to Ivan Necas
  • Target version set to 226

#2 Updated by Ivan Necas over 2 years ago

  • Project changed from foreman-tasks to Foreman
  • Category changed from Foreman plugin to Rails

#3 Updated by The Foreman Bot over 2 years ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/4908 added

#4 Updated by Marek Hulán over 2 years ago

  • Legacy Backlogs Release (now unused) set to 296

#5 Updated by Ivan Necas over 2 years ago

  • % Done changed from 0 to 100
  • Status changed from Ready For Testing to Closed

#6 Updated by Ivan Necas about 2 years ago

  • Related to Bug #22743: Update failed at migrate_foreman: "can't modify frozen Array" added

Also available in: Atom PDF