Project

General

Profile

Bug #24907

Page performance issues when there is a large number subnets

Added by Ondřej Pražák about 3 years ago. Updated 5 months ago.

Status:
Ready For Testing
Priority:
Normal
Assignee:
-
Category:
Performance
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Description of problem:
With 30k subnets, location edit page is unresponsive.

Steps to Reproduce:
1. Create lots of subnets (run this with params "100 200"):
https://github.com/redhat-performance/satperf/blob/master/scripts/create_lots_of_subnets.sh
2. Attempt to interact with the page tight after it is
rendered:
Manage Locations -> location

The subnet show page is slow to load as well.

History

#1 Updated by Ondřej Pražák about 3 years ago

  • Category changed from Network to Performance
  • Subject changed from Page performance issues when there is a large number subnets to Page performance issues when there is a large number subnets

#2 Updated by The Foreman Bot over 1 year ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/7464 added

#3 Updated by Nacho Barrientos 5 months ago

I'm seeing a similar problem on 2.4.0. When editing a location or an organisation with >35k hosts and >6k hostgroups, Foreman freezes the browser tab and the application becomes unusable.

Looking at the developer tools, I can see that all the server side requests finish fine and that there's nothing on the JS console (which is dead). The Firefox process associated to the tab is consuming CPU all the time and the UI is totally clogged (forever?). I'm attaching a screenshot right after the problem starts to happen.

Unfortunately I can't use performance logging because when the issue triggers even the developer console dies.

On the other hand, when I edit an empty location living in the same Foreman instance things work.

From IRC (#):

      <nacho> tbrisker: it must be hostgroups, then. Hostgroup.all.count => 6132.  [14:05]
   <tbrisker> nacho: i suppose so :)  [14:06]
      <nacho> tbrisker: haven't opened the ticket yet, would you rather if I added input to #24907?
   <tbrisker> nacho: sure, better than having duplicate issues
   <tbrisker> nacho: sad thing is it will likely require replacing the multiselect implementation to properly fix :\
   <tbrisker> nacho: i think you could comment out lines 40 and 81 in /usr/share/foreman/app/views/taxonomies/_form.html.erb
              if you don't want to manage hostgroups from that form as a workaround  [14:08]
   <tbrisker> nacho: but i would test that on a non-prod system first to make sure it doesn't break everything :)
      <nacho> tbrisker: our instance running 2.4.0 is staging with a one-month old clone of the production DB; I can patch,
              no problem.  [14:11]
      <nacho> tbrisker: okay so I'll add stuff to that ticket and try that suggested patch :)  [14:12]

As a workaround I'll try to get rid of the hostgroup selector to see if that helps.

#4 Updated by Nacho Barrientos 5 months ago

The attached patch (suggested by Tomer Brisker) works around the issue in my case. The edit form loads fine and we can edit locations/organisations, kinda confirming therefore that the problem is indeed the volume of hostgroups.

However I believe that, after patching, the POST that's issued to modify the object when clicking on Submit returns 200 however the object itself is not saved. I found this in the logs:

2021-06-01T15:26:29 [E|app|8d443936] Failed to save: Hostgroups expecting hostgroups used by hosts or inherited (check mismatches report).

"Full" log"

2021-06-01T15:26:08 [I|app|8d443936] Started PATCH "/locations/1-REDACTED" for REDACTED at 2021-06-01 15:26:08 +0200
2021-06-01T15:26:08 [I|app|8d443936] Processing by LocationsController#update as HTML
2021-06-01T15:26:08 [I|app|8d443936]   Parameters: {"utf8"=>"✓", 
      REDACTED PARAMETERS
2021-06-01T15:26:29 [E|app|8d443936] Failed to save: Hostgroups expecting hostgroups used by hosts or inherited (check mismatches report).
2021-06-01T15:26:29 [I|app|8d443936]   Rendering taxonomies/edit.html.erb within layouts/application
2021-06-01T15:26:42 [I|app|8d443936]   Rendered common_parameters/_parameter.html.erb (Duration: 3.2ms | Allocations: 1620)
2021-06-01T15:26:42 [I|app|8d443936]   Rendered common_parameters/_parameter.html.erb (Duration: 5.5ms | Allocations: 5905)
2021-06-01T15:26:42 [I|app|8d443936]   Rendered common_parameters/_parameters.html.erb (Duration: 13.2ms | Allocations: 9170)
2021-06-01T15:26:42 [I|app|8d443936]   Rendered taxonomies/_form.html.erb (Duration: 12977.9ms | Allocations: 5709090)
2021-06-01T15:26:42 [I|app|8d443936]   Rendered taxonomies/edit.html.erb within layouts/application (Duration: 13677.6ms | Allocations: 5877944)
2021-06-01T15:26:42 [I|app|8d443936]   Rendered layouts/_application_content.html.erb (Duration: 3.1ms | Allocations: 5273)
2021-06-01T15:26:42 [I|app|8d443936]   Rendering layouts/base.html.erb
2021-06-01T15:26:42 [I|app|8d443936]   Rendered layouts/base.html.erb (Duration: 30.1ms | Allocations: 19413)
2021-06-01T15:26:42 [I|app|8d443936] Completed 200 OK in 34341ms (Views: 13451.7ms | ActiveRecord: 1935.6ms | Allocations: 16857705)

What I tried to do was to add a parameter and indeed when I retrieve the object using the Foreman console there's no parameters indicating that the object was not saved.


irb(main):006:0> Location.first.parameters
=> {}

Also available in: Atom PDF