Bug #14924

taxonomies/edit.html.erb takes minutes to render

Added by Taylan Develioglu about 1 year ago. Updated 7 months ago.

Status:Closed
Priority:Normal
Assigned To:Tomer Brisker
Category:Organizations and Locations
Target version:Team Daniel - iteration 6
Difficulty: Bugzilla link:1396642
Found in release:1.11.1 Pull request:https://github.com/theforeman/foreman/pull/3491
Story points-
Velocity based estimate-
Release1.12.0Release relationshipAuto

Description

Trying to open the edit page of either a location or organization takes multiple minutes.

The time is spent rendering taxonomies/edit.html.erb and the passenger process stays ~100% utilization throughout the request.

Log attached is from rails environment set to `development`, notice that it takes over 6 minutes to complete the request.

taxonomies_edit_render.log (30.8 KB) Taylan Develioglu, 05/04/2016 05:40 AM

taxonomies_edit_render_w_sql.log.gz (2.58 MB) Taylan Develioglu, 05/04/2016 06:11 AM

pull_3491_result.log.gz (119 KB) Taylan Develioglu, 05/04/2016 08:19 AM


Related issues

Related to Foreman - Bug #10893: Editing organizations/locations hangs in 1.8.2 Resolved 06/22/2015
Duplicated by Foreman - Bug #10138: Manage Location and Organization unusable once thousands ... Duplicate 04/14/2015

Associated revisions

Revision 956055dc
Added by Tomer Brisker about 1 year ago

Fixes #14924 - Improve performance of taxonomy edit page

Non standard ids were not cached in TaxHost.
This led to multiple repeat queries in the database each time any of the
`*_ids` function were being called.
Used templated were calculated by going over all hosts in the taxonomy
and trying to lookup the provisioning template for each one, we now look
for all templates that match any combination of os, hostgroup and
environment for the taxonomy's hosts.
Hosts in Taxhost did not include the :interface table, leading to n+1
queries for delegated attributes such as subnets and proxies.

History

#1 Updated by Dominic Cleal about 1 year ago

  • Related to Bug #10893: Editing organizations/locations hangs in 1.8.2 added

#2 Updated by Dominic Cleal about 1 year ago

Could be the same as #10893 and related issues, but enabling SQL logging may show for sure (http://theforeman.org/manuals/1.11/index.html#7.2Debugging).

#3 Updated by Taylan Develioglu about 1 year ago

Attached logfile with SQL logging enabled.

It shows a huge amount of this repeating:
2016-05-04 12:01:04 [sql] [D] CACHE (0.0ms) SELECT "templates".* FROM "templates" INNER JOIN "operatingsystems_provisioning_templates" ON "operatingsystems_provisioning_templates"."provisioning_template_id" = "templates"."id" INNER JOIN "operatingsystems" ON "operatingsystems"."id" = "operatingsystems_provisioning_templates"."operatingsystem_id" INNER JOIN "template_kinds" ON "template_kinds"."id" = "templates"."template_kind_id" INNER JOIN "template_combinations" ON "template_combinations"."provisioning_template_id" = "templates"."id" WHERE "templates"."type" IN ('ProvisioningTemplate') AND "operatingsystems"."id" = 1 AND "template_kinds"."name" = 'provision' AND "template_combinations"."hostgroup_id" IS NULL AND "template_combinations"."environment_id" = 1 ORDER BY templates.name LIMIT 1

#4 Updated by Tomer Brisker about 1 year ago

  • Status changed from New to Assigned
  • Assigned To set to Tomer Brisker

#5 Updated by The Foreman Bot about 1 year ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/3491 added

#6 Updated by Taylan Develioglu about 1 year ago

I see a significant improvement with PR 3491 active. Wallclock went down from ~6 minutes to ~1 minute. Kudos!.
I do still notice a lot of repeat sql queries being done, however (see attached log).

#7 Updated by Taylan Develioglu about 1 year ago

The final PR reduced wallclock time to ~20s! \o/

#8 Updated by Anonymous about 1 year ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#9 Updated by Marek Hulán about 1 year ago

  • Release set to 1.12.0

#10 Updated by Ohad Levy 7 months ago

  • Bugzilla link set to 1396642

#11 Updated by Tomer Brisker 7 months ago

  • Duplicated by Bug #10138: Manage Location and Organization unusable once thousands of nodes are assigned added

#12 Updated by Daniel Lobato Garcia 7 months ago

  • Target version set to Team Daniel - iteration 6

Also available in: Atom PDF