Project

General

Profile

Bug #16576

Structured facts importer is extremely slow

Added by Brandon Weeks over 3 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Facts
Target version:
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

After upgrading to 1.13.0-RC1, our Foreman server consumed all available resources and became effectively unusable with response times of 25k to 175k ms. This seems directly related to the changes made in #4528, as Api::V2::HostsController#facts is where the application spends 99% of its time after upgrade according to NewRelic.


Related issues

Related to Foreman - Feature #4528: Support Facter 2 structured factsClosed2014-03-03
Related to Foreman - Feature #12163: Add ability to expire facts based on fact name.Closed2015-10-14
Related to Foreman - Bug #9016: Fact import code consumes lot of memoryClosed2015-01-18

Associated revisions

Revision e5381d3f (diff)
Added by Dominic Cleal over 3 years ago

fixes #16576 - only load FactName IDs, not models during import

Restores behaviour from before c696f74, which only keeps FactName IDs to
avoid loading the entire model. For structured facts, the compose values
are also preloaded, so the model is only loaded and updated if the value
of compose changes from the preloaded value.

Revision c2eca541 (diff)
Added by Dominic Cleal over 3 years ago

fixes #16576 - only load FactName IDs, not models during import

Restores behaviour from before c696f74, which only keeps FactName IDs to
avoid loading the entire model. For structured facts, the compose values
are also preloaded, so the model is only loaded and updated if the value
of compose changes from the preloaded value.
(cherry picked from commit e5381d3ff9001d9fe3ce8d0f90afeec591571e97)

History

#1 Updated by Brandon Weeks over 3 years ago

This was mostly caused by a massive numbers of orphaned rows in the fact_names table. Here is a SQL query that removes any fact_names without corresponding values.

DELETE
FROM fact_names
WHERE fact_names.id NOT IN (SELECT DISTINCT fact_name_id FROM fact_values);

#2 Updated by Dominic Cleal over 3 years ago

  • Legacy Backlogs Release (now unused) set to 160

#3 Updated by Dominic Cleal over 3 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal

#4 Updated by Dominic Cleal over 3 years ago

  • Related to Feature #4528: Support Facter 2 structured facts added

#5 Updated by Dominic Cleal over 3 years ago

  • Related to Feature #12163: Add ability to expire facts based on fact name. added

#6 Updated by Lukas Zapletal over 3 years ago

  • Related to Bug #9016: Fact import code consumes lot of memory added

#7 Updated by Lukas Zapletal over 3 years ago

This patch #9016 fixes it, we are just finishing the review process. Could you try it and share some numbers?

https://github.com/theforeman/foreman/pull/3667

#8 Updated by The Foreman Bot over 3 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/3852 added

#9 Updated by Dominic Cleal over 3 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF