Project

General

Profile

Actions

Bug #16576

closed

Structured facts importer is extremely slow

Added by Brandon Weeks over 8 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Facts
Target version:
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

After upgrading to 1.13.0-RC1, our Foreman server consumed all available resources and became effectively unusable with response times of 25k to 175k ms. This seems directly related to the changes made in #4528, as Api::V2::HostsController#facts is where the application spends 99% of its time after upgrade according to NewRelic.


Related issues 3 (0 open3 closed)

Related to Foreman - Feature #4528: Support Facter 2 structured factsClosedDominic Cleal03/03/2014Actions
Related to Foreman - Feature #12163: Add ability to expire facts based on fact name.ClosedMarek Hulán10/14/2015Actions
Related to Foreman - Bug #9016: Fact import code consumes lot of memoryClosedLukas Zapletal01/18/2015Actions
Actions #1

Updated by Brandon Weeks over 8 years ago

This was mostly caused by a massive numbers of orphaned rows in the fact_names table. Here is a SQL query that removes any fact_names without corresponding values.

DELETE
FROM fact_names
WHERE fact_names.id NOT IN (SELECT DISTINCT fact_name_id FROM fact_values);
Actions #2

Updated by Dominic Cleal over 8 years ago

  • Translation missing: en.field_release set to 160
Actions #3

Updated by Dominic Cleal over 8 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal
Actions #4

Updated by Dominic Cleal over 8 years ago

  • Related to Feature #4528: Support Facter 2 structured facts added
Actions #5

Updated by Dominic Cleal over 8 years ago

  • Related to Feature #12163: Add ability to expire facts based on fact name. added
Actions #6

Updated by Lukas Zapletal over 8 years ago

  • Related to Bug #9016: Fact import code consumes lot of memory added
Actions #7

Updated by Lukas Zapletal over 8 years ago

This patch #9016 fixes it, we are just finishing the review process. Could you try it and share some numbers?

https://github.com/theforeman/foreman/pull/3667

Actions #8

Updated by The Foreman Bot over 8 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/3852 added
Actions #9

Updated by Dominic Cleal over 8 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100
Actions

Also available in: Atom PDF