Project

General

Profile

Refactor #34160

Improve the fact_values api performance when loading a large number of facts

Added by Tomer Brisker 12 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Facts
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

This is due to the current way the build_facts_hash method is implemented, leading to a lot of excessive memory load, and some unneeded preloads from the db.


Related issues

Related to Foreman - Bug #35591: Can't use 'search' with fact_value via APINew

Associated revisions

Revision 21fefa34 (diff)
Added by Tomer Brisker 12 months ago

Fixes #34160 - Improve fact_values api performance

Previously, the fact_values api called loaded a lot of un-needed data
from the database and used a sub-optimal way of converting the data to a
hash for display.
This commit changes the implementations so that we only load the
required columns from the database and generate the hash in a much more
efficient way. It also moves the build_facts_hash method from FactValue
model to the controller, which is its only consumer.

Some benchmarks:
Before, loading 20 facts (default per_page setting):
Completed 200 OK in 52ms (Views: 2.6ms | ActiveRecord: 15.5ms | Allocations: 35603)
Before, loading 1000 facts:
Completed 200 OK in 1181ms (Views: 2.2ms | ActiveRecord: 22.8ms | Allocations: 1390471)
Before, loading 10000 facts:
Completed 200 OK in 14581ms (Views: 7.4ms | ActiveRecord: 75.6ms | Allocations: 22283568)

After, loading 20 facts:
Completed 200 OK in 37ms (Views: 3.8ms | ActiveRecord: 18.3ms | Allocations: 15549)
After, loading 1000 facts:
Completed 200 OK in 53ms (Views: 2.4ms | ActiveRecord: 16.0ms | Allocations: 47083)
After, loading 10000 facts:
Completed 200 OK in 313ms (Views: 5.8ms | ActiveRecord: 38.7ms | Allocations: 433106)

History

#1 Updated by The Foreman Bot 12 months ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/9000 added

#2 Updated by The Foreman Bot 12 months ago

  • Fixed in Releases 3.2.0 added

#3 Updated by Kenny Tordeurs 12 months ago

  • Bugzilla link set to 2033593

#4 Updated by Tomer Brisker 12 months ago

  • Status changed from Ready For Testing to Closed

#5 Updated by Amit Upadhye 6 months ago

  • Subject changed from fact_values api performance issues when loading a large number of facts to Improve the fact_values api performance when loading a large number of facts

#6 Updated by Ewoud Kohl van Wijngaarden 1 day ago

  • Related to Bug #35591: Can't use 'search' with fact_value via API added

Also available in: Atom PDF