Bug #23604

Speed up manifest import with lots of pools

Added by Partha Aji 2 months ago. Updated 8 days ago.

Status:Closed
Priority:Normal
Assignee:Partha Aji
Category:Subscriptions
Target version:Katello 3.7.0
Difficulty: Team Backlog:
Triaged:Yes Fixed in Releases:
Bugzilla link:1573247 Found in Releases:
Pull request:https://github.com/Katello/katello/pull/7373

Description

If a user has lots of guest pools (for example, if there are lots of guests running on hypervisors), manifest import can take a long time.

Specifically, the "ImportProducts" step can take quite a while. For a Satellite with 3300 pools, this step can take 3.5 hours. It appears that during manifest import, all of the pools are reindexed. This happens one pool at a time, and each pool take a couple of seconds which leads to this issue.

Note:
The ImportProducts step in manifest import operation takes over 3 hrs on a database with over 3000 guest pools.

Associated revisions

Revision 6de94a2c
Added by Partha Aji 14 days ago

Fixes #23604 - Speed up manifest import perf

This commit intends to speed up the performance of the manifest import
by cutting down unnecessary operations during the Pool indexing stage.

Before
timer { Katello::Pool.find(446).import_data }
=> 12.708098227

After

timer { Katello::Pool.find(446).import_data }

=> 0.029546651

History

#1 Updated by The Foreman Bot 2 months ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/7373 added

#2 Updated by Partha Aji 2 months ago

  • Bugzilla link set to 1573247

#3 Updated by Justin Sherrill 2 months ago

  • Legacy Backlogs Release (now unused) set to 338

#4 Updated by Partha Aji 14 days ago

  • % Done changed from 0 to 100
  • Target version deleted (245)
  • Status changed from Ready For Testing to Closed

Also available in: Atom PDF