Project

General

Profile

Bug #24024

IndexContent step of initial repo sync takes 20+ minutes

Added by Chris Duryee about 1 year ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Category:
-
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Team Backlog:
Fixed in Releases:
Found in Releases:

Description

If you on-demand sync a large repo like RHEL7 Server x86_64, the Pulp sync will take about 15-20 minutes. However, the IndexContent step will take even longer, perhaps 20-30 minutes.

It looks like most of the time is spent in import_all:

2018-06-20T21:53:50 [W|app|] begin import_all Katello::Pulp::Rpm
2018-06-20T22:06:47 [W|app|] end import_all Katello::Pulp::Rpm
2018-06-20T22:06:49 [W|app|] begin import_all Katello::Pulp::Srpm
2018-06-20T22:06:49 [W|app|] end import_all Katello::Pulp::Srpm
2018-06-20T22:06:51 [W|app|] begin import_all Katello::Pulp::Erratum
2018-06-20T22:18:21 [W|app|] end import_all Katello::Pulp::Erratum
2018-06-20T22:18:21 [W|app|] begin import_all Katello::Pulp::PackageGroup
2018-06-20T22:18:24 [W|app|] end import_all Katello::Pulp::PackageGroup

I think most of the time is spent in the loop in https://github.com/Katello/katello/blob/master/app/models/katello/concerns/pulp_database_unit.rb#L51-L57, where each unit is loaded individually. Resyncs do not take nearly as long.

Syncing one or more large repos is extremely common when setting up a Katello installation, so any time savings here would be a big deal.

Associated revisions

Revision 00eb9331 (diff)
Added by Christine Fouant 7 months ago

fixes #24024 - increases default bulk_load_size for pulp

History

#1 Updated by Chris Duryee about 1 year ago

  • Bugzilla link set to 1593480

#2 Updated by Andrew Kofink about 1 year ago

  • Legacy Backlogs Release (now unused) set to 338

#3 Updated by Jonathon Turel 12 months ago

  • Target version changed from Katello 3.7.0 to Katello 3.9.0

#4 Updated by Jonathon Turel 10 months ago

  • Target version changed from Katello 3.9.0 to Katello Backlog

#5 Updated by The Foreman Bot 8 months ago

  • Assignee set to Christine Fouant
  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/7862 added

#6 Updated by Christine Fouant 7 months ago

  • Status changed from Ready For Testing to Closed

Also available in: Atom PDF