Bug #24024: IndexContent step of initial repo sync takes 20+ minutes - Katello - Foreman

Actions

Copy link

Bug #24024

closed

IndexContent step of initial repo sync takes 20+ minutes

Added by Chris Duryee over 6 years ago. Updated about 6 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Christine Fouant

Category:

Target version:

Katello Backlog

Difficulty:

Triaged:

Yes

Bugzilla link:

1593480

Pull request:

https://github.com/Katello/katello/pull/7862

Fixed in Releases:

Found in Releases:

Red Hat JIRA:

Description

If you on-demand sync a large repo like RHEL7 Server x86_64, the Pulp sync will take about 15-20 minutes. However, the IndexContent step will take even longer, perhaps 20-30 minutes.

It looks like most of the time is spent in import_all:

2018-06-20T21:53:50 [W|app|] begin import_all Katello::Pulp::Rpm
2018-06-20T22:06:47 [W|app|] end import_all Katello::Pulp::Rpm
2018-06-20T22:06:49 [W|app|] begin import_all Katello::Pulp::Srpm
2018-06-20T22:06:49 [W|app|] end import_all Katello::Pulp::Srpm
2018-06-20T22:06:51 [W|app|] begin import_all Katello::Pulp::Erratum
2018-06-20T22:18:21 [W|app|] end import_all Katello::Pulp::Erratum
2018-06-20T22:18:21 [W|app|] begin import_all Katello::Pulp::PackageGroup
2018-06-20T22:18:24 [W|app|] end import_all Katello::Pulp::PackageGroup

I think most of the time is spent in the loop in https://github.com/Katello/katello/blob/master/app/models/katello/concerns/pulp_database_unit.rb#L51-L57, where each unit is loaded individually. Resyncs do not take nearly as long.

Syncing one or more large repos is extremely common when setting up a Katello installation, so any time savings here would be a big deal.