Project

General

Profile

Bug #20233

Importing manifest gets slow with increasing number of organizations

Added by Justin Sherrill about 4 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Category:
Subscriptions
Target version:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1339696

Description of problem:
It looks like candlepin polls every org and every activation key every time a manifest is being uploaded.

this is the section where my manifest has been uploaded to Default Organization, which was the only org on the system:
http://pastebin.com/RTZnhL85

here I created a new organization my_org_1 and did the same thing:
http://pastebin.com/nPZwdryD
(notice, there is polling to Default Organization done too).

I've also extracted the CI machine's candlepin log around the manifest import phase:
http://pastebin.com/1VhrEx1m
This took about 11 minutes

I realize 1000+ is a bit extreme and is not likely to be used in a production, however this causes many timeout issues to our automation. Also it is good to demonstrate the increasing time needed to process the manifest.

Also it looks like even a few organizations with many Activation Keys might cause the issue.

Version-Release number of selected component (if applicable):
6.2.0 beta (GA12.1)

Steps to Reproduce:
1. get multiple organizations, optionally with many activation keys
2. try to upload the manifest
3. experiment by adding more organizations and activation keys

Expected results:
The time taken to import a manifest should be constant and shouldn't exceed 1 minute (subjective)

Additional info:

Associated revisions

Revision a7c056f6 (diff)
Added by Justin Sherrill about 4 years ago

Fixes #20233 - speed up manifest actions

by only importing subscriptions from the org
performing the action

Revision 65f557c1 (diff)
Added by Justin Sherrill about 4 years ago

Fixes #20233 - better handle pool/sub import

the previous commit introduced a bug where it would
delete all subs/pools not being indexed by the current org.
In addition lets use active_record_retry to ensure race conditions
will not result in failed imports

History

#1 Updated by The Foreman Bot about 4 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Justin Sherrill
  • Pull request https://github.com/Katello/katello/pull/6853 added

#2 Updated by Brad Buckingham about 4 years ago

  • Target version set to 202

#3 Updated by Justin Sherrill about 4 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#4 Updated by Justin Sherrill about 4 years ago

  • Legacy Backlogs Release (now unused) set to 267

#5 Updated by The Foreman Bot about 4 years ago

  • Pull request https://github.com/Katello/katello/pull/6873 added

#6 Updated by Justin Sherrill about 4 years ago

  • Status changed from Closed to Ready For Testing
  • Legacy Backlogs Release (now unused) changed from 267 to 281

#7 Updated by Brad Buckingham about 4 years ago

  • Target version changed from 202 to 205

#8 Updated by Justin Sherrill about 4 years ago

  • Status changed from Ready For Testing to Closed

#9 Updated by The Foreman Bot about 4 years ago

  • Pull request https://github.com/Katello/katello/pull/6890 added

Also available in: Atom PDF