Bug #14807
closedCapsule sync redundantly generates metadata for all repos
Description
Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1315326
Description of problem:
Invoking capsule sync:
1) Sat orders capsule to sync all repositories in all content views / lifecycle environments to the capsule
2) for every such repository, pulp on the capsule generates new repo metadata
Assume a use case where Sat having few hundreds of repositories in different content views, and just some repo might need to be synced (as there are doubts if it was synced properly). A capsule sync would do so so much redundant work taking hours.
To see the scope of ridiculous work being done:
- assume a use case where large repos (say rhel5-7 base) are present in many content views as a base - repo metadata takes nontrivial time to be computed, multiply it by # of repos..
- particular example: a customer behind this bug has 1250 repos and capsule sync takes 3-4 hours(!) doing nothing
Please optimize either 1 or 2. While I understand Sat does not know what repo needs and what does not need to be synced to the capsule (i.e. 1 sounds legit), there should be an option to e.g. fetch metadata from Sat to Caps, compare if they are the same and if so, do nothing (and if differ or missing on Caps, then do the sync).
Version-Release number of selected component (if applicable):
Sat 6.1.7
How reproducible:
100%
Steps to Reproduce:
1. Have more pulp repos enabled (e.g. have sat61-tools repo in 10 published content views)
2. do repeatedly capsule sync, without any repo / content view manipulation meantime
3. Check times of execution of 1st sync and other synces
4. ll /var/lib/pulp/published/yum/https/repos/Default_Organization/Library/ContentViewName1/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/repodata/
Actual results:
3. synces takes the same/very similar time, nontrivial
4. repodata recalculated every time
Expected results:
3. sync takes small time, if no work to be done
4. repodata not updated / recalculated every time (if there is no need to)
Additional info:
Updated by Tomáš Strachota over 8 years ago
- Category changed from Web UI to Repositories
- Status changed from New to Assigned
Updated by The Foreman Bot over 8 years ago
- Status changed from Assigned to Ready For Testing
- Pull request https://github.com/Katello/katello/pull/5996 added
Updated by Tomáš Strachota over 8 years ago
- Status changed from Ready For Testing to Closed
I'm closing the issue. Per discussion with Justin optimizations have been made on Pulp side and metadata creation process is much faster now. Therefore it's not necessary to skip it.
Updated by Eric Helms over 8 years ago
- Status changed from Closed to Rejected
- Translation missing: en.field_release set to 114
Updated by Jonathon Turel over 4 years ago
- Target version changed from Katello Backlog to Katello Recycle Bin