Bug #26058
closed
productid is not published in the content view if that is the only item which changed in the sync
Description
Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1659549
Description of problem:
productid is not published in the content view if that is the only item which changed in the sync
Version-Release number of selected component (if applicable):
Satellite 6.3.5
How reproducible:
Always
Steps to Reproduce:
1.We had seen this happen often with kickstart repo like rhel 7.1, rhel 6.9 kickstart repos.
Customer scenario:
- Content view is published and promoted to a lifecycle environment successfully.
- Capsule sync started after the promotion completed
Actual results:
Capsule sync tasks finished with stopped/warning status with /var/log/messages showing. It appears that the productid of the kickstart repo changed in upstream but not other contents. With the current logic of publish to a lifecycle environment, katello thinks there is nothing new to publish to the target lifecycle environment. The orphan clean up job then comes and removes the old productid file in the content view version assuming it wouldn't be used anymore. But in reality the capsule is still referring to this old product id file and hence the error.
Dec 14 03:57:41 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://satellite.example.com/pulp/repos/Default_Organization/UAT/CV_TEST4/content/dist/rhel/server/7/7.1/x86_64/kickstart/repodata/4c4bc87d3301fd34ca1d49b4787c6f8ee4528e298f031bca447f803b1e374f28-productid.gz failed with code 404: Not Found
...
...
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) Not Found
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) Traceback (most recent call last):
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 263, in run
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) metadata_files = self.get_metadata(metadata_files)
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 450, in get_metadata
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) metadata_files.download_metadata_files()
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 217, in download_metadata_files
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) raise IOError(error_report.error_msg)
Dec 14 03:57:46 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [e28dec63] (10008-26688) IOError: Not Found
Dec 14 03:57:46 pulp: pulp.server.async.tasks:INFO: [e28dec63] Task failed : [e28dec63-a96d-4668-a9ee-879fa8a4ef07]
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) Task pulp.server.managers.repo.sync.sync[e28dec63-a96d-4668-a9ee-879fa8a4ef07] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) Traceback (most recent call last):
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) R = retval = fun(*args, **kwargs)
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 527, in call
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) return super(Task, self).__call__(*args, **kwargs)
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in call
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) return super(PulpTask, self).__call__(*args, **kwargs)
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in protected_call
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) return self.run(*args, **kwargs)
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 827, in sync
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Dec 14 03:57:46 pulp: celery.worker.job:ERROR: (9494-26688) PulpExecutionException: Importer indicated a failed response
Dec 14 03:57:46 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[0b79efb6-2771-4d27-890c-a39cab43f7f9] succeeded in 0.00446466822177s: None
Expected results:
When there is no actual change in the rpm contents and the change happens only in the product id and other metadata contents, they all should be pushed out to right life cycle environments during publish.
Additional info:
Temporary workaround is to re-publish the content view version to the desired lifecycle environment with forced yum metadata regeneration:
hammer content-view version promote --organization-id <id> --content-view-id <id> --version <version_number> --to-lifecyle-environment-id <lce_id> --force-yum-metadata-regeneration
Updated by Partha Aji almost 6 years ago
Better reproducer
0) Create a repo with the following bash script (Note it doesnot have a productid file)
DIR=/tmp/my-data
mkdir $DIR
cd $DIR
wget https://partha.fedorapeople.org/test-repos/rpm-with-productid/elephant-0.3-0.8.noarch.rpm
createrepo .
#start serving this dir
python -m SimpleHTTPServer 5050
1) Create and sync repo in sat with feed pointing to http://<fqdn>:5050
2) put it in a CV and publish and promote to an env
3) now update the repo with a product id. Something like
DIR=/tmp/my-data
cd $DIR
echo "100000" > productid
modifyrepo --mdtype=productid productid repodata
4) resync, publish the CV and promote the new version
5) Go to Monitor -> Tasks view and look for the latest promote
6) Go to dynflow console Run tab
7) Search for Actions::Katello::Repository::CheckMatchingContent
8) Expand and see check the value of match content
Expected:
matching_content: false
Actual:
matching_content: true
The point here is that "matching_content" value is used to determine whether metadata needs to be republished. However since katello was not tracking stuff in "repodata" directory (aka Yum Metadata) it ignored the fact that a productid file got added there. This fix indexes the yum metadata and thus katello is able to track changes in the repodata directory including productid.