Bug #10458
closedRPMs were downloaded with invalid checksums and pulp would not download the proper RPMs on future syncs
Description
Please have katello verify checksums of rpms by default to keep from having issues where rpms are not properly download.
[09:12] <jsherrill> so i believe there is an option on the yum importer in pulp called 'verify_checksum'
[09:12] <jsherrill> that defaults to false
[09:12] <jsherrill> so if pulp downloads a file, it doesn't verify its expected checksum
[09:13] <jsherrill> which can lead to situations like this.....
[09:13] <jsherrill> set that on by default, which i thought we already had
[09:18] <jsherrill> but you would edit /etc/pulp/server/plugins.conf.d/yum_importer.json
[09:19] <jsherrill> and add: "verify_checksum": true,
[09:19] <jsherrill> and then do a katello-service restart
Updated by Eric Helms almost 9 years ago
- translation missing: en.field_release set to 51
- Triaged changed from No to Yes
Updated by Eric Helms almost 9 years ago
- Status changed from New to Need more information
Was this a custom repository or one that I could test out myself? From inspecting the Pulp documentation checksum verification appears to be on by default and when I tried to test this locally I was not able to get to a test situation since Pulp also checks and verifies the size of the RPM before the checksum.
Updated by Leah Fisher almost 9 years ago
This occurred on the Centos OS repo I was syncing. I believe it happened because something went wrong with my system while syncing that put the rpms into an invalid state (I manually ran md5sums on the bad packages and they did not match the upstream md5sums I was syncing to) and a new sync did not correct the problem.
Updated by Leah Fisher almost 9 years ago
A good trial would be to modify an rpm on your local filesystem so it no longer passes the checksum value, and then sync again.
Updated by Eric Helms almost 9 years ago
I am still seeing proper behavior. Here is my test scenario:
root@katello-develnew zoo5]# createrepo ./ Spawning worker 0 with 2 pkgs Spawning worker 1 with 2 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@katello-develnew zoo5]# ls cheetah-0.3-0.8.noarch.rpm elasticsearch-0.90.10-7.el7.noarch.rpm katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm lion-0.3-0.8.noarch.rpm repodata [root@katello-develnew zoo5]# ls -la total 9268 drwxr-xr-x. 3 root root 4096 May 15 15:29 . drwxr-xr-x. 4 root root 4096 May 15 13:01 .. -rw-r--r--. 1 root root 2232 Dec 7 2011 cheetah-0.3-0.8.noarch.rpm -rw-r--r--. 1 root root 9458852 May 5 22:44 elasticsearch-0.90.10-7.el7.noarch.rpm -rw-r--r--. 1 root root 8156 May 3 22:19 katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm -rw-r--r--. 1 root root 2212 Dec 7 2011 lion-0.3-0.8.noarch.rpm drwxr-xr-x. 2 root root 4096 May 15 15:29 repodata [root@katello-develnew zoo5]# sha256 katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm bash: sha256: command not found [root@katello-develnew zoo5]# sha256sum katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm 55ff6d247868f1e1c41c36d70b7b434172d65009b4c3eb71969aa645bfa60217 katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm [root@katello-develnew zoo5]# printf '\x31\xc0\xc3' | dd of=katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm bs=1 seek=100 count=3 conv=notrunc 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.000128146 s, 23.4 kB/s [root@katello-develnew zoo5]# ls cheetah-0.3-0.8.noarch.rpm elasticsearch-0.90.10-7.el7.noarch.rpm katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm lion-0.3-0.8.noarch.rpm repodata [root@katello-develnew zoo5]# ls -la total 9268 drwxr-xr-x. 3 root root 4096 May 15 15:29 . drwxr-xr-x. 4 root root 4096 May 15 13:01 .. -rw-r--r--. 1 root root 2232 Dec 7 2011 cheetah-0.3-0.8.noarch.rpm -rw-r--r--. 1 root root 9458852 May 5 22:44 elasticsearch-0.90.10-7.el7.noarch.rpm -rw-r--r--. 1 root root 8156 May 15 15:30 katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm -rw-r--r--. 1 root root 2212 Dec 7 2011 lion-0.3-0.8.noarch.rpm drwxr-xr-x. 2 root root 4096 May 15 15:29 repodata [root@katello-develnew zoo5]# sha256sum katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm 5d9f29aa5e94ef2c01bfc68c01c637bc9b993962f4520a859fde29812423d644 katello-debug-2.3.0-1.201505040130git012d5ba.el7.noarch.rpm
The output I got was:
New packages: 1 (7.96 KB). Failed to download 1 package.
When looking at the Pulp call report:
"error_details"=> [{"checksum_type"=>"sha256", "actual_checksum"=> "5d9f29aa5e94ef2c01bfc68c01c637bc9b993962f4520a859fde29812423d644", "error_code"=>"checksum_mismatch", "name"=>"katello-debug", "expected_checksum"=> "55ff6d247868f1e1c41c36d70b7b434172d65009b4c3eb71969aa645bfa60217"}]},
Updated by Eric Helms almost 9 years ago
- translation missing: en.field_release changed from 51 to 55
Updated by Leah Fisher almost 9 years ago
When looking at your output:
The output I got was:
New packages: 1 (7.96 KB).
Failed to download 1 package.
What does this mean? I would expect the result would be that the invalid checksum file on your system would be replaced with the valid file you tried to download. But as I look at the message, it looks like all it says is that it failed to download package so there is no way to fix the package? It's not clear to me how this is truly working.
Workflow I would like to see with Katello:
1. Sync fails for unknown reason, files left in inconsistent states
2. Run sync again - sync sees that some files were downloaded incorrectly so replaces these with correct versions
3. The local repo is fully functional again.
Updated by Eric Helms almost 9 years ago
You can click the 'More Details' link or if the sync has already occurred, you can go to the tasks page and go into the sync itself. Within the details of the Task information will be a large JSON structure that comes from Pulp and includes the reason for the failed package.
In the case of a checksum issue, Pulp attempts to validate the checksum of the package against the metadata and if they are different it does not keep that package around. Thus, the way to fix that type of issue is fix the origin checksum of the package and then re-sync it. Ultimately Pulp controls the entire sync process and how it handles this. We'd need to file a bug and discuss with them if we wanted to change how the sync is being handled.
Updated by Eric Helms almost 9 years ago
- translation missing: en.field_release changed from 55 to 61
Updated by Eric Helms over 8 years ago
- translation missing: en.field_release changed from 61 to 31
Updated by Stephen Benjamin over 8 years ago
- translation missing: en.field_release changed from 31 to 70
Mass move to 2.4.0
Updated by Justin Sherrill over 8 years ago
- translation missing: en.field_release changed from 70 to 86
Updated by Eric Helms almost 8 years ago
- translation missing: en.field_release changed from 86 to 114
Updated by John Mitsch over 4 years ago
- Status changed from Need more information to Rejected
- Target version deleted (
Katello Backlog)
Thanks for reporting this issue. This issue was created over 4 years ago and hasn't seen an update in 1 year. We are closing this in an effort to keep a realistic backlog. Please open up a new issue that includes a link to this issue if you feel this still needs to be addressed. We can then triage the new issue and reassess.