Project

General

Profile

Actions
Copy link

Bug #37332

closed

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Added by Michael Arlt 8 months ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
RPMs
Target version:
Foreman - 3.11.0
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
https://github.com/theforeman/pulpcore-packaging/pull/969
Fixed in Releases:
Foreman - 3.11.0
Found in Releases:
Katello - Katello 4.12.0, Katello - Katello 4.13.0
Red Hat JIRA:

Description

Tasks Synchronize repository

Tab Task shows Result: warning

Tab Errors shows:

Action:

Actions::Pulp3::Repository::CreatePublication

Input:
{"repository_id"=>70,
 "smart_proxy_id"=>1,
 "contents_changed"=>true,
 "skip_publication_creation"=>false,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{"response"=>
  {"task"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/"},
 "pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/",
    "pulp_created"=>"2024-04-09T14:08:29.286+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.publishing.publish",
    "logging_cid"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
    "created_by"=>"/pulp/api/v3/users/1/",
    "started_at"=>"2024-04-09T14:09:53.181+00:00",
    "finished_at"=>"2024-04-09T14:09:55.293+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/lib/python3.11/site-packages/pulpcore/tasking/tasks.py\", line 61, in _execute_task\n" +
       "    result = func(*args, **kwargs)\n" +
       "             ^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 384, in publish\n" +
       "    generate_repo_metadata(\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 686, in generate_repo_metadata\n" +
       "    checksum_type = cr_checksum_type_from_string(\n" +
       "                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 318, in cr_checksum_type_from_string\n" +
       "    return getattr(cr, checksum_type.upper())\n" +
       "           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "description"=>"module 'createrepo_c' has no attribute 'SHA1'"},
    "worker"=>"/pulp/api/v3/workers/018ec2e7-e086-7e2f-9ff7-7e7f057ffa05/",
    "child_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Generating repository metadata",
       "code"=>"publish.generating_metadata",
       "state"=>"failed",
       "total"=>1,
       "done"=>0}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["shared:/pulp/api/v3/repositories/rpm/rpm/018ec303-a1fd-7ff7-af7b-81104bf6aecb/",
      "shared:/pulp/api/v3/domains/018ec258-d879-75ba-a85e-e6f71108fc5c/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>14, "failed"=>1}}

Exception:

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Backtrace:

/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:108:in `block in check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `each'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:162:in `poll_external_task'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:22:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/cancellable.rb:14:in `run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:10:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:589:in `block (3 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/execute_if_contents_changed.rb:5:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/execution_wrapper.rb:91:in `wrap'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:588:in `block (2 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `block in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `block in with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:582:in `execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:303:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/director.rb:94:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:202:in `execute_job'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:172:in `invoke'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:169:in `block in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:113:in `local'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:44:in `block in <module:Sidekiq>'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:263:in `stats'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:13:in `call'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:80:in `global'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:124:in `block in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:39:in `prepare'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:123:in `dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:168:in `process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:78:in `process_one'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:68:in `run'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:8:in `watchdog'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:17:in `block in safe_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Action:

Actions::Pulp3::Repository::SavePublication

Input:

{"repository_id"=>70,
 "tasks"=>Step(12).output,
 "contents_changed"=>Step(9).output[:contents_changed],
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{}

Actions
Copy link
 #2

Updated by Michael Arlt 8 months ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by Michael Arlt 8 months ago

I am not 100% sure but i think that the synchronisation worked with 3.10.0 rc1 or rc2.

Actions
Copy link
 #4

Updated by Michael Arlt 7 months ago

Adding pulpcore::allowed_content_checksums: ['sha1', 'sha224', 'sha256', 'sha384', 'sha512']
to custom-hiera.yaml creates

ALLOWED_CONTENT_CHECKSUMS = ["sha1", "sha224", "sha256", "sha384", "sha512"]

in /etc/pulp/settings.py but the problem persists.

Actions
Copy link
 #5

Updated by Samir Jha 7 months ago

  • Status changed from New to Need more information
  • Assignee set to Samir Jha
  • Triaged changed from No to Yes

Hi Michael,

Could you open an issue with pulp team here: https://github.com/pulp/pulp_rpm/issues.

We can keep this issue open if this ends up being a katello issue or a configuration issue.

Actions
Copy link
 #7

Updated by Daniel Alley 7 months ago

I believe this is a build issue with the "legacy_hashes" flag not being enabled for the RHEL 9 build of the package https://github.com/rpm-software-management/createrepo_c/blob/master/createrepo_c.spec#L27-L31

I don't believe we've modified those lines from upstream and thus it would likely not be enabled by default. So we should tweak those lines or set that flag.

Actions
Copy link
 #9

Updated by Michael Arlt 7 months ago

@Samir: Could you please let me know if and when there will be a solution to the problem?
I really don't want to cause any stress - this is just for my planning.

Actions
Copy link
 #10

Updated by Michael Arlt 7 months ago

  • Found in Releases Katello 4.13.0, Katello 4.14.0 added

reproducible with Katello 4.13 nightly:

$ rpm -qa|grep -i katello
katello-repos-4.13-0.1.nightly.el9.noarch
katello-common-4.13.0-0.1.master.el9.noarch
katello-certs-tools-2.10.0-1.el9.noarch
katello-selinux-5.0.2-1.el9.noarch
foreman-installer-katello-3.11.0-0.1.develop.20240418031237gitd83807b.el9.noarch
rubygem-katello-4.13.0-0.1.pre.master.20240411211935git58d2cdb.el9.noarch
katello-4.13.0-0.1.master.el9.noarch
rubygem-hammer_cli_katello-1.12.0-0.1.pre.master.el9.noarch
katello-client-bootstrap-1.7.9-2.el9.noarch

Actions
Copy link
 #11

Updated by Michael Arlt 7 months ago

  • Found in Releases deleted (Katello 4.14.0)
Actions
Copy link
 #12

Updated by Michael Arlt 7 months ago

  • Triaged changed from Yes to No
Actions
Copy link
 #13

Updated by Michael Arlt 7 months ago

  • Status changed from Need more information to Feedback
Actions
Copy link
 #14

Updated by Michael Arlt 7 months ago

  • Status changed from Feedback to New
Actions
Copy link
 #15

Updated by Samir Jha 7 months ago

Hi Michael,

Looking at Daniel's comment on the pulp issue, this doesn't seem like something we can fix in katello. I'd propose closing this issue and following up on the community?

Actions
Copy link
 #16

Updated by Michael Arlt 7 months ago

Hi Samir,

I don't know where we should open the issue but it is not possible to use SHA1 Repos in Foreman 3.10 on RHEL 9.
Please don't close - instead please ask others in the foreman project where it should addressed.

Actions
Copy link
 #17

Updated by Ian Ballou 7 months ago

  • Status changed from New to Need more information

This issue cannot be fixed by Pulp or Foreman directly if the cause is due to a built version of createrepo_c that doesn't allow dealing with SHA1 content. I'll leave the issue open for now so we don't forget about it, but the fix will require finding who is in charge of building createrepo_c and seeing if enabling SHA1 via the flag that Daniel mentioned is possible.

Edit: as it turns out, we are packaging createrepo_c, I thought it was coming from the EL9 repos. Changing this into a foreman-packaging issue. To support syncing SHA1 content, we will need to enable that legacy flag in our build of createrepo_c.

Actions
Copy link
 #18

Updated by Ian Ballou 7 months ago

  • Project changed from Katello to Packaging
  • Category changed from Sync Plans to RPMs
  • Assignee deleted (Samir Jha)
  • Target version set to 3.11.0
Actions
Copy link
 #19

Updated by Ian Ballou 7 months ago

  • Pull request https://github.com/theforeman/pulpcore-packaging/pull/969 added
Actions
Copy link
 #20

Updated by Michael Arlt 7 months ago

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

Actions
Copy link
 #21

Updated by Odilon Sousa 7 months ago

Michael Arlt wrote in #note-20:

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

It's published now, you can find the pulpcore 3.39 repo.

Actions
Copy link
 #22

Updated by Michael Arlt 7 months ago

It's published now, you can find the pulpcore 3.39 repo.

Works! Thank you so much!

Actions
Copy link
 #23

Updated by Ian Ballou 7 months ago

  • Status changed from Need more information to Closed

Closing this out now since it sounds like it's all taken care of.

Actions
Copy link
 #24

Updated by Griffin Sullivan 6 months ago

  • Fixed in Releases 3.11.0 added
Actions
Copy link
 #25

Updated by Ewoud Kohl van Wijngaarden 5 months ago

  • Triaged changed from No to Yes
Actions
Copy link

Also available in: Atom PDF