Project

General

Profile

Actions

Bug #37332

closed

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Added by Michael Arlt 3 months ago. Updated 23 days ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
RPMs
Target version:
Difficulty:
Triaged:
Yes
Fixed in Releases:

Description

Tasks Synchronize repository

Tab Task shows Result: warning

Tab Errors shows:

Action:

Actions::Pulp3::Repository::CreatePublication
Input:
{"repository_id"=>70,
 "smart_proxy_id"=>1,
 "contents_changed"=>true,
 "skip_publication_creation"=>false,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{"response"=>
  {"task"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/"},
 "pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/",
    "pulp_created"=>"2024-04-09T14:08:29.286+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.publishing.publish",
    "logging_cid"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
    "created_by"=>"/pulp/api/v3/users/1/",
    "started_at"=>"2024-04-09T14:09:53.181+00:00",
    "finished_at"=>"2024-04-09T14:09:55.293+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/lib/python3.11/site-packages/pulpcore/tasking/tasks.py\", line 61, in _execute_task\n" +
       "    result = func(*args, **kwargs)\n" +
       "             ^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 384, in publish\n" +
       "    generate_repo_metadata(\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 686, in generate_repo_metadata\n" +
       "    checksum_type = cr_checksum_type_from_string(\n" +
       "                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 318, in cr_checksum_type_from_string\n" +
       "    return getattr(cr, checksum_type.upper())\n" +
       "           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "description"=>"module 'createrepo_c' has no attribute 'SHA1'"},
    "worker"=>"/pulp/api/v3/workers/018ec2e7-e086-7e2f-9ff7-7e7f057ffa05/",
    "child_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Generating repository metadata",
       "code"=>"publish.generating_metadata",
       "state"=>"failed",
       "total"=>1,
       "done"=>0}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["shared:/pulp/api/v3/repositories/rpm/rpm/018ec303-a1fd-7ff7-af7b-81104bf6aecb/",
      "shared:/pulp/api/v3/domains/018ec258-d879-75ba-a85e-e6f71108fc5c/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>14, "failed"=>1}}

Exception:

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Backtrace:

/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:108:in `block in check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `each'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:162:in `poll_external_task'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:22:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/cancellable.rb:14:in `run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:10:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:589:in `block (3 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/execute_if_contents_changed.rb:5:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/execution_wrapper.rb:91:in `wrap'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:588:in `block (2 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `block in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `block in with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:582:in `execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:303:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/director.rb:94:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:202:in `execute_job'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:172:in `invoke'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:169:in `block in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:113:in `local'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:44:in `block in <module:Sidekiq>'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:263:in `stats'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:13:in `call'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:80:in `global'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:124:in `block in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:39:in `prepare'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:123:in `dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:168:in `process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:78:in `process_one'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:68:in `run'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:8:in `watchdog'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:17:in `block in safe_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Action:

Actions::Pulp3::Repository::SavePublication

Input:

{"repository_id"=>70,
 "tasks"=>Step(12).output,
 "contents_changed"=>Step(9).output[:contents_changed],
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{}

Actions #2

Updated by Michael Arlt 3 months ago

  • Description updated (diff)
Actions #3

Updated by Michael Arlt 3 months ago

I am not 100% sure but i think that the synchronisation worked with 3.10.0 rc1 or rc2.

Actions #4

Updated by Michael Arlt 3 months ago

Adding pulpcore::allowed_content_checksums: ['sha1', 'sha224', 'sha256', 'sha384', 'sha512']
to custom-hiera.yaml creates

ALLOWED_CONTENT_CHECKSUMS = ["sha1", "sha224", "sha256", "sha384", "sha512"]             

in /etc/pulp/settings.py but the problem persists.

Actions #5

Updated by Samir Jha 3 months ago

  • Status changed from New to Need more information
  • Assignee set to Samir Jha
  • Triaged changed from No to Yes

Hi Michael,

Could you open an issue with pulp team here: https://github.com/pulp/pulp_rpm/issues.

We can keep this issue open if this ends up being a katello issue or a configuration issue.

Actions #7

Updated by Daniel Alley 3 months ago

I believe this is a build issue with the "legacy_hashes" flag not being enabled for the RHEL 9 build of the package https://github.com/rpm-software-management/createrepo_c/blob/master/createrepo_c.spec#L27-L31

I don't believe we've modified those lines from upstream and thus it would likely not be enabled by default. So we should tweak those lines or set that flag.

Actions #9

Updated by Michael Arlt 3 months ago

@Samir: Could you please let me know if and when there will be a solution to the problem?
I really don't want to cause any stress - this is just for my planning.

Actions #10

Updated by Michael Arlt 3 months ago

  • Found in Releases Katello 4.13.0, Katello 4.14.0 added

reproducible with Katello 4.13 nightly:

$ rpm -qa|grep -i katello
katello-repos-4.13-0.1.nightly.el9.noarch
katello-common-4.13.0-0.1.master.el9.noarch
katello-certs-tools-2.10.0-1.el9.noarch
katello-selinux-5.0.2-1.el9.noarch
foreman-installer-katello-3.11.0-0.1.develop.20240418031237gitd83807b.el9.noarch
rubygem-katello-4.13.0-0.1.pre.master.20240411211935git58d2cdb.el9.noarch
katello-4.13.0-0.1.master.el9.noarch
rubygem-hammer_cli_katello-1.12.0-0.1.pre.master.el9.noarch
katello-client-bootstrap-1.7.9-2.el9.noarch

Actions #11

Updated by Michael Arlt 3 months ago

  • Found in Releases deleted (Katello 4.14.0)
Actions #12

Updated by Michael Arlt 3 months ago

  • Triaged changed from Yes to No
Actions #13

Updated by Michael Arlt 3 months ago

  • Status changed from Need more information to Feedback
Actions #14

Updated by Michael Arlt 3 months ago

  • Status changed from Feedback to New
Actions #15

Updated by Samir Jha 3 months ago

Hi Michael,

Looking at Daniel's comment on the pulp issue, this doesn't seem like something we can fix in katello. I'd propose closing this issue and following up on the community?

Actions #16

Updated by Michael Arlt 3 months ago

Hi Samir,

I don't know where we should open the issue but it is not possible to use SHA1 Repos in Foreman 3.10 on RHEL 9.
Please don't close - instead please ask others in the foreman project where it should addressed.

Actions #17

Updated by Ian Ballou 3 months ago

  • Status changed from New to Need more information

This issue cannot be fixed by Pulp or Foreman directly if the cause is due to a built version of createrepo_c that doesn't allow dealing with SHA1 content. I'll leave the issue open for now so we don't forget about it, but the fix will require finding who is in charge of building createrepo_c and seeing if enabling SHA1 via the flag that Daniel mentioned is possible.

Edit: as it turns out, we are packaging createrepo_c, I thought it was coming from the EL9 repos. Changing this into a foreman-packaging issue. To support syncing SHA1 content, we will need to enable that legacy flag in our build of createrepo_c.

Actions #18

Updated by Ian Ballou 3 months ago

  • Project changed from Katello to Packaging
  • Category changed from Sync Plans to RPMs
  • Assignee deleted (Samir Jha)
  • Target version set to 3.11.0
Actions #19

Updated by Ian Ballou 3 months ago

  • Pull request https://github.com/theforeman/pulpcore-packaging/pull/969 added
Actions #20

Updated by Michael Arlt 3 months ago

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

Actions #21

Updated by Odilon Sousa 3 months ago

Michael Arlt wrote in #note-20:

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

It's published now, you can find the pulpcore 3.39 repo.

Actions #22

Updated by Michael Arlt 3 months ago

It's published now, you can find the pulpcore 3.39 repo.

Works! Thank you so much!

Actions #23

Updated by Ian Ballou 3 months ago

  • Status changed from Need more information to Closed

Closing this out now since it sounds like it's all taken care of.

Actions #24

Updated by Griffin Sullivan 2 months ago

  • Fixed in Releases 3.11.0 added
Actions #25

Updated by Ewoud Kohl van Wijngaarden 23 days ago

  • Triaged changed from No to Yes
Actions

Also available in: Atom PDF