Bug #9211: Cannot sync EPEL 7 repo - Katello - Foreman

Custom queries

All issues closed in the last 3 weeks
Assigned to me
Closed Issues without Release
Easy and Trivial Issues
Good first issues - Foreman core
Katello 4.15.1 TODO
Mine
My issues closed in the last 3 weeks
open docker issues 2.1
Open Features
Open Katello issues by target version
Open Pull Requests
Stale
Trackers
Untriaged
Untriaged and opened in the past two weeks

Actions

Copy link

Bug #9211

closed

Cannot sync EPEL 7 repo

Added by Duncan Innes almost 10 years ago. Updated over 6 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Category:

Errata Management

Target version:

Katello 2.2

Difficulty:

Triaged:

Yes

Bugzilla link:

Pull request:

Fixed in Releases:

Found in Releases:

Red Hat JIRA:

Description

Fresh install of Katello 2.1 on CentOS 7. Cannot sync down the EPEL 7 repo.

Looks like it's this bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1161277

Error message given is:

{"state"=>"FAILED",
"error"=>"insertDocument :: caused by :: 17280 Btree::insert: key too large to
index, failing pulp_database.units_erratum.$title_1 1408 { :
\"globus-common-15.26-1.el7,globus-gsi-proxy-ssl-5.7-1.el7,globus-gsi-openssl-error-3.5-1.el7,globus-openssl-module-4.6-1.el7,globus-gsi-cert-utils-9.10...\"
}"},

History
Notes
Property changes

Actions

Copy link

Updated by Eric Helms almost 10 years ago

Category set to Errata Management
Status changed from New to Need more information
Translation missing: en.field_release changed from 14 to 33
Triaged changed from No to Yes

Can you try reproducing with the latest 2.1 and letting us know if this has been fixed?

Actions

Copy link

Updated by Duncan Innes almost 10 years ago

I've just tried it again twice for good measure. OS re-install, Katello 2.1 install, create EPEL product and repo, sync. Result shows the same error message in the Output of the pulp task. I'll past the output for you if it's helpful.

Actions

Copy link

Updated by Duncan Innes almost 10 years ago

Action:
Actions::Pulp::Repository::Sync

Input: {"pulp_id"=>"Default_Organization-EPEL-EPEL",
"task_id"=>nil,
"remote_user"=>"admin-6087addb",
"remote_cp_user"=>"admin",
"locale"=>"en-GB"}

Output: {"pulp_tasks"=>
[{"exception"=>nil,
"task_type"=>"pulp.server.tasks.repository.sync_with_auto_publish",
"_href"=>"/pulp/api/v2/tasks/372d62bc-ce6e-433d-b5c8-ccb2d15de774/",
"task_id"=>"372d62bc-ce6e-433d-b5c8-ccb2d15de774",
"tags"=>
["pulp:repository:Default_Organization-EPEL-EPEL", "pulp:action:sync"],
"finish_time"=>"2015-02-07T15:11:35Z",
"_ns"=>"task_status",
"start_time"=>"2015-02-07T15:10:24Z",
"traceback"=>
"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 315, in call\n return super(Task, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in protected_call\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/tasks/repository.py\", line 210, in sync_with_auto_publish\n sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides)\n File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py\", line 113, in sync\n raise PulpExecutionException(_('Importer indicated a failed response'))\nPulpExecutionException: Importer indicated a failed response\n",
"spawned_tasks"=>[],
"progress_report"=> {"yum_importer"=> {"content"=> {"size_total"=>0,
"items_left"=>0,
"items_total"=>0,
"state"=>"FINISHED",
"size_left"=>0,
"details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0},
"error_details"=>[]},
"comps"=>{"state"=>"NOT_STARTED"},
"distribution"=> {"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"items_left"=>0},
"errata"=> {"state"=>"FAILED",
"error"=>
"insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1408 { : \"globus-common-15.26-1.el7,globus-gsi-proxy-ssl-5.7-1.el7,globus-gsi-openssl-error-3.5-1.el7,globus-openssl-module-4.6-1.el7,globus-gsi-cert-utils-9.10...\" }"},
"metadata"=>{"state"=>"FINISHED"}}},
"queue"=>"reserved_resource_worker-0@foreman.rhel7.local.dq",
"state"=>"error",
"result"=>nil,
"error"=> {"code"=>"PLP0000",
"data"=>{},
"description"=>"Importer indicated a failed response",
"sub_errors"=>[]},
"_id"=>{"$oid"=>"54d62ae005f360d8661e251f"},
"id"=>"54d62ae0fca1631a2926aa25"}],
"poll_attempts"=>{"total"=>27, "failed"=>3}}

Exception:
Katello::Errors::PulpError: PLP0000: Importer indicated a failed response

Backtrace:
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:100:in `block in external_task='
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:98:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:98:in `external_task='
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:86:in `poll_external_task_with_rescue'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:15:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/cancellable.rb:9:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:57:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:443:in `block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:27:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:57:in `block (2 levels) in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/katello/util/thread_session.rb:84:in `pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:43:in `as_pulp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:56:in `block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/katello/util/thread_session.rb:91:in `cp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:38:in `as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:55:in `as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:27:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:16:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:23:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:34:in `with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:23:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:442:in `block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:436:in `execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:230:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `with_meta_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:17:in `on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:82:in `on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:72:in `receive'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `loop'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:13:in `block in initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'

Actions

Copy link

Updated by Stephen Benjamin almost 10 years ago

Status changed from Need more information to New

See https://bugzilla.redhat.com/show_bug.cgi?id=1175616#c5 for a work around, it will be fixed in Pulp 2.6 (Katello 2.2).

Actions

Copy link

Updated by Duncan Innes almost 10 years ago

As EPEL is an important external repo for likely Katello users, would it be possible/practical/reasonable to code this workaround into katello-installer? Is there a 'workarounds' module in the installer?

I've got no issue applying a manual fix myself, but the 2.1 release might seem more complete if it doesn't require manual workarounds to sync one of the more important RHEL family external repos.

Actions

Copy link

Updated by Duncan Innes almost 10 years ago

I can confirm that the index removal in mongo does provide a workaround.
(Just in case there's any doubt that is).
Now to fit it into my install script somewhere.

Actions

Copy link