Project

General

Profile

Actions

Bug #9211

closed

Cannot sync EPEL 7 repo

Added by Duncan Innes about 9 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Errata Management
Target version:
Difficulty:
Triaged:
Yes
Fixed in Releases:
Found in Releases:

Description

Fresh install of Katello 2.1 on CentOS 7. Cannot sync down the EPEL 7 repo.

Looks like it's this bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1161277

Error message given is:

{"state"=>"FAILED",
"error"=>"insertDocument :: caused by :: 17280 Btree::insert: key too large to
index, failing pulp_database.units_erratum.$title_1 1408 { :
\"globus-common-15.26-1.el7,globus-gsi-proxy-ssl-5.7-1.el7,globus-gsi-openssl-error-3.5-1.el7,globus-openssl-module-4.6-1.el7,globus-gsi-cert-utils-9.10...\"
}"},

Actions #1

Updated by Eric Helms about 9 years ago

  • Category set to Errata Management
  • Status changed from New to Need more information
  • translation missing: en.field_release changed from 14 to 33
  • Triaged changed from No to Yes

Can you try reproducing with the latest 2.1 and letting us know if this has been fixed?

Actions #2

Updated by Duncan Innes about 9 years ago

I've just tried it again twice for good measure. OS re-install, Katello 2.1 install, create EPEL product and repo, sync. Result shows the same error message in the Output of the pulp task. I'll past the output for you if it's helpful.

Actions #3

Updated by Duncan Innes about 9 years ago

Action:
Actions::Pulp::Repository::Sync

Input: {"pulp_id"=>"Default_Organization-EPEL-EPEL",
"task_id"=>nil,
"remote_user"=>"admin-6087addb",
"remote_cp_user"=>"admin",
"locale"=>"en-GB"}

Output: {"pulp_tasks"=>
[{"exception"=>nil,
"task_type"=>"pulp.server.tasks.repository.sync_with_auto_publish",
"_href"=>"/pulp/api/v2/tasks/372d62bc-ce6e-433d-b5c8-ccb2d15de774/",
"task_id"=>"372d62bc-ce6e-433d-b5c8-ccb2d15de774",
"tags"=>
["pulp:repository:Default_Organization-EPEL-EPEL", "pulp:action:sync"],
"finish_time"=>"2015-02-07T15:11:35Z",
"_ns"=>"task_status",
"start_time"=>"2015-02-07T15:10:24Z",
"traceback"=>
"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 315, in call\n return super(Task, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in protected_call\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/tasks/repository.py\", line 210, in sync_with_auto_publish\n sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides)\n File \"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py\", line 113, in sync\n raise PulpExecutionException(_('Importer indicated a failed response'))\nPulpExecutionException: Importer indicated a failed response\n",
"spawned_tasks"=>[],
"progress_report"=> {"yum_importer"=> {"content"=> {"size_total"=>0,
"items_left"=>0,
"items_total"=>0,
"state"=>"FINISHED",
"size_left"=>0,
"details"=> {"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0},
"error_details"=>[]},
"comps"=>{"state"=>"NOT_STARTED"},
"distribution"=> {"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"items_left"=>0},
"errata"=> {"state"=>"FAILED",
"error"=>
"insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1408 { : \"globus-common-15.26-1.el7,globus-gsi-proxy-ssl-5.7-1.el7,globus-gsi-openssl-error-3.5-1.el7,globus-openssl-module-4.6-1.el7,globus-gsi-cert-utils-9.10...\" }"},
"metadata"=>{"state"=>"FINISHED"}}},
"queue"=>"",
"state"=>"error",
"result"=>nil,
"error"=> {"code"=>"PLP0000",
"data"=>{},
"description"=>"Importer indicated a failed response",
"sub_errors"=>[]},
"_id"=>{"$oid"=>"54d62ae005f360d8661e251f"},
"id"=>"54d62ae0fca1631a2926aa25"}],
"poll_attempts"=>{"total"=>27, "failed"=>3}}

Exception:
Katello::Errors::PulpError: PLP0000: Importer indicated a failed response

Backtrace:
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:100:in `block in external_task='
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:98:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:98:in `external_task='
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:86:in `poll_external_task_with_rescue'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:15:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/cancellable.rb:9:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/pulp/abstract_async_task.rb:57:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:443:in `block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:27:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:57:in `block (2 levels) in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/katello/util/thread_session.rb:84:in `pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:43:in `as_pulp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:56:in `block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/katello/util/thread_session.rb:91:in `cp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:38:in `as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:55:in `as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/remote_action.rb:27:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:16:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in `pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:23:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:34:in `with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.1.0/app/lib/actions/middleware/keep_locale.rb:23:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:442:in `block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in `block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in `with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:436:in `execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:230:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in `with_meta_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:20:in `block in on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in `block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `tap'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in `assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in `match_value'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in `block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `each'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in `match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:17:in `on_message'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:82:in `on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:72:in `receive'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `loop'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `catch'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in `run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:13:in `block in initialize'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'

Actions #4

Updated by Stephen Benjamin about 9 years ago

  • Status changed from Need more information to New

See https://bugzilla.redhat.com/show_bug.cgi?id=1175616#c5 for a work around, it will be fixed in Pulp 2.6 (Katello 2.2).

Actions #5

Updated by Duncan Innes about 9 years ago

As EPEL is an important external repo for likely Katello users, would it be possible/practical/reasonable to code this workaround into katello-installer? Is there a 'workarounds' module in the installer?

I've got no issue applying a manual fix myself, but the 2.1 release might seem more complete if it doesn't require manual workarounds to sync one of the more important RHEL family external repos.

Actions #6

Updated by Duncan Innes about 9 years ago

I can confirm that the index removal in mongo does provide a workaround.
(Just in case there's any doubt that is).
Now to fit it into my install script somewhere.

Actions #7

Updated by Eric Helms about 9 years ago

  • translation missing: en.field_release changed from 33 to 23
Actions #8

Updated by Eric Helms about 9 years ago

  • Target version set to 67
Actions #9

Updated by Eric Helms about 9 years ago

  • Target version changed from 67 to 68
Actions #10

Updated by Eric Helms almost 9 years ago

  • Status changed from New to Resolved

This fix should be available in 2.2 via the inclusion of Pulp 2.6. If you still see the issue, please re-open.

Actions #11

Updated by Duncan Innes almost 9 years ago

Checked against 2.2 RC2. Works fine now. Thanks.

Actions

Also available in: Atom PDF