Project

General

Profile

Bug #16236

Cannot publish new content view version, pulp error

Added by Dylan Baars over 5 years ago. Updated almost 4 years ago.

Status:
Rejected
Priority:
Normal
Category:
-
Target version:
Difficulty:
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

When promoting a RHEL content view, I receive the following error:
PLP0000: Pulp exception occurred: PulpExecutionException

Resuming the task doesn't help - it immediately errors again, stuck at 80%

from /var/log/messages, grepping for pulp

Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server_Kickstart-4_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server]
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) Traceback (most recent call last):
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) units=transfer_units)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) return associate_copy_for_repo(unit, dest_repo, True)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) new_unit.safe_import_content(unit._storage_path)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) self.import_content(path, location)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4193-14144) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/b6/86a3f3969741abeff744699c9fdd663406ec955470b70649508652cb82a903/productid.gz failed - must be an existing file.
Aug 23 15:34:52 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [623489f7-c7f0-43ce-bd7c-aebb2e6ff105]
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) Task pulp.server.managers.repo.unit_association.associate_from_repo[623489f7-c7f0-43ce-bd7c-aebb2e6ff105] raised unexpected: PulpExecutionException()
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) Traceback (most recent call last):
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) R = retval = fun(*args, **kwargs)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) return super(Task, self).__call__(*args, **kwargs)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) return super(PulpTask, self).__call__(*args, **kwargs)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) return self.run(*args, **kwargs)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) units=transfer_units)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) return associate_copy_for_repo(unit, dest_repo, True)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) new_unit.safe_import_content(unit._storage_path)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) self.import_content(path, location)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1945-14144) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Aug 23 15:34:53 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[13572917-2ea9-4e91-a5ac-94eb61f7cce4] succeeded in 0.0105422929992s: None

In the dynaflow console, the erroring step is copy/pasted below

35: Actions::Pulp::Repository::CopyYumMetadataFile (error) [ 0.68s / 0.16s ] Skip
Started at: 2016-08-23 03:34:51 UTC

Ended at: 2016-08-23 03:34:52 UTC

Real time: 0.68s

Execution time (excluding suspended state): 0.16s

Input:

---
source_pulp_id: NIWA-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
target_pulp_id: NIWA-RHEL7_Server_Kickstart-4_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
clauses:
remote_user: admin
remote_cp_user: admin
Output:

---
pulp_tasks:
- exception:
task_type: pulp.server.managers.repo.unit_association.associate_from_repo
href: "/pulp/api/v2/tasks/623489f7-c7f0-43ce-bd7c-aebb2e6ff105/"
task_id: 623489f7-c7f0-43ce-bd7c-aebb2e6ff105
tags:
- pulp:repository:NIWA-RHEL7_Server_Kickstart-4_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
- pulp:repository:NIWA-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
- pulp:action:associate
finish_time: '2016-08-23T03:34:52Z'
_ns: task_status
start_time: '2016-08-23T03:34:52Z'
traceback: |
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
return super(Task, self).
_call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
return super(PulpTask, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
return self.run(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
units=transfer_units)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
return associate.associate(source_repo, dest_repo, import_conduit, config, units)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
return associate_copy_for_repo(unit, dest_repo, True)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
new_unit.safe_import_content(unit._storage_path)
File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
self.import_content(path, location)
File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
PulpExecutionException: Pulp exception occurred: PulpExecutionException
spawned_tasks: []
progress_report: {}
queue:
state: error
worker_name: reserved_resource_worker-5@wellkatello
error:
code: PLP0000
data: {}
description: 'Pulp exception occurred: PulpExecutionException'
sub_errors: []
_id:
"$oid": 57bbc45c81ad0ec786b9cad2
id: 57bbc45c81ad0ec786b9cad2
poll_attempts:
total: 1
failed: 1
Error:

Katello::Errors::PulpError

PLP0000: Pulp exception occurred: PulpExecutionException

---
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/pulp/abstract_async_task.rb:121:in
`block in external_task='"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/pulp/abstract_async_task.rb:119:in
`each'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/pulp/abstract_async_task.rb:119:in
`external_task='"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/pulp/repository/abstract_copy_content.rb:40:in
`external_task='"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/polling.rb:98:in
`poll_external_task_with_rescue'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/polling.rb:21:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/cancellable.rb:9:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/pulp/abstract_async_task.rb:45:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:506:in
`block (3 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:30:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/remote_action.rb:16:in
`block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/remote_action.rb:40:in
`block in as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/models/katello/concerns/user_extensions.rb:20:in
`cp_config'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/remote_action.rb:27:in
`as_cp_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/remote_action.rb:39:in
`as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/remote_action.rb:16:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:30:in
`with_progress_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:16:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/keep_locale.rb:11:in
`block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/keep_locale.rb:22:in
`with_locale'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.2/app/lib/actions/middleware/keep_locale.rb:11:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:30:in
`run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/world.rb:30:in
`execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:505:in
`block (2 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in
`catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in
`block in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in
`block in with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in
`catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in
`with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:499:in
`execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:260:in
`execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in
`block (2 levels) in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in
`with_meta_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in
`block in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in
`open_action'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in
`execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:15:in
`block in on_message'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in
`block in assigns'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
`tap'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
`assigns'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in
`match_value'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in
`block in match?'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
`each'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
`match?'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in
`match'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:12:in
`on_message'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/actor.rb:26:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in
`process_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in
`process_envelopes?'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in
`on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in
`process_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in
`block in on_envelope'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in
`block (2 levels) in schedule_execution'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`block in synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`synchronize'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in
`block in schedule_execution'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in
`work'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in
`block in call_job'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in
`run_task'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in
`block (3 levels) in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in
`loop'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in
`block (2 levels) in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in
`catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in
`block in create_worker'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in
`call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in
`block in create_with_logging_context'"

History

#1 Updated by Justin Sherrill over 5 years ago

  • Status changed from New to Need more information

Can you try installing pulp-admin and running this command:

pulp-admin rpm repo content metafile --repo-id PULP_ID

replacing PULP_ID with NIWA-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server

this will print out any metadata files in the repo. Then run:

pulp-admin rpm repo  remove metafile --repo-id=PULP_ID  --str-eq checksum=CHECKSUM

replacing CHECKSUM with the checksum from the prevoius command. And then try resuming the content view publish.

This is due to a pulp bug sadly

#2 Updated by Dylan Baars over 5 years ago

Hi Justin,

I did that and it completed successfully. I then tried resuming the promotion, but got the following:

Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[5a2c0351-9f47-4d67-8f9f-53bb85ca3584]
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[5eb3b1d6-650d-4e1d-afce-a31f30678abe]
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[c3e73370-cd86-4b46-84e9-ce526ddd9ea3]
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[129e823e-cc29-4d69-872a-e491a883c080]
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[5a2c0351-9f47-4d67-8f9f-53bb85ca3584] succeeded in 0.164693548984s: None
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[703a8fb5-8f39-4499-8032-e2f3cac0742b]
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[5eb3b1d6-650d-4e1d-afce-a31f30678abe] succeeded in 0.0561442229955s: None
Sep 8 09:02:26 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[3ffc7a49-fce9-4e4e-998e-4002dcdb2d0b]
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) Exception from importer [yum_importer] while importing units into repository [NIWA-Prod-Server-RHEL7_Server-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_-_Supplementary_RPMs_x86_64_7Server]
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) Traceback (most recent call last):
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Exception from importer [yum_importer] while importing units into repository [NIWA-Prod-Server-RHEL7_Server-Oracle_Java__for_RHEL_Server_-Red_Hat_Enterprise_Linux_7_Server_-_Oracle_Java_RPMs_x86_64_7Server]
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Traceback (most recent call last):
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) units=transfer_units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) units=transfer_units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in _associate_unit
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) new_unit.safe_import_content(unit._storage_path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) self.import_content(path, location)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 905, in safe_import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) self.import_content(path, location)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4869-74496) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/63/383e07b79cd0848c5f598cec27537810ec122229724820bf53bc951a0aebab/productid.gz failed - must be an existing file.
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/3f/07e460b4eaf0910dc60e6c183373c7c61fa7f95626c6cd0888f64ae62d2050/productid.gz failed - must be an existing file.
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [c3e73370-cd86-4b46-84e9-ce526ddd9ea3]
Sep 8 09:02:27 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [703a8fb5-8f39-4499-8032-e2f3cac0742b]
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Task pulp.server.managers.repo.unit_association.associate_from_repo[c3e73370-cd86-4b46-84e9-ce526ddd9ea3] raised unexpected: PulpExecutionException()
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Traceback (most recent call last):
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) R = retval = fun(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(Task, self).__call__(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return self.run(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) units=transfer_units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) Task pulp.server.managers.repo.unit_association.associate_from_repo[703a8fb5-8f39-4499-8032-e2f3cac0742b] raised unexpected: PulpExecutionException()
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) Traceback (most recent call last):
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) self.import_content(path, location)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) R = retval = fun(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) return super(Task, self).__call__(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) return self.run(*args, **kwargs)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) units=transfer_units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) new_unit.safe_import_content(unit._storage_path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) self.import_content(path, location)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1975-74496) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[129e823e-cc29-4d69-872a-e491a883c080] succeeded in 0.0227267339942s: None
Sep 8 09:02:27 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[3ffc7a49-fce9-4e4e-998e-4002dcdb2d0b] succeeded in 0.0186565130134s: None

I ended up skipping tasks using the dynaflow console to cancel/complete the promotion, then tried a new publish - it errors @ 81% with the same errors in the GUI

Abnormal termination (previous state: running)
Abnormal termination (previous state: running)
Abnormal termination (previous state: running)
Abnormal termination (previous state: running)
PLP0000: Pulp exception occurred: PulpExecutionException
PLP0000: Pulp exception occurred: PulpExecutionException

and "cat /var/log/messages | grep pulp" gives the following

Sep 8 10:53:24 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[574e62e2-099d-4628-9fa8-315553912b80]
Sep 8 10:53:24 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[701b4718-0a87-4906-a673-a6939f2c54b9]
Sep 8 10:53:24 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[d2e46036-8bfd-4fc7-b1fb-e9eb669b635e]
Sep 8 10:53:25 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[45bb25a0-2f04-4d20-81a8-7125f4832488]
Sep 8 10:53:25 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[574e62e2-099d-4628-9fa8-315553912b80] succeeded in 0.146299279004s: None
Sep 8 10:53:25 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[11526250-a45c-45a6-9f09-49aa2a15b717]
Sep 8 10:53:25 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[701b4718-0a87-4906-a673-a6939f2c54b9] succeeded in 0.0725653019908s: None
Sep 8 10:53:25 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[bc21f1e5-8a37-480c-87d8-67fbd4cf0908]
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_-_Supplementary_RPMs_x86_64_7Server]
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Oracle_Java__for_RHEL_Server_-Red_Hat_Enterprise_Linux_7_Server_-_Oracle_Java_RPMs_x86_64_7Server]
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Traceback (most recent call last):
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Traceback (most recent call last):
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) units=transfer_units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) units=transfer_units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) self.import_content(path, location)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/63/383e07b79cd0848c5f598cec27537810ec122229724820bf53bc951a0aebab/productid.gz failed - must be an existing file.
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) self.import_content(path, location)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/3f/07e460b4eaf0910dc60e6c183373c7c61fa7f95626c6cd0888f64ae62d2050/productid.gz failed - must be an existing file.
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [d2e46036-8bfd-4fc7-b1fb-e9eb669b635e]
Sep 8 10:53:26 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [11526250-a45c-45a6-9f09-49aa2a15b717]
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Task pulp.server.managers.repo.unit_association.associate_from_repo[d2e46036-8bfd-4fc7-b1fb-e9eb669b635e] raised unexpected: PulpExecutionException()
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Traceback (most recent call last):
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) R = retval = fun(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(Task, self).__call__(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return self.run(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) units=transfer_units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) self.import_content(path, location)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Task pulp.server.managers.repo.unit_association.associate_from_repo[11526250-a45c-45a6-9f09-49aa2a15b717] raised unexpected: PulpExecutionException()
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Traceback (most recent call last):
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) R = retval = fun(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(Task, self).__call__(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return self.run(*args, **kwargs)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) units=transfer_units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) self.import_content(path, location)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[45bb25a0-2f04-4d20-81a8-7125f4832488] succeeded in 0.017629820999s: None
Sep 8 10:53:26 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[bc21f1e5-8a37-480c-87d8-67fbd4cf0908] succeeded in 0.0192259759933s: None
Sep 8 10:53:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f2b2c54e-cbee-4289-b4b5-bfe5e3675db1]
Sep 8 10:53:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[66d78357-b730-4dfc-b714-37e06c21674a]
Sep 8 10:53:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[d749f621-4f29-4fc6-b147-ec5de1359e59]
Sep 8 10:53:28 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f2b2c54e-cbee-4289-b4b5-bfe5e3675db1] succeeded in 0.042024740018s: None
Sep 8 10:53:31 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[c6ff238a-76d8-4f6f-bbad-8c343b27f5a4]
Sep 8 10:53:31 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[e25eb9fa-f7d3-4695-88e8-6f7f41927e8a]
Sep 8 10:53:31 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[0f0be623-798e-4754-94a0-e2bab6aea4bb]
Sep 8 10:53:31 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[c6ff238a-76d8-4f6f-bbad-8c343b27f5a4] succeeded in 0.047805307986s: None
Sep 8 10:54:07 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[767245ec-fad2-4138-bac1-e8dfaac46dca]
Sep 8 10:54:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[b56ab6f8-6898-41a3-911b-33ce35682616]
Sep 8 10:54:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c97a1fce-ce85-44e5-89f4-508511718c50]
Sep 8 10:54:08 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[767245ec-fad2-4138-bac1-e8dfaac46dca] succeeded in 0.861468365998s: None
Sep 8 10:54:15 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[b56ab6f8-6898-41a3-911b-33ce35682616] succeeded in 6.76449821901s: {}
Sep 8 10:54:15 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[c97a1fce-ce85-44e5-89f4-508511718c50] succeeded in 0.101093503996s: None
Sep 8 10:54:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[b3a90b08-08e8-4a80-885c-4122786030a0]
Sep 8 10:54:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[360bd69f-081e-471d-9ace-c98ec0b809a8]
Sep 8 10:54:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[ca5662dd-4116-4500-beab-e0ffa7eb5a2d]
Sep 8 10:54:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[b3a90b08-08e8-4a80-885c-4122786030a0] succeeded in 0.0972960079962s: None
Sep 8 10:54:38 wellkatello.niwa.local pulp: pulp.plugins.pulp_katello.distributors.yum_clone_distributor.distributor:INFO: Start publish time 1473288878.32
Sep 8 10:55:29 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[360bd69f-081e-471d-9ace-c98ec0b809a8] succeeded in 51.484811642s: {'exception': None, 'repo_id': u'NIWA-RHEL7_Server-38_0-EPEL7-EPEL7_x86_64', 'traceback': None, 'started':...
Sep 8 10:55:29 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ca5662dd-4116-4500-beab-e0ffa7eb5a2d] succeeded in 0.0129727140011s: None
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[66d78357-b730-4dfc-b714-37e06c21674a] succeeded in 126.628453008s: {'units_successful': [{'unit_key': {'name': u'bash-doc', 'checksum': u'2080d42a45a52fce50561bdf140bfa998deec786', 'epoch':...
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[d749f621-4f29-4fc6-b147-ec5de1359e59] succeeded in 0.0132555550081s: None
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f888a405-5153-4ae2-88e8-b02172f9f865]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[d5123fe7-d3b2-4f24-aeed-a328143c50d8]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[62554146-b2da-4446-8722-0c5da02d09d5]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[b54c6c5e-d14a-45fb-9bc4-3cf471071a49]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[8f2ffba2-a18d-4ea3-911d-0ff7f07ff4ed]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[f638ca19-2e45-4900-b92a-041165498c22]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f888a405-5153-4ae2-88e8-b02172f9f865] succeeded in 0.0736431789992s: None
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[49de8ea1-d252-4b61-a295-1ef15d6305a4]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[7b77b543-717c-436a-85df-d2e215bb163d]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[1e9224a8-47b7-4fca-9594-d105e474e557]
Sep 8 10:55:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[d5123fe7-d3b2-4f24-aeed-a328143c50d8] succeeded in 0.0459754579933s: None
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[f56467eb-fb3a-4e0b-a8df-99c50773ec8d]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[270295cc-21e9-4ef1-b855-ba28d3936b45]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[62554146-b2da-4446-8722-0c5da02d09d5] succeeded in 0.050469748996s: None
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[4827003d-e9bd-42d7-b00f-9fa45cd89e81]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[dcb8d9f5-ce68-49d0-93f4-02f36921985a]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[8f2ffba2-a18d-4ea3-911d-0ff7f07ff4ed] succeeded in 0.0546836199937s: None
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.unassociate_by_criteria[f1f387fc-0bff-4675-acb9-a8f426ce4482]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[e9d17def-dd74-4ee1-8b06-c68ceb496cdc]
Sep 8 10:55:36 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[49de8ea1-d252-4b61-a295-1ef15d6305a4] succeeded in 0.0405820939923s: None
Sep 8 10:55:54 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[edc7a968-fc3d-4588-89d4-35d1ea55e96a]
Sep 8 10:55:54 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[127129cf-38f0-4581-8acf-18af556bb48a]
Sep 8 10:55:54 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c1f10b60-e84e-419b-b8d7-e7eb2ee58ae4]
Sep 8 10:55:54 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[edc7a968-fc3d-4588-89d4-35d1ea55e96a] succeeded in 0.144787079014s: None
Sep 8 10:56:25 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[e25eb9fa-f7d3-4695-88e8-6f7f41927e8a] succeeded in 173.956768844s: {'units_successful': [{'unit_key': {'name': u'gtk2-devel', 'checksum': u'a248ff9cbaf75975c9edfeb2ea9afa4f9ef44b6a', 'epoch':...
Sep 8 10:56:25 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[0f0be623-798e-4754-94a0-e2bab6aea4bb] succeeded in 0.0621961670113s: None
Sep 8 10:56:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[127129cf-38f0-4581-8acf-18af556bb48a] succeeded in 43.38713461s: {'units_successful': [{'unit_key': {'id': u'RHSA-2015:0750'}, 'type_id': u'erratum'}, {'unit_key': {'id': u'RHBA-2015:0571'},...
Sep 8 10:56:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[c1f10b60-e84e-419b-b8d7-e7eb2ee58ae4] succeeded in 0.0177942959999s: None
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[942603cc-64f8-405b-90bf-7ce7bcdb450d]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[13184454-9fc7-433b-86f1-79f42db04a7d]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[942603cc-64f8-405b-90bf-7ce7bcdb450d] succeeded in 0.0524683679978s: None
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[a039acbb-aae4-4486-92dd-99b10e0b6119]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[a34ff51f-69cf-412c-9985-f46ce031c7a0]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[d4d5a664-ed96-44d5-b58b-cbf42e493017]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[816a08a6-083f-4d8a-964c-750b4ec505b2]
Sep 8 10:56:41 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[a34ff51f-69cf-412c-9985-f46ce031c7a0] succeeded in 0.109481453983s: None
Sep 8 10:56:43 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[13184454-9fc7-433b-86f1-79f42db04a7d] succeeded in 2.42906220403s: {'units_successful': [{'unit_key': {'repo_id':...
Sep 8 10:56:43 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[a039acbb-aae4-4486-92dd-99b10e0b6119] succeeded in 0.0146584959875s: None
Sep 8 10:56:45 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[5e22d39e-fa69-4ea6-90e7-f4d57fb72c30]
Sep 8 10:56:45 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[baf6b0d2-09cd-4c2a-9dc9-5f9a8087c3b3]
Sep 8 10:56:45 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[2f395661-1adc-486e-8fff-aea59fcfee59]
Sep 8 10:56:45 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[5e22d39e-fa69-4ea6-90e7-f4d57fb72c30] succeeded in 0.0543049300031s: None
Sep 8 10:56:45 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[baf6b0d2-09cd-4c2a-9dc9-5f9a8087c3b3] succeeded in 0.511203121016s: {'units_successful': []}
Sep 8 10:56:46 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[2f395661-1adc-486e-8fff-aea59fcfee59] succeeded in 0.0343226199911s: None
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[1c7f571d-a61f-466c-be54-970eadd765ed]
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[56ff3072-a676-4bd7-8df2-af12c8cf168b]
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[ce2fb698-e05b-49b6-99b5-421d0c2bb7ae]
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[1c7f571d-a61f-466c-be54-970eadd765ed] succeeded in 0.139945103001s: None
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[56ff3072-a676-4bd7-8df2-af12c8cf168b] succeeded in 0.417085174005s: {'units_successful': []}
Sep 8 10:56:47 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ce2fb698-e05b-49b6-99b5-421d0c2bb7ae] succeeded in 0.0270795639954s: None
Sep 8 10:57:50 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[d4d5a664-ed96-44d5-b58b-cbf42e493017] succeeded in 68.49646863s: {'units_successful': [{'unit_key': {'id': u'RHSA-2015:0750'}, 'type_id': u'erratum'}, {'unit_key': {'id': u'RHSA-2015:0809'},...
Sep 8 10:57:50 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[816a08a6-083f-4d8a-964c-750b4ec505b2] succeeded in 0.129913523007s: None
Sep 8 10:57:50 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[ca99b144-c9ce-4b02-b57d-625a74476021]
Sep 8 10:57:51 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[2347f524-739a-4764-8a6d-7dfb2b516e90]
Sep 8 10:57:51 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[4830f42b-719a-465e-bcbe-7ba0eb6a8eb3]
Sep 8 10:57:51 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[ca99b144-c9ce-4b02-b57d-625a74476021] succeeded in 0.275352558994s: None
Sep 8 10:58:07 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[5fad0830-5ad8-4a41-addc-7bcf87adce8b]
Sep 8 10:58:07 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[8888ceda-f0cb-4f0c-a5c0-029800444f69]
Sep 8 10:58:07 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[5fad0830-5ad8-4a41-addc-7bcf87adce8b] succeeded in 0.127984193998s: None
Sep 8 10:58:07 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[0799c376-eaa5-4f28-95d8-12b1bf22283f]
Sep 8 10:58:07 wellkatello.niwa.local pulp: pulp.plugins.pulp_katello.distributors.yum_clone_distributor.distributor:INFO: Start publish time 1473289087.68
Sep 8 10:58:08 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[2347f524-739a-4764-8a6d-7dfb2b516e90] succeeded in 16.78391742s: {'units_successful': [{'unit_key': {'repo_id':...
Sep 8 10:58:08 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[4830f42b-719a-465e-bcbe-7ba0eb6a8eb3] succeeded in 0.0144910110103s: None
Sep 8 10:58:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f3aee75f-94b9-48ae-ae32-03af5c0a47fc]
Sep 8 10:58:09 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[1dde4d34-a0ef-49e7-a0e4-da3f0d4a2fe2]
Sep 8 10:58:09 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[f64839ff-c6a7-4148-9bc5-ea3101cc8416]
Sep 8 10:58:09 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f3aee75f-94b9-48ae-ae32-03af5c0a47fc] succeeded in 0.0488242589927s: None
Sep 8 10:58:09 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[1dde4d34-a0ef-49e7-a0e4-da3f0d4a2fe2] succeeded in 0.335719447001s: {'units_successful': []}
Sep 8 10:58:09 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f64839ff-c6a7-4148-9bc5-ea3101cc8416] succeeded in 0.0226005370205s: None
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[27fa3e51-52b7-4db1-8ae4-fa4c19245071]
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[68656db5-9c81-4d9d-88bd-00551eab9355]
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[27fa3e51-52b7-4db1-8ae4-fa4c19245071] succeeded in 0.0739029820252s: None
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[53d33587-5707-4bbc-b9e6-0f50c458c9ab]
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[68656db5-9c81-4d9d-88bd-00551eab9355] succeeded in 0.203018465021s: {'units_successful': []}
Sep 8 10:58:15 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[53d33587-5707-4bbc-b9e6-0f50c458c9ab] succeeded in 0.0187846399785s: None
Sep 8 10:58:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[8888ceda-f0cb-4f0c-a5c0-029800444f69] succeeded in 49.595732146s: {'exception': None, 'repo_id':...
Sep 8 10:58:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[0799c376-eaa5-4f28-95d8-12b1bf22283f] succeeded in 0.117050561996s: None
Sep 8 10:59:05 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[73cb8404-3e2a-4f04-a519-837f3b62a13b]
Sep 8 10:59:05 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[455ec810-6018-4fe9-aa88-0f7b9d7ed04e]
Sep 8 10:59:05 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[73cb8404-3e2a-4f04-a519-837f3b62a13b] succeeded in 0.0813978049846s: None
Sep 8 10:59:05 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[22e898f2-4cc9-45cb-97c7-f07ffa317d71]
Sep 8 10:59:05 wellkatello.niwa.local pulp: pulp.plugins.pulp_katello.distributors.yum_clone_distributor.distributor:INFO: Start publish time 1473289145.55
Sep 8 10:59:32 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[455ec810-6018-4fe9-aa88-0f7b9d7ed04e] succeeded in 27.334196631s: {'exception': None, 'repo_id':...
Sep 8 10:59:32 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[22e898f2-4cc9-45cb-97c7-f07ffa317d71] succeeded in 0.144605631009s: None
Sep 8 10:59:37 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 10:59:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[10446e84-e6dc-4e87-aba0-3667e8c24e18]
Sep 8 11:01:31 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[b54c6c5e-d14a-45fb-9bc4-3cf471071a49] succeeded in 355.505229575s: {'units_successful': [{'unit_key': {'name': u'openblas-threads64_', 'checksum':...
Sep 8 11:01:31 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f638ca19-2e45-4900-b92a-041165498c22] succeeded in 0.052406561008s: None
Sep 8 11:01:31 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[7b77b543-717c-436a-85df-d2e215bb163d] succeeded in 0.22501918301s: {}
Sep 8 11:01:31 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[1e9224a8-47b7-4fca-9594-d105e474e557] succeeded in 0.02146963001s: None
Sep 8 11:01:55 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[f56467eb-fb3a-4e0b-a8df-99c50773ec8d] succeeded in 23.974074165s: {'units_successful': [{'unit_key': {'id': u'FEDORA-EPEL-2015-7915'}, 'type_id': u'erratum'}, {'unit_key': {'id':...
Sep 8 11:01:55 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[270295cc-21e9-4ef1-b855-ba28d3936b45] succeeded in 0.0138396040129s: None
Sep 8 11:01:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[4827003d-e9bd-42d7-b00f-9fa45cd89e81] succeeded in 1.953165702s: {'units_successful': [{'unit_key': {'repo_id': u'NIWA-Library-RHEL7_Server-EPEL7-EPEL7_x86_64', 'id': u'urdu-support'},...
Sep 8 11:01:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[dcb8d9f5-ce68-49d0-93f4-02f36921985a] succeeded in 0.0178056329896s: None
Sep 8 11:01:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.unassociate_by_criteria[f1f387fc-0bff-4675-acb9-a8f426ce4482] succeeded in 0.0232274440059s: {}
Sep 8 11:01:57 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[e9d17def-dd74-4ee1-8b06-c68ceb496cdc] succeeded in 0.0186770019936s: None
Sep 8 11:02:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[b467be0b-dab3-48a5-a0da-8d6a936864c0]
Sep 8 11:02:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[699ce844-8ad8-42ae-bfa1-7c84cd9a5a5c]
Sep 8 11:02:08 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[85992821-b4d9-47a2-ba3c-bc823f2c5ff9]
Sep 8 11:02:08 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[b467be0b-dab3-48a5-a0da-8d6a936864c0] succeeded in 0.0951077309728s: None
Sep 8 11:06:37 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[699ce844-8ad8-42ae-bfa1-7c84cd9a5a5c] succeeded in 269.244588079s: {'units_successful': [{'unit_key': {'name': u'kf5-kidletime-devel', 'checksum':...
Sep 8 11:06:37 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[85992821-b4d9-47a2-ba3c-bc823f2c5ff9] succeeded in 0.0635264040029s: None
Sep 8 11:06:53 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[1ebcd1d0-f247-420c-a141-76e527a2edea]
Sep 8 11:06:53 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[f89cf697-f836-4038-a8b4-5016fe297ad5]
Sep 8 11:06:53 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[1ebcd1d0-f247-420c-a141-76e527a2edea] succeeded in 0.140439137002s: None
Sep 8 11:06:53 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[538daacd-fe42-4ab9-8a48-5f2fbf0c0366]
Sep 8 11:07:43 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[f89cf697-f836-4038-a8b4-5016fe297ad5] succeeded in 50.156583518s: {'units_successful': [{'unit_key': {'id': u'FEDORA-EPEL-2015-02bd65205b'}, 'type_id': u'erratum'}, {'unit_key': {'id':...
Sep 8 11:07:44 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[538daacd-fe42-4ab9-8a48-5f2fbf0c0366] succeeded in 0.0144047749927s: None
Sep 8 11:07:48 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[b1af1556-70d8-4c13-9a2a-a3ed6ad3794e]
Sep 8 11:07:48 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[5d1af99d-30e7-4d7a-9d19-37b976b5c1e8]
Sep 8 11:07:48 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[3c06310b-26d9-4f51-bd3a-5b0cfb1038b7]
Sep 8 11:07:48 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[b1af1556-70d8-4c13-9a2a-a3ed6ad3794e] succeeded in 0.0458398750052s: None
Sep 8 11:07:59 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[5d1af99d-30e7-4d7a-9d19-37b976b5c1e8] succeeded in 10.826253671s: {'units_successful': [{'unit_key': {'repo_id': u'NIWA-Library-RHEL7_Server-EPEL7-EPEL7_x86_64', 'id': u'portuguese-support'},...
Sep 8 11:07:59 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[3c06310b-26d9-4f51-bd3a-5b0cfb1038b7] succeeded in 0.0189374179754s: None
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[576ed61e-2df8-4a9c-be01-24cb8bb1d93a]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[2ab752bc-d217-4d19-ab5f-496fae7d097a]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[fa0958b3-cc82-4f03-a90f-970b60a0660e]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[576ed61e-2df8-4a9c-be01-24cb8bb1d93a] succeeded in 0.0398729440058s: None
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[2ab752bc-d217-4d19-ab5f-496fae7d097a] succeeded in 0.229311425996s: {'units_successful': []}
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[fa0958b3-cc82-4f03-a90f-970b60a0660e] succeeded in 0.0156515289855s: None
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[7f1d481a-041d-4ed7-830d-4eff241c87b7]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[e855e1e0-bedb-472f-bbe1-cd859e9f82ee]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[35025dd8-3b7c-41b5-95ff-f640e22e4c72]
Sep 8 11:08:01 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[7f1d481a-041d-4ed7-830d-4eff241c87b7] succeeded in 0.0520689660043s: None
Sep 8 11:08:02 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.unit_association.associate_from_repo[e855e1e0-bedb-472f-bbe1-cd859e9f82ee] succeeded in 0.272148850985s: {'units_successful': []}
Sep 8 11:08:02 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[35025dd8-3b7c-41b5-95ff-f640e22e4c72] succeeded in 0.0254619189946s: None
Sep 8 11:09:37 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:09:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[1a784115-d283-4203-ad09-9c495f7e6aa5]
Sep 8 11:09:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[9c83cf4e-95f8-4f88-8734-dbf299bed495]
Sep 8 11:09:37 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[1a784115-d283-4203-ad09-9c495f7e6aa5] succeeded in 0.228168186004s: None
Sep 8 11:09:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[9c83cf4e-95f8-4f88-8734-dbf299bed495] succeeded in 0.545683681994s: None
Sep 8 11:10:17 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[2da331cf-dc7b-4bce-86bf-cf3069ed2827]
Sep 8 11:10:17 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[89b1f83f-24f8-45c6-a7bb-9fb12566dc8d]
Sep 8 11:10:17 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[f2257769-2fbb-4ddd-91be-52d3edacc6a1]
Sep 8 11:10:17 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[2da331cf-dc7b-4bce-86bf-cf3069ed2827] succeeded in 0.148560326023s: None
Sep 8 11:10:18 wellkatello.niwa.local pulp: pulp.plugins.pulp_katello.distributors.yum_clone_distributor.distributor:INFO: Start publish time 1473289818.04
Sep 8 11:11:20 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[89b1f83f-24f8-45c6-a7bb-9fb12566dc8d] succeeded in 62.996837719s: {'exception': None, 'repo_id': u'NIWA-Library-RHEL7_Server-EPEL7-EPEL7_x86_64', 'traceback': None, 'started':...
Sep 8 11:11:21 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f2257769-2fbb-4ddd-91be-52d3edacc6a1] succeeded in 0.0992966120248s: None
Sep 8 11:12:11 wellkatello.niwa.local pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[06842c3e-aae9-4009-985b-9003d1c8d45f]
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[c385328a-0113-4355-b8be-c469507279b5]
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[06842c3e-aae9-4009-985b-9003d1c8d45f] succeeded in 0.0485560729867s: None
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[d06d36a7-778c-436e-bc53-c46ee0bf7b97]
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[c385328a-0113-4355-b8be-c469507279b5] succeeded in 0.174754285981s: None
Sep 8 11:12:11 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[d06d36a7-778c-436e-bc53-c46ee0bf7b97] succeeded in 0.0130736190185s: None
Sep 8 11:18:55 wellkatello.niwa.local pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Sep 8 11:18:55 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[fb12c297-083f-4c58-8129-adcb62ecce35]
Sep 8 11:18:56 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[3d72a663-6299-4611-9310-9de4bd32b3df]
Sep 8 11:18:56 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[ceba55f0-51e9-452e-a692-06fc8ed79de3]
Sep 8 11:18:56 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[fb12c297-083f-4c58-8129-adcb62ecce35] succeeded in 0.0356784009782s: None
Sep 8 11:18:56 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[3d72a663-6299-4611-9310-9de4bd32b3df] succeeded in 0.183780814987s: None
Sep 8 11:18:56 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ceba55f0-51e9-452e-a692-06fc8ed79de3] succeeded in 0.0112175999966s: None
Sep 8 11:19:37 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:19:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[fa94e350-5fa7-4f0e-99f9-727e3c1baba0]
Sep 8 11:19:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[0a419dde-2bfa-40a6-82d0-788cc6c04bd5]
Sep 8 11:19:37 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[fa94e350-5fa7-4f0e-99f9-727e3c1baba0] succeeded in 0.059766780003s: None
Sep 8 11:29:37 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:29:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[8cba5d4c-1f5e-4535-abd4-e90ddb44b3c3]
Sep 8 11:39:37 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:39:37 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[df06d343-1ec3-42e9-913c-21f108ceef89]
Sep 8 11:49:38 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:49:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[0523769b-80a8-44df-922e-cdf22cf38b7f]
Sep 8 11:49:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[739cee2f-fe8f-4a7d-be29-41626236abcb]
Sep 8 11:49:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[0523769b-80a8-44df-922e-cdf22cf38b7f] succeeded in 0.212480821006s: None
Sep 8 11:49:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[739cee2f-fe8f-4a7d-be29-41626236abcb] succeeded in 0.475297120021s: None
Sep 8 11:59:38 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 11:59:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[28fa3749-83d6-4bad-82b1-2f282c478962]
Sep 8 11:59:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[a0eeaad9-7353-4e50-910b-51ae59dc62cf]
Sep 8 11:59:38 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[28fa3749-83d6-4bad-82b1-2f282c478962] succeeded in 0.336193786992s: None
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[292ff01a-6bcd-4446-8fba-c3eee5d63ca2]
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f066244e-cb13-4ad0-a279-e29d3a582b73]
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[7519a9ff-6519-41e8-a7c0-59e6efdbcacb]
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[459f3ab1-5ffb-4722-bbb5-8f3076612aaa]
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[292ff01a-6bcd-4446-8fba-c3eee5d63ca2] succeeded in 0.335648933018s: None
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[7d2716b8-1a74-4aaf-999f-94fc30589d8e]
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f066244e-cb13-4ad0-a279-e29d3a582b73] succeeded in 0.032532407s: None
Sep 8 12:06:28 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[660c8a4c-571d-4690-ac60-5302c134cc27]
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Oracle_Java__for_RHEL_Server_-Red_Hat_Enterprise_Linux_7_Server_-_Oracle_Java_RPMs_x86_64_7Server]
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_-_Supplementary_RPMs_x86_64_7Server]
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Traceback (most recent call last):
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Traceback (most recent call last):
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) units=transfer_units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) units=transfer_units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in _associate_unit
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) self.import_content(path, location)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/3f/07e460b4eaf0910dc60e6c183373c7c61fa7f95626c6cd0888f64ae62d2050/productid.gz failed - must be an existing file.
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 905, in safe_import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) self.import_content(path, location)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/63/383e07b79cd0848c5f598cec27537810ec122229724820bf53bc951a0aebab/productid.gz failed - must be an existing file.
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [7519a9ff-6519-41e8-a7c0-59e6efdbcacb]
Sep 8 12:06:30 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [7d2716b8-1a74-4aaf-999f-94fc30589d8e]
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Task pulp.server.managers.repo.unit_association.associate_from_repo[7519a9ff-6519-41e8-a7c0-59e6efdbcacb] raised unexpected: PulpExecutionException()
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Traceback (most recent call last):
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) R = retval = fun(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(Task, self).__call__(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return self.run(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) units=transfer_units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) self.import_content(path, location)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Task pulp.server.managers.repo.unit_association.associate_from_repo[7d2716b8-1a74-4aaf-999f-94fc30589d8e] raised unexpected: PulpExecutionException()
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Traceback (most recent call last):
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) R = retval = fun(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(Task, self).__call__(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return self.run(*args, **kwargs)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) units=transfer_units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) self.import_content(path, location)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[459f3ab1-5ffb-4722-bbb5-8f3076612aaa] succeeded in 0.0172819209984s: None
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 12:06:30 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[660c8a4c-571d-4690-ac60-5302c134cc27] succeeded in 0.0200169169984s: None
Sep 8 12:09:38 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 12:09:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[434b56d0-bfa1-4892-b3c2-3d17eee0162b]
Sep 8 12:19:38 wellkatello.niwa.local pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Sep 8 12:19:38 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[4509e1f1-4aa0-4c0e-9851-855c3cfd9dab]
Sep 8 12:21:34 wellkatello.niwa.local pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[5beaa5ea-e88a-4ee0-8c17-5e24ea1ccba7]
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[77852834-cdb7-4e10-ab9b-62357bda49f1]
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[5beaa5ea-e88a-4ee0-8c17-5e24ea1ccba7] succeeded in 0.206822028995s: None
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[0681e2e3-ffb7-470e-8ae4-70dda1d1530a]
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[77852834-cdb7-4e10-ab9b-62357bda49f1] succeeded in 0.252903002984s: None
Sep 8 12:21:35 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[0681e2e3-ffb7-470e-8ae4-70dda1d1530a] succeeded in 0.0732807430031s: None
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[667daf55-175d-4bd1-8a30-4e088781f2be]
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[acbcd14b-133e-4dc7-987e-dbd168594e52]
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[2090ddd3-e217-4d34-943d-ec688f36932d]
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[667daf55-175d-4bd1-8a30-4e088781f2be] succeeded in 0.079722319002s: None
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[6cb4590c-800f-45a6-90c0-61c007c123f9]
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[14c3c244-e86c-4143-9c4c-faa01aadcf63]
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[acbcd14b-133e-4dc7-987e-dbd168594e52] succeeded in 0.0412728080119s: None
Sep 8 12:21:49 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[1990640b-85b6-4eed-a76b-a6534e4c7e6a]
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_-_Supplementary_RPMs_x86_64_7Server]
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Exception from importer [yum_importer] while importing units into repository [NIWA-RHEL7_Server-38_0-Oracle_Java__for_RHEL_Server_-Red_Hat_Enterprise_Linux_7_Server_-_Oracle_Java_RPMs_x86_64_7Server]
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) Traceback (most recent call last):
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) Traceback (most recent call last):
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) units=transfer_units)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) units=transfer_units)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in _associate_unit
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) self.import_content(path, location)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 905, in safe_import_content
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) self.import_content(path, location)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4868-25184) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/3f/07e460b4eaf0910dc60e6c183373c7c61fa7f95626c6cd0888f64ae62d2050/productid.gz failed - must be an existing file.
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.managers.repo.unit_association:ERROR: (4881-70784) PulpCodedException: Content import of /var/lib/pulp/content/units/yum_repo_metadata_file/63/383e07b79cd0848c5f598cec27537810ec122229724820bf53bc951a0aebab/productid.gz failed - must be an existing file.
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [14c3c244-e86c-4143-9c4c-faa01aadcf63]
Sep 8 12:21:50 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [2090ddd3-e217-4d34-943d-ec688f36932d]
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Task pulp.server.managers.repo.unit_association.associate_from_repo[2090ddd3-e217-4d34-943d-ec688f36932d] raised unexpected: PulpExecutionException()
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) Traceback (most recent call last):
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) R = retval = fun(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(Task, self).__call__(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return self.run(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) units=transfer_units)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Task pulp.server.managers.repo.unit_association.associate_from_repo[14c3c244-e86c-4143-9c4c-faa01aadcf63] raised unexpected: PulpExecutionException()
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) Traceback (most recent call last):
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) R = retval = fun(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(Task, self).
_call__(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 905, in safe_import_content
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return super(PulpTask, self).__call__(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) self.import_content(path, location)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return self.run(*args, **kwargs)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (1961-70784) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) units=transfer_units)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 57, in import_units
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 53, in associate
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) associated_units = set([_associate_unit(dest_repo, unit) for unit in units])
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 333, in associate_unit
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) return associate_copy_for_repo(unit, dest_repo, True)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 373, in associate_copy_for_repo
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) new_unit.safe_import_content(unit._storage_path)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/
_init__.py", line 905, in safe_import_content
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) self.import_content(path, location)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 877, in import_content
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) raise exceptions.PulpCodedException(error_code=error_codes.PLP0037, path=path)
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:ERROR: (2001-25184) PulpExecutionException: Pulp exception occurred: PulpExecutionException
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[6cb4590c-800f-45a6-90c0-61c007c123f9] succeeded in 0.0143542540027s: None
Sep 8 12:21:51 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[1990640b-85b6-4eed-a76b-a6534e4c7e6a] succeeded in 0.017355215

#3 Updated by Justin Sherrill over 5 years ago

Dylan,

I think this is happening for multiple repositories.

Can you repeat this process and each time you see a failed step in the dynflow console, click on it and look for:
```
source_pulp_id: NIWA-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
target_pulp_id: NIWA-RHEL7_Server_Kickstart-4_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server_RPMs_x86_64_7Server
```

In this case the pulp id you want to run the commands against is the 'source_pulp_id' in the above output.

A simpler command you can run is below, just replace REPO_ID with the source_pulp_id from each step that has failed.

```
pulp-admin rpm repo remove metafile --repo-id=REPO_ID --match='repo_id=.*'
```

#4 Updated by Justin Sherrill over 5 years ago

related pulp issue: https://pulp.plan.io/issues/2248

a second person saw this.

#5 Updated by Dylan Baars over 5 years ago

Hi Justin,

thanks for that - you are correct! There were a couple more repos I had to run the command on,

pulp-admin u admin -p $password rpm repo remove metafile --repo-id=NIWA-Oracle_Java__for_RHEL_Server_-Red_Hat_Enterprise_Linux_7_Server_Oracle_Java_RPMs_x86_64_7Server --match='repo_id=.*'
pulp-admin u admin -p $password rpm repo remove metafile --repo-id=NIWA-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_7_Server
_Supplementary_RPMs_x86_64_7Server --match='repo_id=.*'

I've now noticed all my RHEL repos are not able to sync - the error in the GUI is RPM1004: Error retrieving metadata: Not found and from /var/log/messages I see

Sep 13 14:30:58 wellkatello.niwa.local pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Sep 13 14:31:00 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f284f6e7-215a-493e-9bc6-bf3d1a433ec8]
Sep 13 14:31:00 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[981a01c6-fd95-42d4-8d6f-e60470bf0fe9]
Sep 13 14:31:00 wellkatello.niwa.local pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[9dd3b103-26d7-4c8f-9ced-9dc785ca8f03]
Sep 13 14:31:00 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f284f6e7-215a-493e-9bc6-bf3d1a433ec8] succeeded in 0.125369669s: None
Sep 13 14:31:01 wellkatello.niwa.local systemd: Started Session 134 of user root.
Sep 13 14:31:01 wellkatello.niwa.local systemd: Starting Session 134 of user root.
Sep 13 14:31:01 wellkatello.niwa.local pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/fastrack/rhel/server/7/x86_64/optional/os/.
Sep 13 14:31:01 wellkatello.niwa.local pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Sep 13 14:31:03 wellkatello.niwa.local pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Sep 13 14:31:03 wellkatello.niwa.local pulp: pulp.server.async.tasks:INFO: Task failed : [981a01c6-fd95-42d4-8d6f-e60470bf0fe9] : Error retrieving metadata: Not found
Sep 13 14:31:04 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.sync.sync[981a01c6-fd95-42d4-8d6f-e60470bf0fe9] raised expected: PulpCodedException()
Sep 13 14:31:04 wellkatello.niwa.local pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[9dd3b103-26d7-4c8f-9ced-9dc785ca8f03] succeeded in 0.0117141319997s: None

I've seen various posts about this happening when manifests are re-imported (I haven't done that) and when the server is behind a proxy (we're not) ... any ideas? This is probably unrelated to this particular issue I guess!

#6 Updated by Eric Helms over 5 years ago

  • Assignee set to Justin Sherrill

#7 Updated by Justin Sherrill over 5 years ago

That 'Not found' error often means 'Forbidden' which indicates either:

a) an proxy in the way

b) your manifest is out of date

Most likely its b) if other external repos are syncing. You can try clicking the 'refresh' button on the manfiest import page to see if that helps. You mean need to redownload it from the customer support portal.

#8 Updated by Justin Sherrill over 5 years ago

  • Legacy Backlogs Release (now unused) set to 166

#9 Updated by Justin Sherrill over 5 years ago

  • Status changed from Need more information to Rejected

Closing due to inactivity, please feel free to reopen this issue and respond here.

Also available in: Atom PDF