Plugins » Katello

Foreman 2.5.2 / Katello 4.1.2

New smart proxy from scratch has the issue.
A working smart proxy which I just upgraded has now the issue.

Actions:

Actions::Pulp3::ContentGuard::Refresh

Exception:

RuntimeError: There was an issue with the backend service pulp3: Not all necessary pulp workers running at https://<smart-proxy>/pulp/api/v3.

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/middleware/backend_services_check.rb:17:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/middleware/backend_services_check.rb:15:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/middleware/backend_services_check.rb:15:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/middleware/remote_action.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/load_setting_values.rb:15:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:34:in `with_current_request_id'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:31:in `with_current_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:30:in `with_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:41:in `with_current_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:513:in `block (2 levels) in execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:384:in `switch_flow'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:417:in `concurrence'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:512:in `block in execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:511:in `execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:285:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/plan_step.rb:55:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/plan_step.rb:54:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:445:in `plan_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/katello/capsule_content/sync.rb:42:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:384:in `switch_flow'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:422:in `sequence'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/lib/actions/katello/capsule_content/sync.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:514:in `block (3 levels) in execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/load_setting_values.rb:15:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:34:in `with_current_request_id'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:31:in `with_current_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:30:in `with_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:36:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:10:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:41:in `with_current_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:9:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:513:in `block (2 levels) in execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:384:in `switch_flow'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:417:in `concurrence'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:512:in `block in execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:511:in `execute_plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:285:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/plan_step.rb:55:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/plan_step.rb:54:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:286:in `block (2 levels) in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:384:in `switch_flow'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:374:in `with_planning_scope'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:285:in `block in plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/common/transaction.rb:17:in `block in rollback_on_error'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/transaction_adapters/active_record.rb:6:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/common/transaction.rb:16:in `rollback_on_error'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/common/transaction.rb:6:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:44:in `plan_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan.rb:284:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:211:in `block (2 levels) in plan_with_options'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/coordinator.rb:326:in `acquire'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:209:in `block in plan_with_options'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:208:in `tap'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:208:in `plan_with_options'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:204:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/world.rb:180:in `trigger'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:23:in `trigger'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:29:in `block in trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:49:in `block in rails_safe_trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:48:in `rails_safe_trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:27:in `trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks.rb:54:in `async_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.2/lib/foreman_tasks/triggers.rb:18:in `async_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/controllers/katello/api/v2/capsule_content_controller.rb:58:in `sync'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:195:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:30:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:139:in `run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:41:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rescue.rb:22:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `block in instrument'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `instrument'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/app/controllers/katello/concerns/api/api_controller.rb:50:in `process_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:136:in `process'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:39:in `process'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:190:in `dispatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:254:in `dispatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:33:in `serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:48:in `serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in `block in serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:48:in `serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in `block in serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `serve'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/lib/katello/middleware/organization_created_enforcer.rb:18:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/lib/katello/middleware/event_daemon.rb:10:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.4.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:40:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/cookies.rb:648:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:101:in `run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:37:in `call_app'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:28:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0/lib/katello/prevent_json_parsing.rb:12:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/executor.rb:14:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
/opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/configuration.rb:249:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:77:in `block in handle_request'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:338:in `with_force_shutdown'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:76:in `handle_request'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/server.rb:438:in `process_client'
/opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:145:in `block in spawn_thread'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Hey Shubhra,

We noticed that you're running Katello 4.1.0. The latest 4.1.2 contains a large Pulp update which may alleviate your problem. I recommend upgrading your main server and any proxies as soon as possible and checking if the error goes away. There was a similar thread in our community recently: https://community.theforeman.org/t/smart-proxy-with-2-5-2-4-1-2-not-all-necessary-pulp-workers-running/24707/10

Restarting the pulp workers may also work: https://community.theforeman.org/t/smart-proxy-with-2-5-2-4-1-2-not-all-necessary-pulp-workers-running/24707/4
If none of this works for you, please open a new discussion in the Support area of our community - it's a much better way to get help than filing against our issue tracker! https://community.theforeman.org/c/support/10

Jonathon