Plugins » Salt

Since upgrading from foreman 3.1 to 3.2 and its plugins I'm unable to run host jobs. Upgrading from 3.2 to 3.3 didn't help too. I try to run a salt highstate:

Action:

Actions::RemoteExecution::RunHostsJob

Input:

{"job_invocation"=>
  {"id"=>27,
   "name"=>"Salt",
   "description"=>"Run Salt state.highstate on host"},
 "job_category"=>"Salt",
 "job_invocation_id"=>27,
 "proxy_batch_size"=>100,
 "trigger_run_step_id"=>5,
 "current_request_id"=>"51735317-ef8c-4952-a286-b37f31a59a32",
 "current_timezone"=>"Berlin",
 "current_organization_id"=>2,
 "current_location_id"=>1,
 "current_user_id"=>3}

Output:

{"host_count"=>1,
 "remote_triggered_count"=>0,
 "planned_count"=>1,
 "cancelled_count"=>0,
 "total_count"=>1,
 "failed_count"=>1,
 "pending_count"=>0,
 "success_count"=>0}

Exception:

RuntimeError: A sub task failed

Backtrace:

/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/with_sub_plans.rb:231:in `check_for_errors!'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/with_sub_plans.rb:137:in `try_to_finish'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/with_polling_sub_plans.rb:19:in `poll'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/with_polling_sub_plans.rb:11:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman_remote_execution-7.1.0/app/lib/actions/remote_execution/run_hosts_job.rb:140:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:582:in `block (3 levels) in execute_run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/watch_delegated_proxy_sub_tasks.rb:17:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/activesupport-6.0.4.8/lib/active_support/execution_wrapper.rb:91:in `wrap'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/foreman/vendor/ruby/2.7.0/gems/foreman-tasks-6.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:581:in `block (2 levels) in execute_run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in `catch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in `block in execute_run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `block in with_error_handling'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `catch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `with_error_handling'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:575:in `execute_run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/action.rb:296:in `execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/director.rb:94:in `execute'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/foreman/vendor/ruby/2.7.0/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:192:in `execute_job'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:164:in `block in process'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:109:in `local'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq.rb:37:in `block in <module:Sidekiq>'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:250:in `stats'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/job_logger.rb:8:in `call'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:74:in `global'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:125:in `block in dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:48:in `with_context'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:124:in `dispatch'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:163:in `process'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:83:in `process_one'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:71:in `run'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:16:in `watchdog'
/usr/share/foreman/vendor/ruby/2.7.0/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:25:in `block in safe_thread'
/usr/share/foreman/vendor/ruby/2.7.0/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

foreman_remote_execution Version is: 7.1.0

Unfortunately I don't find any further information in the logs.