Project

General

Profile

Actions
Copy link

Bug #21621

closed

remote execution failed for some hosts if executed for multiple hosts

Added by Lukas Plattner over 7 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Smart Proxy - SSH
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

When I'm executing a job to a host collecection (Filter 'host_collection ~ "mycollection"'; collection consists of 5 hosts) I always get following errors for 2 Hosts:

Error initializing command: SocketError - getaddrinfo: Name or service not known
Exit status: EXCEPTION

To exclude some kind of DNS Problems, I've added all the hosts to /etc/hosts --> still the same. Executing the same job for every hosts one by one works as it should.

[root@capsule ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)
[root@capsule ~]# rpm -qa | grep foreman
foreman-release-scl-3-1.el7.noarch
foreman-selinux-1.15.6-1.el7.noarch
foreman-proxy-1.15.6-1.el7.noarch
foreman-installer-1.15.6-2.el7.noarch
capsule.example.at-foreman-proxy-1.0-1.noarch
katello-ca-consumer-foreman.apa.at-1.0-1.noarch
foreman-proxy-content-3.4.5-1.el7.noarch
capsule.example.at-foreman-proxy-client-1.0-1.noarch
tfm-rubygem-foreman_remote_execution_core-1.0.6-1.fm1_15.el7.noarch
foreman-release-1.15.6-1.el7.noarch
foreman-installer-katello-3.4.5-1.el7.noarch
tfm-rubygem-foreman-tasks-core-0.1.8-1.fm1_15.el7.noarch
foreman-debug-1.15.6-1.el7.noarch

Errortrace from Smartproxy:

[2017-11-09 17:27:42.063 #26948] ERROR -- dynflow: Error initializing command - SocketError getaddrinfo: Name or service not known:
/opt/rh/rh-ruby22/root/usr/share/ruby/socket.rb:232:in `getaddrinfo'
/opt/rh/rh-ruby22/root/usr/share/ruby/socket.rb:232:in `foreach'
/opt/rh/rh-ruby22/root/usr/share/ruby/socket.rb:328:in `tcp'
/opt/theforeman/tfm/root/usr/share/gems/gems/net-ssh-4.0.1/lib/net/ssh/transport/session.rb:70:in `initialize'
/opt/theforeman/tfm/root/usr/share/gems/gems/net-ssh-4.0.1/lib/net/ssh.rb:233:in `new'
/opt/theforeman/tfm/root/usr/share/gems/gems/net-ssh-4.0.1/lib/net/ssh.rb:233:in `start'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:134:in `session'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:191:in `run_sync'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:288:in `ensure_remote_directory'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:265:in `upload_data'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:261:in `cp_script_to_remote'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:48:in `prepare_start'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution_core-1.0.6/lib/foreman_remote_execution_core/script_runner.rb:34:in `start'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.1.8/lib/foreman_tasks_core/runner/dispatcher.rb:32:in `start_runner'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/actor.rb:6:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.1.8/lib/foreman_tasks_core/runner/dispatcher.rb:24:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.30/lib/dynflow/actor.rb:26:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:161:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'

Actions
Copy link
 #1

Updated by Ivan Necas over 7 years ago

This is strange indeed. I've not seen this issue before, so I guess it is something about the environment. I wish I could be more helpful.

Actions
Copy link
 #2

Updated by Lukas Plattner over 7 years ago

Any Idea how to fix that?

Actions
Copy link
 #3

Updated by Adam Ruzicka over 7 years ago

You mentioned the jobs pass if you run them against the hosts one by one. If you take a look at the tasks, there should be a ProxyAction action. Is there any difference in the ProxyAction's input/output when run on a single host and when run using a collection containing the host?

Actions
Copy link
 #4

Updated by Lukas Plattner over 7 years ago

In both cases the action is "Actions::RemoteExecution::RunHostsJob". Also the input is the same (without the id and job_invocation_id which is ok). Output seems to make sense, as only the counts deferre. What I saw for a short time, was that the Output was "{}" for a short time at the multiple host request. I'm not sure this is correct.

Actions
Copy link
 #5

Updated by Adam Ruzicka about 4 years ago

  • Status changed from New to Closed

Last movement here was 3 years ago, closing.

Actions
Copy link

Also available in: Atom PDF