Bug #14601
closed
virt-who hypervisor check in failing
Description
virt-who sends vmware hypervisor information to Katello, but the task fails with "Runtime Error null at org.candlepin.resource.HypervisorResource.hypervisorCheckIn:137"
#Trackeback from /var/log/rhsm/rhsm.log:
2016-04-12 21:29:23,049 [ERROR] @virtwho.py:123 - Error in communication with subscription manager: Traceback (most recent call last): File "/usr/share/virt-who/virtwho.py", line 107, in send self._sendGuestAssociation(report) File "/usr/share/virt-who/virtwho.py", line 134, in _sendGuestAssociation result = manager.hypervisorCheckIn(report.config, report.association, report.config.type) File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 152, in hypervisorCheckIn return self.connection.hypervisorCheckIn(config.owner, config.env, serialized_mapping) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 903, in hypervisorCheckIn res = self.conn.request_post(url, host_guest_mapping) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 647, in request_post return self._request("POST", method, params) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 611, in validateResponse raise RestlibException(response['status'], error_msg, response.get('headers')) RestlibException: Task 273eec3c-ae92-4f8c-b8d2-3e69c6007aa3: Katello::Errors::CandlepinError: Runtime Error null at org.candlepin.resource.HypervisorResource.hypervisorCheckIn:137
Traceback from foreman task:
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:inrescue in propagate_candlepin_errors'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in propagate_candlepin_errors' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:9:inrun'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/remote_action.rb:16:inblock in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/remote_action.rb:40:in block in as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/models/katello/concerns/user_extensions.rb:20:incp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/remote_action.rb:27:in as_cp_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/remote_action.rb:39:inas_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/remote_action.rb:16:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:incall'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:30:in with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action/progress.rb:16:inrun'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/keep_locale.rb:11:inblock in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/keep_locale.rb:22:in with_locale' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.1.0/app/lib/actions/middleware/keep_locale.rb:11:inrun'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:26:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:17:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware.rb:30:inrun'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/middleware/world.rb:30:inexecute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:505:in block (2 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:incatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:504:in block in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:incall'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:incatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:419:in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:499:inexecute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/action.rb:260:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:inblock (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract.rb:155:inwith_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:inopen_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:15:inblock in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in block in assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:intap'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in assigns' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:inmatch_value'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in block in match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:ineach'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in match?' /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:inmatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/executors/parallel/worker.rb:12:in on_message' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:inon_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.10/lib/dynflow/actor.rb:26:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:inprocess_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:inpass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:inprocess_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in block in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:inblock (2 levels) in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/synchronization/mri_lockable_object.rb:38:in block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/synchronization/mri_lockable_object.rb:38:insynchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/synchronization/mri_lockable_object.rb:38:in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:inblock in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/serialized_execution.rb:18:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/serialized_execution.rb:18:incall'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/serialized_execution.rb:96:in work' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/serialized_execution.rb:77:inblock in call_job'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:inrun_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in block (3 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:inloop'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in block (2 levels) in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:incatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in block in create_worker' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:incall'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in block in create_with_logging_context'
/etc/sysconfig/virt-who:
`
VIRTWHO_BACKGROUND=1
VIRTWHO_DEBUG=0
VIRTWHO_SATELLITE6=0
VIRTWHO_ESX=0
VIRTWHO_ESX_OWNER=6972987
VIRTWHO_ESX_ENV=Library
VIRTWHO_ESX_SERVER=domain.org
VIRTWHO_ESX_USERNAME=DOMAIN\VCentersvcaccount
VIRTWHO_ESX_PASSWORD=secretpassword
`
#/etc/virt-who.d/vsphere.conf:
`
[vsphere]
type=esx ; insert one of libvirt/esx/hyperv/rhevm/vdsm/fake
server=vsphere.domain.org ; insert hostname or ip address of the server to connect to
username=DOMAIN\VCentersvcaccount ; username for server authentication
password=ultrasecretpassword ; password for server authentication
owner=admin ; owner for use with SAM, Customer Portal, or Satellite 6
env=Library ; environment for use with SAM, Customer Portal, or Satellite 6
hypervisor_id=hostname ; how will be the hypervisor identified, one of: uuid, hostname, hwuuid
`
Updated by Dominic Cleal over 8 years ago
- Project changed from Foreman to Katello
- Category deleted (
Reporting)
Updated by Eric Helms over 8 years ago
- Category set to 78
- Assignee set to Thomas McKay
- Translation missing: en.field_release set to 144
Updated by Eric Helms over 8 years ago
- Translation missing: en.field_release changed from 144 to 168
Updated by Eric Helms over 8 years ago
- Translation missing: en.field_release deleted (
168)
Updated by Klaas D over 8 years ago
I think I'm experiencing the same bahavoir with current stable release (katello 3.0)
virt-who -d output:
2016-06-30 15:57:44,053 ERROR: Communication with server failed:
Traceback (most recent call last):
File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 167, in hypervisorCheckIn
result = self.connection.hypervisorCheckIn(config.owner, config.env, serialized_mapping)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 903, in hypervisorCheckIn
res = self.conn.request_post(url, host_guest_mapping)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 647, in request_post
return self._request("POST", method, params)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
self.validateResponse(result, request_type, handler)
File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 611, in validateResponse
raise RestlibException(response['status'], error_msg, response.get('headers'))
RestlibException: Task f7babe9d-4c01-4002-a83b-cdb417f73f19: Katello::Errors::CandlepinError: Runtime Error null
2016-06-30 15:57:44,053 ERROR: Unable to send data: Communication with subscription manager failed with code 500: Task f7babe9d-4c01-4002-a83b-cdb417f73f19: Katello::Errors::CandlepinError: Runtime Error null
2016-06-30 15:57:44,054 ERROR: Sending data failed 5 times, report skipped
2016-06-30 15:57:44,055 DEBUG: Association for config vcenter.tld.com gathered
2016-06-30 15:57:45,057 DEBUG: virt-who shut down started
candlepin Log:
2016-06-30 15:57:34,176 [thread=http-bio-8443-exec-7] [req=f39e484b-c8b8-4595-95bc-9e16ae862400, org=] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2016-06-30 15:57:34,178 [thread=http-bio-8443-exec-7] [req=f39e484b-c8b8-4595-95bc-9e16ae862400, org=] INFO org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=2
2016-06-30 15:57:34,206 [thread=http-bio-8443-exec-4] [req=cc3fcd55-9227-40e0-bb49-d7851db3f149, org=] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2016-06-30 15:57:34,207 [thread=http-bio-8443-exec-4] [req=cc3fcd55-9227-40e0-bb49-d7851db3f149, org=] INFO org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=2
2016-06-30 15:57:34,523 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/hypervisors?owner=admin&env=Library
2016-06-30 15:57:34,550 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=ORG] INFO org.candlepin.resource.HypervisorResource - Hypervisor check-in by principal: {"type":"trusteduser","name":"foreman_admin"}
2016-06-30 15:57:34,561 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=ORG] ERROR org.candlepin.common.exceptions.mappers.CandlepinExceptionMapper - Runtime Error null
java.lang.NullPointerException: null
production.log
2016-06-30 16:05:28 [app] [I] Completed 500 Internal Server Error in 1546ms (Views: 0.3ms | ActiveRecord: 9.0ms)
2016-06-30 16:05:28 [app] [I] Started GET "/rhsm/status/" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2016-06-30 16:05:28 [app] [I] Completed 200 OK in 142ms (Views: 0.4ms | ActiveRecord: 0.0ms)
2016-06-30 16:05:28 [app] [I] Started GET "/rhsm/status" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2016-06-30 16:05:28 [app] [I] Completed 200 OK in 139ms (Views: 0.4ms | ActiveRecord: 0.0ms)
2016-06-30 16:05:28 [app] [I] Started POST "/rhsm/hypervisors?owner=admin&env=Library" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#hypervisors_update as JSON
2016-06-30 16:05:28 [app] [I] Parameters: {"esx001.tld.com"=>[{"guestId"=>"423a5b5c-1df0-84bf-3dad-d0250322db72", "state"=>1, "attributes"=>{"active"=>1, "virtWhoTyp
e"=>"esx", "hypervisorType"=>"vmware"}}, ...
2016-06-30 16:08:38 [foreman-tasks/action] [E] Runtime Error null (Katello::Errors::CandlepinError)
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in `rescue in propagate_candlepin_errors'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:9:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/models/katello/concerns/user_extensions.rb:20:in `cp_config'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:16:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:16:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:11:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:30:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/world.rb:30:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:505:in `block (2 levels) in execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in `catch'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in `block in execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `block in with_error_handling'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `catch'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `with_error_handling'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:499:in `execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:260:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?'
| /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:12:in `on_message'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/actor.rb:26:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch'
| /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker'
| /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
2016-06-30 16:08:38 [katello/cp_proxy] [E] ForemanTasks::TaskError: Task b25d1915-d09e-4e5b-abdb-900663603a4e: Katello::Errors::CandlepinError: Runtime Error null
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:40:in `block in sync_task'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `tap'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `sync_task'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks/triggers.rb:22:in `sync_task'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:116:in `block in hypervisors_update'
| /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:72:in `as'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:115:in `hypervisors_update'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/base.rb:189:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rendering.rb:10:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:27:in `repackage_message'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /usr/share/foreman/app/controllers/api/v2/base_controller.rb:152:in `disable_json_root'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rails-observers-0.1.2/lib/rails/observers/action_controller/caching/sweeping.rb:73:in `around'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `public_send'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /usr/share/foreman/app/controllers/concerns/application_shared.rb:13:in `set_timezone'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rails-observers-0.1.2/lib/rails/observers/action_controller/caching/sweeping.rb:73:in `around'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `public_send'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `block in make_lambda'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `run_callbacks'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/callbacks.rb:19:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rescue.rb:29:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications.rb:159:in `block in instrument'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications.rb:159:in `instrument'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/concerns/api/api_controller.rb:39:in `process_action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/base.rb:136:in `process'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/rendering.rb:30:in `process'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal.rb:196:in `dispatch'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal.rb:232:in `block in action'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:82:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:50:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:71:in `block in call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `each'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:678:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/engine.rb:514:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `public_send'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `method_missing'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:71:in `block in call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `each'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:678:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/static_dispatcher.rb:65:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/extractor/recorder.rb:132:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/conditionalget.rb:35:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
| /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:9:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/flash.rb:254:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/cookies.rb:560:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/query_cache.rb:36:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:82:in `run_callbacks'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/rack/logger.rb:38:in `call_app'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/rack/logger.rb:22:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/request_id.rb:21:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/engine.rb:514:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/application.rb:144:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `public_send'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `method_missing'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in `block in call'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `each'
| /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `call'
| /usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
| /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
| /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
| /usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
| /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
2016-06-30 16:08:38 [app] [I] Completed 500 Internal Server Error in 1543ms (Views: 0.3ms | ActiveRecord: 9.1ms)
virt-who-0.14-9.el7_2.1.noarch
candlepin-0.9.54.4-1.el7.noarch
candlepin-selinux-0.9.54.4-1.el7.noarch
foreman-installer-katello-3.0.2-1.el7.noarch
katello-3.0.0-9.el7.noarch
katello-agent-2.5.0-3.el7.noarch
katello-ca-consumer-deployc001.srv.muenchen.de-1.0-1.noarch
katello-certs-tools-2.4.0-1.el7.noarch
katello-client-bootstrap-1.1.0-1.el7.noarch
katello-common-3.0.0-9.el7.noarch
katello-debug-3.0.0-9.el7.noarch
katello-default-ca-1.0-1.noarch
katello-installer-base-3.0.2-1.el7.noarch
katello-repos-3.0.0-3.el7.noarch
katello-selinux-3.0.1-1.el7.noarch
katello-server-ca-1.0-1.noarch
katello-service-3.0.0-9.el7.noarch
pulp-katello-1.0-1.el7.noarch
tfm-rubygem-hammer_cli_katello-0.0.25-1.el7.noarch
tfm-rubygem-katello-3.0.1-1.el7.noarch
Updated by Dylan Baars over 8 years ago
I am seeing the same thing (katello 3.0), same errors as Klaas in my logs. Happy to supply any logs/details that may help you out just let me know what you need
Dylan
Updated by Klaas D over 8 years ago
Thomas McKay suspected missing backports of fixes and told me to try latest nightly. To make it short, latest nightly works and content hosts get added correctly (although names have changed but I suspect thats expected behaviour (hostname to virt-who-hostname-1).
Not sure how we should continue now - is this backportable to stable or do we have to wait for the next release to fix this issue?
Updated by Eric Helms over 8 years ago
- Status changed from New to Resolved
- Translation missing: en.field_release set to 168
Klaas reports that candlepin-0.9.54.6 fixes this issue, so I am moving this into 3.0.2 to tag in this newer version of Candlepin to be released and resolve this issue.
Updated by Klaas D over 8 years ago
I manually upgraded candlepin and candlepin-selinux and it works again in katello 3.0
- rpm -qa|grep candlepin
candlepin-0.9.54.6-1.el7.noarch
candlepin-selinux-0.9.54.6-1.el7.noarch
I used the two rpms from the katello nightly repository.