Plugins » Katello

I think I'm experiencing the same bahavoir with current stable release (katello 3.0)

virt-who -d output:

2016-06-30 15:57:44,053 ERROR: Communication with server failed:
Traceback (most recent call last):
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 167, in hypervisorCheckIn
    result = self.connection.hypervisorCheckIn(config.owner, config.env, serialized_mapping)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 903, in hypervisorCheckIn
    res = self.conn.request_post(url, host_guest_mapping)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 647, in request_post
    return self._request("POST", method, params)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 564, in _request
    self.validateResponse(result, request_type, handler)
  File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 611, in validateResponse
    raise RestlibException(response['status'], error_msg, response.get('headers'))
RestlibException: Task f7babe9d-4c01-4002-a83b-cdb417f73f19: Katello::Errors::CandlepinError: Runtime Error null
2016-06-30 15:57:44,053 ERROR: Unable to send data: Communication with subscription manager failed with code 500: Task f7babe9d-4c01-4002-a83b-cdb417f73f19: Katello::Errors::CandlepinError: Runtime Error null
2016-06-30 15:57:44,054 ERROR: Sending data failed 5 times, report skipped
2016-06-30 15:57:44,055 DEBUG: Association for config vcenter.tld.com gathered
2016-06-30 15:57:45,057 DEBUG: virt-who shut down started

candlepin Log:

2016-06-30 15:57:34,176 [thread=http-bio-8443-exec-7] [req=f39e484b-c8b8-4595-95bc-9e16ae862400, org=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2016-06-30 15:57:34,178 [thread=http-bio-8443-exec-7] [req=f39e484b-c8b8-4595-95bc-9e16ae862400, org=] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=2
2016-06-30 15:57:34,206 [thread=http-bio-8443-exec-4] [req=cc3fcd55-9227-40e0-bb49-d7851db3f149, org=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2016-06-30 15:57:34,207 [thread=http-bio-8443-exec-4] [req=cc3fcd55-9227-40e0-bb49-d7851db3f149, org=] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=2
2016-06-30 15:57:34,523 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/hypervisors?owner=admin&env=Library
2016-06-30 15:57:34,550 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=ORG] INFO  org.candlepin.resource.HypervisorResource - Hypervisor check-in by principal: {"type":"trusteduser","name":"foreman_admin"}
2016-06-30 15:57:34,561 [thread=http-bio-8443-exec-2] [req=4e432e53-1de2-4535-abc0-c0d29e6863b1, org=ORG] ERROR org.candlepin.common.exceptions.mappers.CandlepinExceptionMapper - Runtime Error null
java.lang.NullPointerException: null

production.log

2016-06-30 16:05:28 [app] [I] Completed 500 Internal Server Error in 1546ms (Views: 0.3ms | ActiveRecord: 9.0ms)
2016-06-30 16:05:28 [app] [I] Started GET "/rhsm/status/" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2016-06-30 16:05:28 [app] [I] Completed 200 OK in 142ms (Views: 0.4ms | ActiveRecord: 0.0ms)
2016-06-30 16:05:28 [app] [I] Started GET "/rhsm/status" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2016-06-30 16:05:28 [app] [I] Completed 200 OK in 139ms (Views: 0.4ms | ActiveRecord: 0.0ms)
2016-06-30 16:05:28 [app] [I] Started POST "/rhsm/hypervisors?owner=admin&env=Library" for 10.165.1.21 at 2016-06-30 16:05:28 +0200
2016-06-30 16:05:28 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#hypervisors_update as JSON
2016-06-30 16:05:28 [app] [I]   Parameters: {"esx001.tld.com"=>[{"guestId"=>"423a5b5c-1df0-84bf-3dad-d0250322db72", "state"=>1, "attributes"=>{"active"=>1, "virtWhoTyp
e"=>"esx", "hypervisorType"=>"vmware"}}, ...
2016-06-30 16:08:38 [foreman-tasks/action] [E] Runtime Error null (Katello::Errors::CandlepinError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in `rescue in propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/propagate_candlepin_errors.rb:9:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/models/katello/concerns/user_extensions.rb:20:in `cp_config'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/remote_action.rb:16:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action/progress.rb:16:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/lib/actions/middleware/keep_locale.rb:11:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:26:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:17:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware.rb:30:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/stack.rb:22:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/middleware/world.rb:30:in `execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:505:in `block (2 levels) in execute_run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in `catch'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:504:in `block in execute_run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `catch'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:419:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:499:in `execute_run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/action.rb:260:in `execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns'
  | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/executors/parallel/worker.rb:12:in `on_message'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.11/lib/dynflow/actor.rb:26:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in `run_task'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in `block (3 levels) in create_worker'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `loop'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in `block (2 levels) in create_worker'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `catch'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in `block in create_worker'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
 2016-06-30 16:08:38 [katello/cp_proxy] [E] ForemanTasks::TaskError: Task b25d1915-d09e-4e5b-abdb-900663603a4e: Katello::Errors::CandlepinError: Runtime Error null
 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:40:in `block in sync_task'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `tap'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks.rb:39:in `sync_task'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.18/lib/foreman_tasks/triggers.rb:22:in `sync_task'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:116:in `block in hypervisors_update'
 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:72:in `as'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:115:in `hypervisors_update'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/base.rb:189:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rendering.rb:10:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:113:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:27:in `repackage_message'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
  | /usr/share/foreman/app/controllers/api/v2/base_controller.rb:152:in `disable_json_root'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rails-observers-0.1.2/lib/rails/observers/action_controller/caching/sweeping.rb:73:in `around'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `public_send'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
 | /usr/share/foreman/app/controllers/concerns/application_shared.rb:13:in `set_timezone'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:229:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:166:in `block in halting'
  | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rails-observers-0.1.2/lib/rails/observers/action_controller/caching/sweeping.rb:73:in `around'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `public_send'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:447:in `block in make_lambda'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:298:in `block in halting'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `run_callbacks'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/callbacks.rb:19:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rescue.rb:29:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications.rb:159:in `block in instrument'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications.rb:159:in `instrument'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.1/app/controllers/katello/concerns/api/api_controller.rb:39:in `process_action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/abstract_controller/base.rb:136:in `process'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/rendering.rb:30:in `process'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal.rb:196:in `dispatch'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_controller/metal.rb:232:in `block in action'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:82:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:50:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:71:in `block in call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `each'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:678:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/engine.rb:514:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `public_send'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `method_missing'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:71:in `block in call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `each'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/journey/router.rb:59:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/routing/route_set.rb:678:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/static_dispatcher.rb:65:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/extractor/recorder.rb:132:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
  | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.3.6/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/conditionalget.rb:35:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:9:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/flash.rb:254:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/cookies.rb:560:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/query_cache.rb:36:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:82:in `run_callbacks'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/rack/logger.rb:38:in `call_app'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/rack/logger.rb:22:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/request_id.rb:21:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/actionpack-4.1.5/lib/action_dispatch/middleware/static.rb:64:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/engine.rb:514:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/application.rb:144:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `public_send'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/railties-4.1.5/lib/rails/railtie.rb:194:in `method_missing'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in `block in call'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `each'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `call'
 | /usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
 | /usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
2016-06-30 16:08:38 [app] [I] Completed 500 Internal Server Error in 1543ms (Views: 0.3ms | ActiveRecord: 9.1ms)

virt-who-0.14-9.el7_2.1.noarch

candlepin-0.9.54.4-1.el7.noarch
candlepin-selinux-0.9.54.4-1.el7.noarch
foreman-installer-katello-3.0.2-1.el7.noarch
katello-3.0.0-9.el7.noarch
katello-agent-2.5.0-3.el7.noarch
katello-ca-consumer-deployc001.srv.muenchen.de-1.0-1.noarch
katello-certs-tools-2.4.0-1.el7.noarch
katello-client-bootstrap-1.1.0-1.el7.noarch
katello-common-3.0.0-9.el7.noarch
katello-debug-3.0.0-9.el7.noarch
katello-default-ca-1.0-1.noarch
katello-installer-base-3.0.2-1.el7.noarch
katello-repos-3.0.0-3.el7.noarch
katello-selinux-3.0.1-1.el7.noarch
katello-server-ca-1.0-1.noarch
katello-service-3.0.0-9.el7.noarch
pulp-katello-1.0-1.el7.noarch
tfm-rubygem-hammer_cli_katello-0.0.25-1.el7.noarch
tfm-rubygem-katello-3.0.1-1.el7.noarch

I am seeing the same thing (katello 3.0), same errors as Klaas in my logs. Happy to supply any logs/details that may help you out just let me know what you need

Dylan

Thomas McKay suspected missing backports of fixes and told me to try latest nightly. To make it short, latest nightly works and content hosts get added correctly (although names have changed but I suspect thats expected behaviour (hostname to virt-who-hostname-1).
Not sure how we should continue now - is this backportable to stable or do we have to wait for the next release to fix this issue?

I manually upgraded candlepin and candlepin-selinux and it works again in katello 3.0

1. rpm -qa|grep candlepin
   candlepin-0.9.54.6-1.el7.noarch
   candlepin-selinux-0.9.54.6-1.el7.noarch

I used the two rpms from the katello nightly repository.