Bug #33348
closedNoticed "event_queue_error: type=delete_host_agent_queue, object_id=XX" error logging during concurrent host build/rebuild/re-registration/deletion in Satellite 6.10
Description
Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1997225
Description of problem:
While doing certain host build/rebuild/re-registration/deletion-related tasks, I have noticed the following errors in production.log at certain times.
2021-08-24T21:15:40 [E|kat|25e729cf] event_queue_error: type=delete_host_agent_queue, object_id=21
2021-08-24T21:15:40 [E|kat|25e729cf] Connection refused - connect(2) for "localhost" port 5671
2021-08-24T21:15:40 [E|kat|25e729cf] /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `initialize\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `new\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `connect\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.11/app/lib/katello/qpid/connection.rb:18:in `on_container_start\'
Version-Release number of selected component (if applicable):
Satellite 6.10 [ snap 13 ]
How reproducible:
Not always
Steps to Reproduce:
1. Install a satellite 6 and set it up for system deployment.
2. Deploy some RHEL 7 hosts concurrently from Satellite server and wait for their build to get completed.
3. Perform some re-registrations, host deletion etc
4. Execute --> grep \'Connection refused - connect(2) for "localhost"\' /var/log/foreman/production.log -B3 -A3
Actual results:
- grep \'Connection refused - connect(2) for "localhost"\' /var/log/foreman/production.log
B3 -A3
2021-08-24T18:45:23 [I|app|312ab092] ForemanWebhooks::EventSubscriber: host_updated.event.foreman event received
2021-08-24T18:45:23 [I|app|312ab092] Completed 201 Created in 1144ms (ActiveRecord: 298.3ms | Allocations: 225772)
2021-08-24T18:50:19 [E|kat|25e729cf] event_queue_error: type=delete_host_agent_queue, object_id=20
2021-08-24T18:50:19 [E|kat|25e729cf] Connection refused - connect(2) for "localhost" port 5671
2021-08-24T18:50:19 [E|kat|25e729cf] /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `initialize\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `new\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `connect\'
-
2021-08-24T20:02:54 [I|app|f39c138b] Processing by NotificationRecipientsController#index as JSON
2021-08-24T20:02:54 [I|app|f39c138b] Completed 200 OK in 46ms (Views: 0.2ms | ActiveRecord: 2.9ms | Allocations: 17943)
2021-08-24T20:05:13 [E|kat|25e729cf] event_queue_error: type=delete_host_agent_queue, object_id=20
2021-08-24T20:05:13 [E|kat|25e729cf] Connection refused - connect(2) for "localhost" port 5671
2021-08-24T20:05:13 [E|kat|25e729cf] /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `initialize\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `new\'
25e729cf | /opt/theforeman/tfm/root/usr/share/gems/gems/qpid_proton-0.33.0/lib/core/container.rb:130:in `connect\'
Expected results:
These should not occur at all.
Additional info:
I don\'t have any qpidd service running and based on the Engineering team, this scenario is occurring when satellite attempts to delete qpid queue when katello agent is disabled