Plugins » Katello

Shalomi/Dominic

Any advise on this issue?

Sameer Syed wrote:

I've a katello setup with a bunch of Capsules. The clients are not registered directly to Katello server, they are registered to the Capsules by installing the cert from the Capsules and running the subscription-manager. The Capsules were build by running the following:

capsule-installer --parent-fqdn "katello.example.com"\
--register-in-foreman "true"\
--foreman-oauth-key "xxxxxxxxxxxxxxxxxxxxxxx"\
--foreman-oauth-secret "xxxxxxxxxxxxxxxxxxxxxxx"\
--pulp-oauth-secret "xxxxxxxxxxxxxxxxxxxxxxx"\
--certs-tar "capsule.exmaple.com.tar"\
--puppet "true"\
--puppetca "true"\
--pulp "true"\
--qpid-router "true"\
--reverse-proxy "true"

I see the registered clients in the foreman console; when I try to apply the errats/patches, it fails with the error "Host did not respond within 30 seconds. Is katello-agent installed and goferd running on the Host?". I've verified goferd is running on the client (and have restarted it). I also see the following info when I restart goferd on the client:

Nov 9 19:27:06 katello.client.example.com goferd: [INFO][worker-0] root:481 - connecting to capsule.example.com:5647...
Nov 9 19:27:06 katello.client.example.com goferd: [INFO][worker-0] root:521 - Disconnected
Nov 9 19:27:06 katello.client.example.com goferd: [ERROR][worker-0] gofer.messaging.adapter.proton.connection:106 - connect: proton+amqps://capsule.example.com:5647, failed: Connection amqps://capsule.example.com:5647 disconnected
Nov 9 19:27:06 katello.client.example.com goferd: [INFO][worker-0] gofer.messaging.adapter.proton.connection:108 - retry in 106 seconds

Please advise on any resolutions.

Thanks.

Apologies for the delay.

Is there a firewall at all blocking port 5647 on the capsule server? A full foreman-debug from the katello server may help as well.

Justin Sherrill wrote:

Apologies for the delay.

Is there a firewall at all blocking port 5647 on the capsule server? A full foreman-debug from the katello server may help as well.

Hi Justin,

Yes there is a firewall, but the ports are open.

I'm having the same problem for some time now. I upgraded to 2.4.1 yesterday to try a fix, my qpid package versions now, are:
qpid-tools-0.30-4.el7.noarch
python-qpid-proton-0.9-13.el7.x86_64
qpid-dispatch-router-0.4-11.el7.x86_64
qpid-cpp-server-0.30-8.el7.x86_64
qpid-proton-c-0.9-13.el7.x86_64
python-qpid-qmf-0.30-5.el7.x86_64
libqpid-dispatch-0.4-11.el7.x86_64
python-qpid-0.30-9.el7.noarch
qpid-cpp-client-devel-0.30-8.el7.x86_64
python-gofer-qpid-2.6.2-1.el7.noarch
tfm-rubygem-qpid_messaging-0.30.0-3.el7.x86_64
qpid-cpp-server-linearstore-0.30-8.el7.x86_64
qpid-cpp-client-0.30-8.el7.x86_64
qpid-qmf-0.30-5.el7.x86_64

On my capsules I'm receiving this:

Mar 15 07:09:56 server01 goferd: [INFO][MainThread] katelloplugin:344 - reporting: {'enabled_repos': {'repos': [{'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/CentOS_7/x86_64'], 'repositoryid': 'mydomain_com_CentOS_7_x86_64'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/CentOS_7/x86_64_extras'], 'repositoryid': 'mydomain_com_CentOS_7_x86_64_extras'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/CentOS_7/x86_64_updates'], 'repositoryid': 'mydomain_com_CentOS_7_x86_64_updates'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/EPEL_7/x86_64'], 'repositoryid': 'mydomain_com_EPEL_7_x86_64'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/InfluxDB_7/x86_64_stable'], 'repositoryid': 'mydomain_com_InfluxDB_7_x86_64_stable'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/Katello_Client_el7/x86_64'], 'repositoryid': 'mydomain_com_Katello_Client_el7_x86_64'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/PuppetLabs_el7/dependencies_x86_64'], 'repositoryid': 'mydomain_com_PuppetLabs_el7_dependencies_x86_64'}, {'baseurl': ['https://foreman01.mydomain.com/pulp/repos/mydomain_com/Library/Production_7/custom/PuppetLabs_el7/products_x86_64'], 'repositoryid': 'mydomain_com_PuppetLabs_el7_products_x86_64'}]}}
Mar 15 07:09:58 server01 goferd: [INFO][worker-0] gofer.messaging.adapter.proton.connection:100 - connecting: URL: amqps://foreman01.mydomain.com:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
Mar 15 07:09:58 server01 goferd: [INFO][worker-0] root:505 - connecting to foreman01.mydomain.com:5647...
Mar 15 07:09:58 server01 goferd: [INFO][MainThread] gofer.agent.main:79 - agent started.
Mar 15 07:09:58 server01 goferd: [INFO][worker-0] root:547 - Disconnected
Mar 15 07:09:58 server01 goferd: [ERROR][worker-0] gofer.messaging.adapter.proton.connection:106 - connect: proton+amqps://foreman01.mydomain.com:5647, failed: Connection amqps://foreman01.mydomain.com:5647 disconnected
Mar 15 07:09:58 server01 goferd: [INFO][worker-0] gofer.messaging.adapter.proton.connection:108 - retry in 10 seconds
Mar 15 07:10:08 server01 goferd: [INFO][worker-0] gofer.messaging.adapter.proton.connection:100 - connecting: URL: amqps://foreman01.mydomain.com:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
Mar 15 07:10:08 server01 goferd: [INFO][worker-0] root:505 - connecting to foreman01.mydomain.com:5647...
Mar 15 07:10:08 server01 goferd: [INFO][worker-0] root:547 - Disconnected

This continues all over, always trying to reconnect.

SSL is working fine:

openssl s_client -connect foreman01.mydomain.com:5647 -cert /etc/pki/consumer/bundle.pem -key /etc/pki/consumer/bundle.pem -state -debug
...
    Start Time: 1458037569
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

this was caused by IPv6 connection. foreman01.mydomain.com has both A and AAAA records. it prefers IPv6... soon as I removed the AAAA record, it started working. I'll try to debug qpid-dispatcher now, why it's not listening on IPv6.

problem fixed:
/etc/qpid-dispatch/qdrouterd.conf shoud replace addr: 0.0.0.0 to addr: :: on listeners. this will make it ipv6 ready.

Hi Wagner,

We are tracking the issue here: http://projects.theforeman.org/issues/12386

The current version of qpid-dispatch we are using only allows for either ipv4 OR ipv6 but not both. We filed a bug with them and it has been resolved here: https://issues.apache.org/jira/browse/DISPATCH-190

I need to see if that fix is available in epel yet as we will likely be moving to those qpid packages soon.

Thanks for the heads up Justin.

Hi Justin,

I've upgraded to katello 2.4.1 version on the Katello Server and the Capsules, but I still see the error 'Host did not respond within 20 seconds. Sync has been cancelled. Is katello-agent installed and goferd running on the Host?' when publishing new content or when I sync using hammer. Do I have to change/add any configurations?

Hi Sameer,

Did you update any of the packages on the client? New versions of python-qpid-proton and qpid-proton-c were released for the clients as well.

Try updating those (if you haven't) and restart goferd. If the problem persists, the output generated from /var/log/messages would be useful.

Thanks