Bug #11285
closedKatello 2.3 Repo Sync issue
Added by Duncan Innes about 9 years ago. Updated about 6 years ago.
Description
Clean install of base OS - CentOS 7.1
Clean install of Katello 2.3 RC
katello-installer run without any options to keep everything vanilla.
After creating a product (CentOS) and 3 repos (CentOS, CentOS Updates, CentOS Extras), then syncing these repos, I end up in a state where 2 out of the 3 repos hang during sync and I no longer have the 'event' Exchange when listing the Exchanges via qpid.
Files
candlepin.tgz | candlepin.tgz | 12.7 KB | /var/log/candleping tarball | Duncan Innes, 08/04/2015 10:13 AM | |
katello-installer.log | katello-installer.log | 1.67 MB | /var/log/katello-installer/katello-installer.log | Duncan Innes, 08/04/2015 10:14 AM |
Updated by Duncan Innes about 9 years ago
- File candlepin.tgz candlepin.tgz added
- File katello-installer.log katello-installer.log added
Updated by Duncan Innes about 9 years ago
[root@foreman log]# rpm -qa | grep katello
katello-installer-2.3.1-3.el7.noarch
pulp-katello-0.4-2.el7.noarch
katello-server-ca-1.0-1.noarch
katello-debug-2.3.0-4.el7.noarch
katello-repos-2.3.0-7.el7.noarch
katello-service-2.3.0-4.el7.noarch
katello-common-2.3.0-4.el7.noarch
katello-certs-tools-2.3.0-4.el7.noarch
katello-2.3.0-4.el7.noarch
katello-installer-base-2.3.1-3.el7.noarch
ruby193-rubygem-katello-2.3.0-3.el7.noarch
katello-default-ca-1.0-1.noarch
rubygem-hammer_cli_katello-0.0.15-1.el7.noarch
katello-selinux-2.2.1-1.el7.noarch
[root@foreman log]#
Updated by Duncan Innes about 9 years ago
The system is a libvirt guest running on a laptop installed with RHEL 7.1 and fully updated.
The guest system is kickstarted with a basic install of CentOS 7.1.1503 which is served from httpd running on the host OS. The kickstart has the following installation lines in the %post section:
echo "Modify NTP settings"
sed -i -e 's/0\..*\.pool\.ntp\.org/192.168.122.1/g' /etc/chrony.conf
sed -i -e '/org iburst/d' /etc/chrony.conf
echo "yum update"
yum -y update
echo "Install Katello repos"
#yum -y localinstall http://fedorapeople.org/groups/katello/releases/yum/2.2/katello/RHEL/7Server/x86_64/katello-repos-latest.rpm
#yum -y localinstall http://yum.theforeman.org/releases/1.8/el7/x86_64/foreman-release.rpm
#yum -y localinstall http://mirror.pnl.gov/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
#yum -y localinstall http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
#yum -y localinstall https://www.softwarecollections.org/en/scls/rhscl/v8314/epel-7-x86_64/download/rhscl-v8314-epel-7-x86_64.noarch.rpm
#yum -y localinstall https://www.softwarecollections.org/en/scls/rhscl/ruby193/epel-7-x86_64/download/rhscl-ruby193-epel-7-x86_64.noarch.rpm
yum -y localinstall http://fedorapeople.org/groups/katello/releases/yum/2.3/katello/RHEL/7Server/x86_64/katello-repos-latest.rpm
yum -y localinstall http://yum.theforeman.org/releases/1.9/el7/x86_64/foreman-release.rpm
yum -y localinstall http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
yum -y localinstall http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install foreman-release-scl
echo "Install Katello packages"
yum -y install katello
%end
Once the install is complete, the guest is booted, I log ssh in as root and run
[root@foreman ~]# katello-installer
without any options. Installation logs are uploaded, but I'll also upload the kickstart logs as there are some warnings about the tomcat user not being available when gutterball is installed.
Once Katello is installed, I log in and change the admin password to something I can remember.
I have then created a product called CentOS and 3 repos within that: CentOS 7, CentOS 7 Updates, and CentOS 7 Extras - all using the www.mirrorservice.org/sites/mirror.centos.org/7.1.1503(insert_relevant_yum_directory_here) as their source.
Initial sync of the 3 repos was mainly successful - each synched the majority of packages down, but left a handful behind. The second sync completed the repositories, leaving no packages unaccounted for. The third sync hung on 2 of the repos and I can no longer see the Product page in the Katello WebUI. Not sure of the hammer command to list products & repos as yet.
It appears that the 'event' Exchange is now missing on my system:
[root@foreman ~]# qpid-config -b amqps://localhost:5671 --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key exchanges
Type Exchange Name Attributes
==================================================
direct --replicate=none
direct C.dq --durable
direct amq.direct --durable --replicate=none
fanout amq.fanout --durable --replicate=none
headers amq.match --durable --replicate=none
topic amq.topic --durable --replicate=none
direct celery --durable
fanout celery.pidbox
topic celeryev --durable
direct qmf.default.direct --replicate=none
topic qmf.default.topic --replicate=none
topic qpid.management --replicate=none
direct resource_manager --durable
[root@foreman ~]#
Updated by Stephen Benjamin about 9 years ago
- Priority changed from Normal to High
Updated by Stephen Benjamin about 9 years ago
- Category set to 143
- Triaged changed from No to Yes
Updated by Alex Wood about 9 years ago
I've looked at this a bit with Duncan and I'm not sure why the event exchange is not being created. The installer log shows:
[DEBUG 2015-08-02 19:30:48 main] Exec[create candlepin qpid exchange](provider=posix): Executing 'qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://foreman.rhel7.local:5671' add exchange topic event --durable' [DEBUG 2015-08-02 19:30:48 main] Executing 'qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://foreman.rhel7.local:5671' add exchange topic event --durable' [ WARN 2015-08-02 19:30:48 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/returns: executed successfully
It's a bit strange that a command that "executed successfully" would be logged at WARN level, but aside from that general observation, I don't have a specific suggestion. I do see Puppet verify that the exchange exists later:
[DEBUG 2015-08-02 19:30:48 main] Exec[create candlepin qpid exchange](provider=posix): Executing check 'qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://foreman.rhel7.local:5671' exchanges event' [DEBUG 2015-08-02 19:30:48 main] Executing 'qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://foreman.rhel7.local:5671' exchanges event' [DEBUG 2015-08-02 19:30:48 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/unless: Type Exchange Name Attributes [DEBUG 2015-08-02 19:30:48 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/unless: ============================================= [DEBUG 2015-08-02 19:30:48 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/unless: topic event --durable [ WARN 2015-08-02 19:30:48 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]: Triggered 'refresh' from 4 events
Is the qpid-cpp-server-store package getting installed?
Updated by Peter Verbist about 9 years ago
No, the qpid-cpp-server-store package is missing.
[vagrant@katello ~]$ rpm -qa | grep qpid-cpp
qpid-cpp-client-0.30-7.proton.0.9.el7.x86_64
qpid-cpp-client-devel-0.30-7.proton.0.9.el7.x86_64
qpid-cpp-server-linearstore-0.30-7.proton.0.9.el7.x86_64
qpid-cpp-server-0.30-7.proton.0.9.el7.x86_64
When I try to install the package I receive:
Package qpid-cpp-server-store-0.32-3.el7.x86_64 is obsoleted by qpid-cpp-server-linearstore-0.30-7.proton.0.9.el7.x86_64 which is already installed
Updated by Alex Wood about 9 years ago
I will wager that qpid-cpp-server-store/linearstore missing on Duncan's machine is the problem. According to a fellow I spoke with, "If you create an exchange and restart qpidd, the exchange will be gone (if there's no store installed)."
Updated by Duncan Innes about 9 years ago
Peter is right. qpid-cpp-server-store is missing.
When I try to install it, I'm told the same thing (that it's been obsoleted by linearstore), but I don't have that package installed on my server.
Updated by Stephen Benjamin about 9 years ago
- Assignee set to Stephen Benjamin
qpid-cpp-server-linearstore was mistanekly removed as a dep of pulp-katello.
https://github.com/Katello/pulp-katello-plugins/commit/befa66c09bce0df73131dc2c25cbe45eee6fc1cc
Will fix in the next RC. Thanks guys!
Updated by Stephen Benjamin about 9 years ago
- Related to Bug #10574: pulp-katello plugin depends on qpid added
Updated by Peter Verbist about 9 years ago
Installed from scratch and ran qpid-config command:
qpid-config -b amqps://localhost:5671 --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key exchanges
Type Exchange Name Attributes
==================================================
direct --replicate=none
direct C.dq --durable
direct amq.direct --durable --replicate=none
fanout amq.fanout --durable --replicate=none
headers amq.match --durable --replicate=none
topic amq.topic --durable --replicate=none
direct celery --durable
fanout celery.pidbox
topic celeryev --durable
topic event --durable
direct qmf.default.direct --replicate=none
topic qmf.default.topic --replicate=none
topic qpid.management --replicate=none
direct resource_manager --durable
Seems to be quiet normal. Stop/start katello using the katello-service and issue the same command:
qpid-config -b amqps://localhost:5671 --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key exchanges
Type Exchange Name Attributes
==================================================
direct --replicate=none
direct C.dq --durable
direct amq.direct --durable --replicate=none
fanout amq.fanout --durable --replicate=none
headers amq.match --durable --replicate=none
topic amq.topic --durable --replicate=none
direct celery --durable
fanout celery.pidbox
topic celeryev --durable
direct qmf.default.direct --replicate=none
topic qmf.default.topic --replicate=none
topic qpid.management --replicate=none
direct resource_manager --durable
--> the exchange name called event is missing.
Updated by Stephen Benjamin about 9 years ago
Peter, install qpid-cpp-server-linearstore and run that command again and it'll fix it.
We'll correct this in 2.3 RC2.
Updated by The Foreman Bot about 9 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/Katello/katello/pull/5399 added
- Pull request deleted (
)
Updated by Eric Helms about 9 years ago
- Status changed from Ready For Testing to Closed