Project

General

Profile

Actions
Copy link

Bug #15353

closed

Katello 2.4 -> 3.0 upgrade issue (update_metadata_expire)

Added by John Beranek over 8 years ago. Updated over 6 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Katello Recycle Bin
Difficulty:
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Another upgrade error, I'm afraid:

[ INFO 2016-03-08 14:23:57 main] Upgrade Step: errata_import (this may take a while) ...
[DEBUG 2016-03-08 14:28:48 main] Importing Errata
Updating Content Host Repositories padevprt01.staging.ixdev.pacpservices.net
There was an error updating Content Host padevprt01.staging.ixdev.pacpservices.net with id 88
Updating Content Host Repositories padevprt02.staging.ixdev.pacpservices.net
There was an error updating Content Host padevprt02.staging.ixdev.pacpservices.net with id 89
Generating applicability for 181 Content Hosts

[ INFO 2016-03-08 14:28:48 main] Upgrade Step: update_gpg_urls (this may take a while) ...
[DEBUG 2016-03-08 14:29:29 main] Importing GPG Key Urls to support Capsule Communication

[ INFO 2016-03-08 14:29:29 main] Upgrade Step: update_repository_metadata (this may take a while) ...
[ERROR 2016-03-08 14:30:56 main] rake aborted!
Required lock is already taken by other running tasks.
Please inspect their state, fix their errors and resume them.

Required lock: read
Conflicts with tasks:
- https://katello.ad.pvt//foreman_tasks/tasks/decd9e67-1961-449c-a8b0-c16bccc18e52

Tasks: TOP => katello:upgrades:2.2:update_metadata_expire
(See full trace by running task with --trace)
Updating Expire Metadata for Custom Content

[ERROR 2016-03-08 14:30:56 main] Upgrade step update_repository_metadata failed. Check logs for more information.
[DEBUG 2016-03-08 14:30:56 main] Exit with status code: 1 (signal was 1)
[ERROR 2016-03-08 14:30:56 main] Repeating errors encountered during run:
[ERROR 2016-03-08 14:30:56 main] <NilClass> nil
[ERROR 2016-03-08 14:30:56 main]  /Stage[main]/Certs::Qpid/Exec[add-broker-cert-to-nss-db]: Failed to call refresh: certutil -A -d '/etc/pki/katello/nssdb' -n 'broker' -t ',,' -a -i '/etc/pki/katello/certs/pal130.ad.pvt-qpid-broker.crt' returned 255 instead of one of [0]
[ERROR 2016-03-08 14:30:56 main]  /Stage[main]/Certs::Qpid/Exec[add-broker-cert-to-nss-db]: certutil -A -d '/etc/pki/katello/nssdb' -n 'broker' -t ',,' -a -i '/etc/pki/katello/certs/pal130.ad.pvt-qpid-broker.crt' returned 255 instead of one of [0]
[ERROR 2016-03-08 14:30:56 main]  /Stage[main]/Certs::Candlepin/Exec[candlepin-add-client-cert-to-nss-db]: Failed to call refresh: certutil -A -d '/etc/pki/katello/nssdb' -n 'amqp-client' -t ',,' -a -i '/etc/pki/katello/certs/java-client.crt' returned 255 instead of one of [0]
[ERROR 2016-03-08 14:30:56 main]  /Stage[main]/Certs::Candlepin/Exec[candlepin-add-client-cert-to-nss-db]: certutil -A -d '/etc/pki/katello/nssdb' -n 'amqp-client' -t ',,' -a -i '/etc/pki/katello/certs/java-client.crt' returned 255 instead of one of [0]
Actions
Copy link
 #2

Updated by John Beranek over 8 years ago

After this partially incomplete upgrade also noticed problems with host subscriptions:

# subscription-manager register --force
The system with UUID cd5f2426-3cd4-48fc-aaac-da09663cbf65 has been unregistered
Validation failed: Interfaces host must have one primary interface

and:

$ hammer -u admin -p MyForemanPassword content-host update --id cd5f2426-3cd4-48fc-aaac-da09663cbf65 --new-name server001.example.com --organization "Example Com" 
Could not update content host:
  Validation failed: Interfaces host must have one primary interface
Actions
Copy link
 #3

Updated by Eric Helms over 8 years ago

  • Status changed from New to Need more information

Did you have an active or broken tasks when you upgraded? You could try applying this change locally and running the rake task to check:

https://github.com/Katello/katello/pull/6097/files

Actions
Copy link
 #4

Updated by John Beranek over 8 years ago

OK, output of katello:upgrade_check:

This script makes no modifications and can be re-run multiple times for the most up to date results.
Checking upgradeability...

Checking for running tasks...
[PASS] - There are 0 active tasks.

Checking the current version...
[PASS] - Current version of Katello is 2.4.2 and needs to be 2.4.2

Checking content hosts...
Calculating Host changes on upgrade.  This may take a few minutes.

Summary:
Content hosts to be preserved: 201
Content hosts to be deleted: 3
Details on Content Hosts planned for deletion saved to /tmp/pre-upgrade-1465460056.csv
You may want to manually resolve some of the conflicts and duplicate Content Hosts.
Upon upgrade those found for deletion will be removed permanently.

3 duplicate content hosts...

Actions
Copy link
 #5

Updated by John Beranek over 8 years ago

Of the 3 systems listed:

  • 2 were different systems with different names in Katello, but with the same OS hostname, as they were using short hostnames rather than FQDNs, and differed only by the name. These I have re-registered (the first at least told me "This system is not yet registered. Try 'subscription-manager register --help' for more information.")
  • The last was a system which I couldn't find in the GUI, so I deleted it by hand in psql
Actions
Copy link
 #6

Updated by John Beranek over 8 years ago

It may be worth pointing out that the Katello 2.4 system before the upgrade had 5 hosts with the base name "padevsup01", and after the upgrade there was only one host in the "hosts" table, and 3 in the "katello_systems" table.

Actions
Copy link
 #7

Updated by Justin Sherrill over 8 years ago

Hey John,

I assume you've rolled back to 2.4, would you be able to run this:

https://gist.github.com/jlsherrill/35c7a27a9bcf0b0d885e23f6bc25b57d

and upload the resulting file /tmp/system_debug.txt. You can upload it to the ticket, foreman-debug, or if you are more comfortable with email (jsherril <@@> redhat.com)

For your re-registration errors, i've opened an issue here: http://projects.theforeman.org/issues/15357 and already have a fix open and under review.

-Justin

Actions
Copy link
 #8

Updated by John Beranek over 8 years ago

I've rolled back to 2.4, but also have a clone of the 3.0 upgrade up and running if it's helpful (just fooled the installation with /etc/hosts)

Actions
Copy link
 #9

Updated by John Beranek over 8 years ago

Debug sent by private email

Actions
Copy link
 #10

Updated by Eric Helms over 8 years ago

  • Translation missing: en.field_release changed from 86 to 144
Actions
Copy link
 #11

Updated by Eric Helms over 8 years ago

  • Translation missing: en.field_release changed from 144 to 168
Actions
Copy link
 #12

Updated by Eric Helms over 8 years ago

  • Translation missing: en.field_release changed from 168 to 171
Actions
Copy link
 #13

Updated by Eric Helms over 8 years ago

  • Translation missing: en.field_release deleted (171)
Actions
Copy link
 #14

Updated by Justin Sherrill over 8 years ago

Hey John,

I a bunch of upgrade fixes were added in 3.0.1 and 3.0.2, have you tried again recently?

Actions
Copy link
 #15

Updated by John Beranek over 8 years ago

I did eventually manage to get the 3.0.1 upgrade to complete without any errors, but the upgraded system was subtly broken in other ways and I decided to start afresh with a new 3.0.1 installation, which does run a lot smoother, and gets rid of some the legacy things I wanted to get rid of, like the attempt to add in RHEL client support

Actions
Copy link
 #16

Updated by Justin Sherrill over 8 years ago

  • Status changed from Need more information to Rejected
  • Translation missing: en.field_release set to 166

Thanks john, i'll go ahead and close this out. Feel free to reopen if you see this again.

Actions
Copy link

Also available in: Atom PDF