Project

General

Profile

Bug #17519

Problem with pulp db upgrade after install and upgrade

Added by Uldis Biks over 5 years ago. Updated almost 4 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
Repositories
Target version:
Difficulty:
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Hi,
system rhel7
Installed clean install of 3.1 katello some time ago:
[root@lv-katello ~]# rpm -qa | grep katello
katello-debug-3.1.0-5.el7.noarch
lv-katello.ne.1dc.com-foreman-proxy-1.0-2.noarch
lv-katello.ne.1dc.com-puppet-client-1.0-1.noarch
lv-katello.ne.1dc.com-apache-1.0-1.noarch
lv-katello.ne.1dc.com-qpid-broker-1.0-1.noarch
tfm-rubygem-katello-3.1.0.1-3.el7.noarch
katello-service-3.1.0-5.el7.noarch
katello-server-ca-1.0-1.noarch
lv-katello.ne.1dc.com-qpid-router-client-1.0-1.noarch
lv-katello.ne.1dc.com-foreman-client-1.0-1.noarch
katello-repos-3.1.0-5.el7.noarch
katello-selinux-3.0.1-1.el7.noarch
katello-certs-tools-2.4.0-1.el7.noarch
katello-installer-base-3.1.0-2.el7.noarch
katello-3.1.0-5.el7.noarch
tfm-rubygem-hammer_cli_katello-0.0.26-1.201606161944gita9fa180.el7.noarch
lv-katello.ne.1dc.com-qpid-router-server-1.0-1.noarch
lv-katello.ne.1dc.com-foreman-proxy-client-1.0-1.noarch
python-urllib3-1.10.2-2.katello.el7.noarch
katello-common-3.1.0-5.el7.noarch
foreman-installer-katello-3.1.0-2.el7.noarch
lv-katello.ne.1dc.com-qpid-client-cert-1.0-1.noarch
katello-default-ca-1.0-1.noarch
katello-client-bootstrap-1.1.0-1.el7.noarch
pulp-katello-1.0.2-1.el7.noarch

But there were problems with pulp - it was returning error 500 and logs showed
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) The database has not been migrated to the current version. Run pulp-manage-db and restart the application.
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) Traceback (most recent call last):
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 111, in wsgi_application
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) _initialize_web_services()
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 74, in _initialize_web_services
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) migration_models.check_package_versions()
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 310, in check_package_versions
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) raise Exception(error_message)
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:ERROR: (2168-64448) InitializationException: The database has not been migrated to the current version. Run pulp-manage-db and restart the application.
Nov 29 14:52:32 lv-katello.ne.1dc.com pulp2168: pulp.server.webservices.application:CRITICAL: *****************************************************

Tried to run upgrade db but no luck
[root@lv-katello log]# sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Updating the database with types []
Found the following type definitions that were not present in the update collection [puppet_module, srpm, erratum, docker_blob, drpm, distribution, package_group, package_category, iso, package_environment, docker_tag, package_langpacks, rpm, yum_repo_metadata_file, docker_image, docker_manifest]
Updating the database with types [puppet_module, docker_tag, erratum, docker_blob, docker_manifest, yum_repo_metadata_file, package_group, package_category, iso, package_environment, drpm, distribution, rpm, srpm, docker_image]
Found the following type definitions that were not present in the update collection [package_langpacks]
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
The database for migration package pulp.server.db.migrations is at version 24, which is larger than the latest version available, 23.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 193, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 73, in migrate_database
raise DataError(msg)
DataError: The database for migration package pulp.server.db.migrations is at version 24, which is larger than the latest version available, 23.

So I left it alone for a while, now I noticed that 3.2 is out so tried to upgrade, downloaded packages, upgrade check was successful, but running actual upgrade script returned error again with pulp but bit different:
[root@lv-katello ~]# foreman-installer --scenario katello --upgrade
Upgrading...
Upgrade Step: stop_services...
Redirecting to /bin/systemctl stop foreman-tasks.service

Redirecting to /bin/systemctl stop httpd.service

Redirecting to /bin/systemctl stop foreman-proxy.service

Redirecting to /bin/systemctl stop tomcat.service

Redirecting to /bin/systemctl stop squid.service

Redirecting to /bin/systemctl stop qpidd.service

Success!

Upgrade Step: start_databases...
Redirecting to /bin/systemctl start mongod.service

Redirecting to /bin/systemctl start postgresql.service

Success!

Upgrade Step: update_http_conf...

Upgrade Step: migrate_pulp...

938

Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Updating the database with types []
Found the following type definitions that were not present in the update collection [puppet_module, srpm, erratum, docker_blob, drpm, distribution, package_group, package_category, iso, package_environment, docker_tag, package_langpacks, rpm, yum_repo_metadata_file, docker_image, docker_manifest]
Updating the database with types [puppet_module, drpm, package_langpacks, erratum, docker_blob, docker_manifest, yum_repo_metadata_file, package_group, package_category, iso, package_environment, docker_tag, distribution, rpm, srpm, docker_image]
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 24
Migration package pulp_docker.plugins.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 5
The database for migration package pulp_rpm.plugins.migrations is at version 35, which is larger than the latest version available, 33.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 193, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 73, in migrate_database
raise DataError(msg)
DataError: The database for migration package pulp_rpm.plugins.migrations is at version 35, which is larger than the latest version available, 33.

Upgrade step migrate_pulp failed. Check logs for more information.

Any suggestions would be much appreciated. Tnx!

History

#1 Updated by Eric Helms over 5 years ago

  • Status changed from New to Need more information

What versions of Pulp were on the box before the upgrade? Did you run yum update or switch to the Katello 3.2 repositories without running the installer upgrade?

#2 Updated by Uldis Biks over 5 years ago

[root@lv-katello ~]# cat /var/log/yum.log* | grep pulp-serv
Nov 29 15:28:41 Updated: pulp-server-2.9.1-1.el7.noarch
Oct 24 13:38:49 Installed: pulp-server-2.8.7-1.el7.noarch

2.8.7 with 3.1 katello version

I checked updates with 3.1 repo - there were none so I follwed instructions and switched to 3.2
---
https://theforeman.org/plugins/katello/3.2/upgrade/index.html
---

#3 Updated by Uldis Biks over 5 years ago

Couple of reinstalls and katello is working now.

#4 Updated by Justin Sherrill over 5 years ago

  • Status changed from Need more information to Rejected
  • Legacy Backlogs Release (now unused) set to 166

Also available in: Atom PDF