Bug #32284
Updated by Sven Solberg about 3 years ago
Hi, I have (had) 9had) a 3.14 install and I followed the upgrade process notes on https://theforeman.org/plugins/katello/3.18/upgrade/index.html It all seemed to go well, no errors encountered. My next step is to migrate to Pulp3. I followed the instructions for correcting the permissions, and ran the migration routine. It failed with a backtrace <pre> [root@foreman ~]# chmod -R g+rwX /var/lib/pulp/content [root@foreman ~]# find /var/lib/pulp/content -type d -perm -g-s -exec sudo chmod g+s {} \; [root@foreman ~]# chgrp -R pulp /var/lib/pulp/content [root@foreman ~]# foreman-maintain content migration-stats Running Retrieve Pulp 2 to Pulp 3 migration statistics ================================================================================ Retrieve Pulp 2 to Pulp 3 migration statistics: ============Migration Summary================ Migrated/Total RPMs: 0/54175 Migrated/Total errata: 0/178581 Migrated/Total repositories: 0/634 Estimated migration time based on yum content: 1 hours, 23 minutes Note: ensure there is sufficient storage space for /var/lib/pulp/published to triple in size before starting the migration process. Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/' [OK] -------------------------------------------------------------------------------- [root@foreman ~]# foreman-maintain content prepare Running Prepare content for Pulp 3 ================================================================================ Prepare content for Pulp 3: Starting task. 2021-04-10 16:13:42 +0100: Pre-migrating Pulp 2 ERRATUM content (general info) 5027/5132Migration failed, You will want to investigate: https://foreman.mydom/foreman_tasks/tasks/201e3c25-55d2-4361-a845-699f019cf940 rake aborted! ForemanTasks::TaskError: Task 201e3c25-55d2-4361-a845-699f019cf940: Katello::Errors::Pulp3Error: PackageReleaseComponent matching query does not exist. /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.2.1/lib/katello/tasks/pulp3_migration.rake:33:in `block (2 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>' Tasks: TOP => katello:pulp3_migration (See full trace by running task with --trace) [FAIL] Failed executing foreman-rake katello:pulp3_migration, exit status 1 -------------------------------------------------------------------------------- Scenario [Prepare content for Pulp 3] failed. The following steps ended up in failing state: [content-prepare] Resolve the failed steps and rerun the command. In case the failures are false positives, use --whitelist="content-prepare" [root@foreman ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 5.8G 0 5.8G 0% /dev tmpfs 5.8G 40K 5.8G 1% /dev/shm tmpfs 5.8G 98M 5.7G 2% /run tmpfs 5.8G 0 5.8G 0% /sys/fs/cgroup /dev/mapper/foreman-root 10G 5.6G 4.5G 56% / /dev/vda1 1014M 329M 686M 33% /boot /dev/mapper/foreman-var 27G 13G 15G 47% /var nas1:/volume7/foreman/pulp 576G 180G 397G 32% /var/lib/pulp tmpfs 1.2G 0 1.2G 0% /run/user/0 </pre> here's the full raw output from the task: <pre> { "pulp_tasks": [ { "pulp_href": "/pulp/api/v3/tasks/1711b0a4-c05f-4b1e-aaeb-95a94499e802/", "pulp_created": "2021-04-10T15:13:14.087+00:00", "state": "failed", "name": "pulp_2to3_migration.app.tasks.migrate.migrate_from_pulp2", "started_at": "2021-04-10T15:13:14.180+00:00", "finished_at": "2021-04-10T15:13:43.901+00:00", "error": { "traceback": " File \"/usr/lib/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n rv = job.perform()\n File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 684, in perform\n self._result = self._execute()\n File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 690, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py\", line 81, in migrate_from_pulp2\n migrate_content(plan, skip_corrupted=skip_corrupted)\n File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py\", line 47, in migrate_content\n plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)\n File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/deb/migrator.py\", line 85, in migrate_content_to_pulp3\n loop.run_until_complete(dm.create())\n File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n return future.result()\n File \"/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py\", line 89, in create\n await pipeline\n File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py\", line 105, in run\n d_content.content.q()\n File \"/usr/lib/python3.6/site-packages/django/db/models/manager.py\", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File \"/usr/lib/python3.6/site-packages/django/db/models/query.py\", line 408, in get\n self.model._meta.object_name\n", "description": "PackageReleaseComponent matching query does not exist." }, "worker": "/pulp/api/v3/workers/a70acf17-2574-4172-99d8-7545f61a4802/", "child_tasks": [], "task_group": "/pulp/api/v3/task-groups/e3fa2a74-4623-4a2a-bd90-95e3e1c3daee/", "progress_reports": [ { "message": "Processing Pulp 2 repositories, importers, distributors", "code": "processing.repositories", "state": "completed", "total": 7, "done": 7 }, { "message": "Pre-migrating Pulp 2 DEB content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 DEB content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 DEB_RELEASE content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 1, "done": 1 }, { "message": "Pre-migrating Pulp 2 DEB_RELEASE content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 1, "done": 1 }, { "message": "Pre-migrating Pulp 2 DEB_COMPONENT content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 1, "done": 1 }, { "message": "Pre-migrating Pulp 2 DEB_COMPONENT content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 1, "done": 1 }, { "message": "Pre-migrating Pulp 2 RPM content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 RPM content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_CATEGORY content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 SRPM content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 SRPM content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 DISTRIBUTION content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 DISTRIBUTION content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_CATEGORY content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 ERRATUM content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 5751, "done": 5697 }, { "message": "Pre-migrating Pulp 2 ERRATUM content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 5751, "done": 5697 }, { "message": "Pre-migrating Pulp 2 YUM_REPO_METADATA_FILE content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 YUM_REPO_METADATA_FILE content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Migrating deb content to Pulp 3 deb_component", "code": "migrating.deb.content", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_LANGPACKS content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_LANGPACKS content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_ENVIRONMENT content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_GROUP content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_GROUP content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 PACKAGE_ENVIRONMENT content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Migrating deb content to Pulp 3 deb_component2", "code": "migrating.deb.content", "state": "completed", "total": 367, "done": 367 }, { "message": "Pre-migrating Pulp 2 MODULEMD content (general info)", "code": "premigrating.content.general", "state": "completed", "total": 0, "done": 0 }, { "message": "Pre-migrating Pulp 2 MODULEMD content (detail info)", "code": "premigrating.content.detail", "state": "completed", "total": 0, "done": 0 }, { "message": "Creating repositories in Pulp 3", "code": "creating.repositories", "state": "completed", "total": 0, "done": 0 }, { "message": "Migrating importers to Pulp 3", "code": "migrating.importers", "state": "completed", "total": 0, "done": 0 }, { "message": "Migrating content to Pulp 3", "code": "migrating.content", "state": "failed", "total": 0, "done": 0 }, { "message": "Migrating deb content to Pulp 3 deb_release", "code": "migrating.deb.content", "state": "completed", "total": 0, "done": 0 }, { "message": "Migrating deb content to Pulp 3 deb", "code": "migrating.deb.content", "state": "completed", "total": 13, "done": 0 }, { "message": "Migrating deb content to Pulp 3 deb_component3", "code": "migrating.deb.content", "state": "completed", "total": 1, "done": 1 } ], "created_resources": [ "/pulp/api/v3/task-groups/e3fa2a74-4623-4a2a-bd90-95e3e1c3daee/" ], "reserved_resources_record": [ "pulp_2to3_migration" ] } ], "task_groups": [ { "pulp_href": "/pulp/api/v3/task-groups/e3fa2a74-4623-4a2a-bd90-95e3e1c3daee/", "description": "Migration Sub-tasks", "all_tasks_dispatched": false, "waiting": 0, "skipped": 0, "running": 0, "completed": 0, "canceled": 0, "failed": 1, "group_progress_reports": [ { "message": "Repo version creation", "code": "create.repo_version", "total": 0, "done": 0 }, { "message": "Distribution creation", "code": "create.distribution", "total": 0, "done": 0 } ] } ], "poll_attempts": { "total": 18, "failed": 1 } } </pre>