Project

General

Profile

Bug #28644

inconsistent repository enabled state for RHEL8 repository sets in the API

Added by Evgeni Golov 5 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Ohai,

when enabling repositories of a repository set, I see some very weird behavior (besides #28555).

First, please see the attached screenshots. I enable the "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)" repo, but in the enabled list is called "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs x86_64 8". This is probably "fine" as the first name comes from the "product", but the second is the actual generated "repo" name.

However, when we look at the API, the issue becomes more obvious (I think?):

With everything disabled:

# GET /katello/api/repository_sets/7416?organization_id=5
{
  "enabled": true,
  "product": {
    "id": 98,
    "name": "Red Hat Enterprise Linux for x86_64" 
  },
  "repositories": [],
  "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
  "vendor": "Red Hat",
  "label": "rhel-8-for-x86_64-baseos-rpms",
  "id": "7416",
  "type": "yum",
  "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
  "contentUrl": "/content/dist/rhel8/$releasever/x86_64/baseos/os" 
}

# GET /katello/api/products/98/repository_sets/7416/available_repositories?organization_id=5
{
  "total": 3,
  "subtotal": 3,
  "page": null,
  "per_page": null,
  "error": null,
  "search": null,
  "sort": {
    "by": null,
    "order": null
  },
  "results": [
    {
      "substitutions": {
        "releasever": "8" 
      },
      "path": "/content/dist/rhel8/8/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8",
      "pulp_id": "508c7fd9-8e83-4131-8436-a473eb6eecae",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "8.1" 
      },
      "path": "/content/dist/rhel8/8.1/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.1",
      "pulp_id": "a9238936-dc82-410d-8247-e6bbb84202d9",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "8.0" 
      },
      "path": "/content/dist/rhel8/8.0/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.0",
      "pulp_id": "2778ea7b-af64-405e-b29c-2a783a0bb1ef",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    }
  ]
}

After clicking the "enable" button for EL8 baseos:

# GET /katello/api/repository_sets/7416?organization_id=5
{
  "enabled": true,
  "product": {
    "id": 98,
    "name": "Red Hat Enterprise Linux for x86_64" 
  },
  "repositories": [
    {
      "id": 91,
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs x86_64 8",
      "arch": "x86_64",
      "releasever": "8" 
    }
  ],
  "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
  "vendor": "Red Hat",
  "label": "rhel-8-for-x86_64-baseos-rpms",
  "id": "7416",
  "type": "yum",
  "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
  "contentUrl": "/content/dist/rhel8/$releasever/x86_64/baseos/os" 
}

# GET /katello/api/products/98/repository_sets/7416/available_repositories?organization_id=5
{
  "total": 3,
  "subtotal": 3,
  "page": null,
  "per_page": null,
  "error": null,
  "search": null,
  "sort": {
    "by": null,
    "order": null
  },
  "results": [
    {
      "substitutions": {
        "releasever": "8" 
      },
      "path": "/content/dist/rhel8/8/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8",
      "pulp_id": "508c7fd9-8e83-4131-8436-a473eb6eecae",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "8.1" 
      },
      "path": "/content/dist/rhel8/8.1/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.1",
      "pulp_id": "a9238936-dc82-410d-8247-e6bbb84202d9",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "8.0" 
      },
      "path": "/content/dist/rhel8/8.0/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.0",
      "pulp_id": "2778ea7b-af64-405e-b29c-2a783a0bb1ef",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    }
  ]
}

As you see, the repository set now lists an repository, but the available repositories list still lists all of them as enabled: false.

This is however different for EL7 repos:

# GET /katello/api/repository_sets/2456?organization_id=5
{
  "enabled": true,
  "product": {
    "id": 88,
    "name": "Red Hat Enterprise Linux Server" 
  },
  "repositories": [],
  "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
  "vendor": "Red Hat",
  "label": "rhel-7-server-rpms",
  "id": "2456",
  "type": "yum",
  "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
  "contentUrl": "/content/dist/rhel/server/7/$releasever/$basearch/os" 
}

# GET /katello/api/products/88/repository_sets/2456/available_repositories?organization_id=5
{
  "total": 9,
  "subtotal": 9,
  "page": null,
  "per_page": null,
  "error": null,
  "search": null,
  "sort": {
    "by": null,
    "order": null
  },
  "results": [
    {
      "substitutions": {
        "releasever": "7Server",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7Server/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server",
      "pulp_id": "2efa9251-3cd9-4d76-a96d-4c029e221870",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.7",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.7/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.7",
      "pulp_id": "895d59fa-0962-4dec-ac7c-793cdce2a212",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.6",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.6/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.6",
      "pulp_id": "d1076412-9a93-4f7d-a527-036dd3b2de0c",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.5",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.5/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.5",
      "pulp_id": "040f0903-b2d6-45a7-b2ea-05666c29a814",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.4",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.4/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.4",
      "pulp_id": "c29545c8-0f1c-4889-9b2a-6c9ca0127237",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.3",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.3/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.3",
      "pulp_id": "4ced4ac0-5eeb-4803-958e-851dd0d39a1e",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.2",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.2/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.2",
      "pulp_id": "07692b7c-d144-40db-9b8b-0c48e0cd864c",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.1",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.1/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.1",
      "pulp_id": "7e0f7c0b-d6d0-4bdd-b837-2a0d71fd7314",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "7.0",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7.0/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7.0",
      "pulp_id": "b89a8f39-e03a-4c76-a0ce-25f77be97d71",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    }
  ]
}

And after enabling:

# GET /katello/api/repository_sets/2456?organization_id=5
{
  "enabled": true,
  "product": {
    "id": 88,
    "name": "Red Hat Enterprise Linux Server" 
  },
  "repositories": [
    {
      "id": 92,
      "name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server",
      "arch": "x86_64",
      "releasever": "7Server" 
    }
  ],
  "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
  "vendor": "Red Hat",
  "label": "rhel-7-server-rpms",
  "id": "2456",
  "type": "yum",
  "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
  "contentUrl": "/content/dist/rhel/server/7/$releasever/$basearch/os" 
}

# GET /katello/api/products/88/repository_sets/2456/available_repositories?organization_id=5
{
  "total": 9,
  "subtotal": 9,
  "page": null,
  "per_page": null,
  "error": null,
  "search": null,
  "sort": {
    "by": null,
    "order": null
  },
  "results": [
    {
      "substitutions": {
        "releasever": "7Server",
        "basearch": "x86_64" 
      },
      "path": "/content/dist/rhel/server/7/7Server/x86_64/os",
      "repo_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server",
      "pulp_id": "9a7531b7-4ea3-48bd-9f25-3c01a018196e",
      "name": "Red Hat Enterprise Linux 7 Server (RPMs)",
      "enabled": true,
      "promoted": false,
      "repository_id": 92
    },
    …
  ]
}

This behavior confuses our Ansible modules, as they try to enable the repository (even if it's already enabled) due to the "enabled:false" reporting, and this produces errors.

This works fine if I omit the "basearch:x86_64" from the API enable request (which I can do in Ansible, but not the UI), but then we face #28555 again.

Screenshot from 2020-01-06 10-20-57.png View Screenshot from 2020-01-06 10-20-57.png 24.9 KB screenshot of the repo to be enabled Evgeni Golov, 01/06/2020 09:40 AM
Screenshot from 2020-01-06 10-21-14.png View Screenshot from 2020-01-06 10-21-14.png 23.8 KB screenshot of the enabled repo Evgeni Golov, 01/06/2020 09:41 AM
Screenshot%20from%202020 01 06%2010 20 57 Screenshot%20from%202020 01 06%2010 21 14

Related issues

Related to Katello - Bug #28555: enabling a repository set without providing a basearch yields unexpected resultsNew
Related to Foreman - Tracker #27680: API bugs that need workarounds in foreman-ansible-modulesNew

Associated revisions

Revision 1a440ef3 (diff)
Added by Partha Aji 4 months ago

Fixes #28644 - Repo Mapper prunes unused subs (#8551)

Prior to this commit any action that did something like
PUT /katello/api/v2/products/572/repository_sets/7441/enable/
-d '{"releasever":"8", "basearch":"x86_64"}'
would set the arch for a repository using the basearch value even if
basearch is an invalid/unused url substitution. This caused weird bugs
with repo enablement unacknowleged etc

This commit clears out unused subtitutions before the url path is mapped
via repo mapper. This sets the correct value for the arch in the
repository object

History

#1 Updated by Evgeni Golov 5 months ago

  • Related to Bug #28555: enabling a repository set without providing a basearch yields unexpected results added

#2 Updated by Evgeni Golov 5 months ago

  • Related to Tracker #27680: API bugs that need workarounds in foreman-ansible-modules added

#3 Updated by Jonathon Turel 5 months ago

  • Triaged changed from No to Yes
  • Target version set to Katello 3.15.0

#4 Updated by Partha Aji 5 months ago

Not able to reproduce this error. I enabled "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)" and saw this instead.

$ curl -k -u admin:changeme http://example.com:3000/katello/api/products/543/repository_sets/7416?organization_id=3|jq .
{
  "enabled": true,
  "product": {
    "id": 543,
    "name": "Red Hat Enterprise Linux for x86_64" 
  },
  "repositories": [
    {
      "id": 11,
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8",
      "arch": "noarch",
      "releasever": "8" 
    }
  ],
  "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
  "vendor": "Red Hat",
  "label": "rhel-8-for-x86_64-baseos-rpms",
  "id": "7416",
  "type": "yum",
  "gpgUrl": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
  "contentUrl": "/content/dist/rhel8/$releasever/x86_64/baseos/os" 
}

$ curl -k -u admin:changeme http://example.com:3000/katello/api/products/543/repository_sets/7416/available_repositories?organization_id=3|jq .
{
  "total": 3,
  "subtotal": 3,
  "page": null,
  "per_page": null,
  "error": null,
  "search": null,
  "sort": {
    "by": null,
    "order": null
  },
  "results": [
    {
      "substitutions": {
        "releasever": "8" 
      },
      "path": "/content/dist/rhel8/8/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8",
      "pulp_id": "62048009-2729-4c59-a783-779142ce88c0",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": true,
      "promoted": false,
      "repository_id": 11
    },
    {
      "substitutions": {
        "releasever": "8.1" 
      },
      "path": "/content/dist/rhel8/8.1/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.1",
      "pulp_id": "6b979211-3410-45dd-986d-7910b164ea87",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    },
    {
      "substitutions": {
        "releasever": "8.0" 
      },
      "path": "/content/dist/rhel8/8.0/x86_64/baseos/os",
      "repo_name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8.0",
      "pulp_id": "7e44d672-578c-4581-8bdd-27a7f0ca8857",
      "name": "Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)",
      "enabled": false,
      "promoted": false,
      "repository_id": null
    }
  ]
}

Which seems ok. Can you share me your manifest ?

#5 Updated by Evgeni Golov 5 months ago

Partha, thanks for looking into this!

I'm using Rich's "RHCI" staging manifest, will send it to you via mail just to be sure.

How did you enable the repository? UI? API? The repo name in your output is "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8", not "Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs x86_64 8" as in mine, which makes me think your enable call did not include "basearch=x86_64"?

#6 Updated by Partha Aji 4 months ago

Reproduced it going to fix

#7 Updated by The Foreman Bot 4 months ago

  • Assignee set to Partha Aji
  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/8551 added

#8 Updated by The Foreman Bot 4 months ago

  • Fixed in Releases Katello 3.16.0 added

#9 Updated by Partha Aji 4 months ago

  • Status changed from Ready For Testing to Closed

#10 Updated by Jonathon Turel 3 months ago

  • Category changed from API to Repositories

Also available in: Atom PDF