Project

General

Profile

Feature #26829

Generate yum repo metadata in sqlite format

Added by Baptiste Agasse 6 months ago. Updated 6 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

By default, pulp and katello don't generate yum repo metadata in sqlite format, but we can configure pulp to generate it as repo level or global level as stated in https://docs.pulpproject.org/plugins/pulp_rpm/tech-reference/yum-plugins.html.

Using sqlite metadata in our case decrease the /var/cache/yum usage on our clients:
  • From 254MB to 209MB (CentOS 7 + CentOS RHCL + EPEL + Puppet + some internal repos)
  • From 1.5GB to 600MB (CentOS 7 + CentOS RHCL + EPEL + Puppet + some internal repos) when we use yum provides command usage that download additionnal filelist metadata.

To enable it as global level: create a file named /etc/pulp/server/plugins.conf.d/yum_distributor.json with content:

{                                                                                                                                                                                                                                             
  "generate_sqlite": true                                                                                                                                                                                                                     
} 

Adding this configuration need to regenerate products's repos metadata and then content views metadata.

As a user, I want be to able to configure sqlite yum repo metadata generation at katello installation time and/or at yum repository creation time.

History

#1 Updated by John Mitsch 6 months ago

  • Triaged changed from No to Yes
  • Target version set to Katello Backlog
  • Category set to Repositories

Thanks for the suggestion!

#2 Updated by Baptiste Agasse 6 months ago

After some testing, it seems that it break EL6 with the following error:


bash-4.1# yum update
Loaded plugins: fastestmirror, ovl
Setting up Update Process
Determining fastest mirrors
centos-os                                                                                              | 4.2 kB     00:00     
centos-base/primary_db                                                                                 | 6.7 MB     00:15     
centos-extras                                                                                          | 4.2 kB     00:00     
centos-extras/primary_db                                                                               |  29 kB     00:00     
centos-update                                                                                          | 4.2 kB     00:00     
centos-update/primary_db                                                                               | 2.6 MB     00:06     
epel                                                                                                   | 4.2 kB     00:00     
epel/primary_db                                                                                        | 6.0 MB     00:13     
internal                                                                                               | 4.2 kB     00:00     
internal/primary_db                                                                                    |  33 kB     00:00     
Error: file is encrypted or is not a database
bash-4.1# rpm -qa | grep -e yum -e centos-release
centos-release-6-10.el6.centos.12.3.x86_64
yum-metadata-parser-1.1.2-16.el6.x86_64
yum-3.2.29-81.el6.centos.0.1.noarch
yum-plugin-fastestmirror-1.1.30-42.el6_10.noarch
yum-plugin-ovl-1.1.30-42.el6_10.noarch

As the yum version provided with EL6 don't provides the mddownloadpolicy config option to force yum to fallback to xml metadata format, enabling this on global scope is not an option at short term.

Also available in: Atom PDF