Plugins » Katello

h1. Pulp 3 Integration

This page is meant to capture links to various resources around pulp3 integration

* Initial Integration Document: https://docs.google.com/document/d/1eFM4H9BteA_nTDbUkRJgEiqSttXfAtB9-51a_OuwVJw/edit?usp=sharing

* Initial Community Post: https://community.theforeman.org/t/pulp-3-migration/10474

* Youtube Videos: https://www.youtube.com/playlist?list=PL59WSIBtIpv7tbjnsEAWNcdiOKAmjjgqE

* Tasks List: https://docs.google.com/spreadsheets/d/1PFx-VAEXlivlqDYR9INMW26AUWtuMS_XtEnwhFVP5Rc/edit?usp=sharing

* Main Tracker: https://projects.theforeman.org/issues/26987

h2. Katello Dev box with yum plugin using vagrant:

+*Install and enable pulp_rpm plugin*+ 

* vagrant up katello-dev-box

* vagrant ssh katello-dev-box

* Execute commands below: 

<pre>

sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel

sudo pip3 install scikit-build nose

sudo pip3 install pulp-rpm==3.0.0

sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' python3-django-admin migrate --no-input

sudo systemctl restart pulpcore-resource-manager pulpcore-content pulpcore-api pulpcore-worker@1 pulpcore-worker@2

</pre>

* Refresh the smart proxy 

# login to katello/foreman

# navigate to infrastructure > smart proxies

# click 'refresh' in the list of actions beside your smart proxy

You should now see file, docker and yum as supported content type under pulpcore in the *Services tab*  

h2. Katello Dev environment with pulp 3 using ansible playbook:

* Install a katello dev environment with forklift

* Install the following packages in your dev box: 

<pre>sudo yum -y install https://people.redhat.com/iballou/libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>

<pre>sudo yum -y install https://people.redhat.com/iballou/python3-libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>

* checkout this forklift pr if its not merged: https://github.com/theforeman/forklift/pull/980

* setup ansible-pulp:

<pre>

cd forklift

git clone https://github.com/pulp/ansible-pulp.git

cd ansible-pulp

ansible-galaxy install -r requirements.yml -p ./roles

ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles

</pre>

* within forkflift run:

<pre>

ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-katello-devel

</pre>

* Refresh the smart proxy: 

  1. login to katello/foreman

  2. navigate to infrastructure > smart proxies

  3. click 'refresh' in the list of actions beside your smart proxy

You're now rockin' Pulp3

* *Known Issue:* Can't seem to use pulp3 on the box since pulp-api.service fails to start.

<pre>

sudo chmod -R 777 /etc/pki/pulp/

sudo systemctl restart pulpcore-api.service

</pre>

* Reset the Katello Pulp 3 Dev Environment

From within the Foreman directory:

<pre>bundle exec rake katello:reset</pre>

h2. Smart Proxy Setup:

* Install a smart proxy to point to your pulp3_dev box

* ssh into the smart proxy

* sudo mkdir /etc/httpd/conf.d/05-foreman-ssl.d

* sudo vi /etc/httpd/conf/httpd.conf

* Add IncludeOptional "/etc/httpd/conf.d/05-foreman-ssl.d/*.conf to the end of the file and save

* In forklift > Run ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-proxy-devel

h3. Other Resources

* pulp3 gap analysis wiki https://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis

* pulp3 gap analysis etherpad https://etherpad.net/p/pulp3_container

* pulp3 docker roadmap https://pulp.plan.io/projects/pulp/wiki/Pulp_Docker_Roadmap

h3. Old way to deploy pulp3 on other box

* <pre>yum install vagrant-sshfs</pre>

* Checkout pulplift: https://github.com/pulp/pulplift 

* Follow pulplift readme instructions, copy the centos7-pulp3-github box definition from vagrant/boxes.d/99-local.yaml.example 

* vagrant up centos7-pulp3-github

* On katello dev env, edit /etc/foreman-proxy/settings.d/pulp3.yml    

** add/edit options (replacing dev.pulp3 with ip address or hostname of pulplift box):

<pre>

:enabled: true

:pulp_url: http://dev.pulp3/pulp/

</pre> 

* restart foreman-proxy:  systemctl restart foreman-proxy

* refresh your smart proxy, from the Infrastructure > Smart Proxies page

* Confirm that you see 'pulp3' show up in the list of features for that smart proxy

h3. Deploying pulp3 on new dev boxes with postgres12:

In forklift, on branch with https://github.com/theforeman/forklift/pull/980 apply patch https://gist.github.com/sjha4/5d4189eae804bd2b3229c10870f56ac2#file-forklift-patch

cd ansible-pulp

Apply patch https://gist.github.com/sjha4/5d4189eae804bd2b3229c10870f56ac2#file-0001-psql12-patch 

vagrant ssh into devel box

run 

sudo ln -s /opt/rh/rh-postgresql12/root/usr/bin/pg_config /usr/sbin/pg_config

sudo ln -s /opt/rh/rh-postgresql12/root/usr/lib64/libpq.so.rh-postgresql12-5 /usr/lib64/libpq.so.rh-postgresql12-5

exit

Run the playbook.

It might fail with postmaster.pid exists. In which case, ssh into dev box. 

sudo systemctl stop *postgresql*

find / -name postmaster.pid

rm -f ^^

re-run the playbook.

Candlepin will probably stop working at this point. To fix this:

sudo vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/pg_hba.conf

in the file set all the auth from peer to trust.

sudo systemctl restart *postgres*

h3. Updating of pulp client binding Gems

Every two weeks on a Friday, a PR should be open against katello to bump the client gems.  A script is provided https://gist.githubusercontent.com/jlsherrill/836268d9ee9b33fc1fff06e9061db4f4/raw/7a1044725b00c44246fd241e9bed644c5fab6321/generate_pulp3_deps.rb to output the relevant requirements that can be put into katello.gemspec.

Example PR: https://github.com/Katello/katello/pull/8294

If there are test failures, investigate and attempt to fix them.  VCR re-recording may be necessary. 

If there is some expected change, this process can be done more often.

We will ask for a volunteer the thursday prior to each friday.  Current Schedule:

Sept 6th

Sept 20th

Oct  4th

Oct 18th