Plugins » Katello

h1. Pulp 3 Integration

This page is meant to capture links to various resources around pulp3 integration

* Initial Integration Document: https://docs.google.com/document/d/1eFM4H9BteA_nTDbUkRJgEiqSttXfAtB9-51a_OuwVJw/edit?usp=sharing

* Initial Community Post: https://community.theforeman.org/t/pulp-3-migration/10474

* Youtube Videos: https://www.youtube.com/playlist?list=PL59WSIBtIpv7tbjnsEAWNcdiOKAmjjgqE

* Tasks List: https://docs.google.com/spreadsheets/d/1PFx-VAEXlivlqDYR9INMW26AUWtuMS_XtEnwhFVP5Rc/edit?usp=sharing

* Main Tracker: https://projects.theforeman.org/issues/26987

h2. Katello Dev box with pulp3:

* vagrant up katello-dev-box

* vagrant ssh katello-dev-box

h2. Adding the yum plugin:

<pre>

sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig3

sudo pip3 install scikit-build nose

sudo pip3 install pulp-rpm==3.0.0

cd /usr/lib/python3.6/site-packages/pulpcore

sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' python3-django-admin migrate --no-input

sudo systemctl restart pulpcore-resource-manager pulpcore-content pulpcore-api pulpcore-worker@1 pulpcore-worker@2

</pre>

h2. Adding the certguard plugin:

<pre>

sudo pip3 install pulp_certguard

sudo pip3 install python-dateutil

sudo pip3 install rhsm

sudo systemctl restart pulpcore*

</pre>

* Refresh the smart proxy 

# login to katello/foreman

# navigate to infrastructure > smart proxies

# click 'refresh' in the list of actions beside your smart proxy

* Enable Katello Applicability

# Add the following to ~/foreman/config/settings.plugins.d/katello.yaml:

<pre>:katello_applicability: true</pre>

You should now see file, docker and yum as supported content type under pulpcore in the *Services tab*

h2. Katello Dev box with pulp3 on master:

On a devel box provisioned with steps above, run :

<pre>

sudo pip3 install scikit-build nose

sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig3

sudo pip3 install "git+https://github.com/pulp/pulpcore.git" \

  "git+https://github.com/pulp/pulp_file.git" \

  "git+https://github.com/pulp/pulp_rpm.git" \

  "git+https://github.com/pulp/pulp_container.git" \

  "git+https://github.com/pulp/pulp-2to3-migration.git"

cd /tmp

sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py'  /usr/local/bin/pulpcore-manager migrate

cd ~/foreman

bundle exec rails katello:reset

</pre>

Add the following to your /etc/pulp/settings.py

<pre>

AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']

REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (

    'rest_framework.authentication.SessionAuthentication',

    'pulpcore.app.authentication.PulpRemoteUserAuthentication'

)

</pre>

h2. Updating to a New Pulp Version in master & nightly:

1.  Install new version locally ontop of existing version with a pip3 install command

2.  Identify any installer changes, open installer PRs

3.  Identify katello updates:

* Updating pulp client gem bindings in katello.gemspec 

* re-record vcr cassettes

* Identify any katello code changes needed to fix test cases

* open katello PR with changes

4. Update client binding rpms at foreman-packaging

5.  Request rpm builds email satellite6-build@redhat.com or open a pr ourselves: https://github.com/evgeni/pulpcore-packaging#build

h2. Old (ansible-based) way to deploy pulp3:

* Install a katello dev environment with forklift

* Install the following packages in your dev box: 

<pre>sudo yum -y install https://people.redhat.com/iballou/libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>

<pre>sudo yum -y install https://people.redhat.com/iballou/python3-libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>

* checkout this forklift pr if its not merged: https://github.com/theforeman/forklift/pull/980

* setup ansible-pulp:

<pre>

cd forklift

git clone https://github.com/pulp/ansible-pulp.git

cd ansible-pulp

ansible-galaxy install -r requirements.yml -p ./roles

ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles

</pre>

* within forkflift run:

<pre>

ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-katello-devel

</pre>

* Refresh the smart proxy: 

  1. login to katello/foreman

  2. navigate to infrastructure > smart proxies

  3. click 'refresh' in the list of actions beside your smart proxy

You're now rockin' Pulp3

* *Known Issue:* Can't seem to use pulp3 on the box since pulp-api.service fails to start.

<pre>

sudo chmod -R 777 /etc/pki/pulp/

sudo systemctl restart pulpcore-api.service

</pre>

* Reset the Katello Pulp 3 Dev Environment

From within the Foreman directory:

<pre>bundle exec rake katello:reset</pre>

h2. Smart Proxy Setup:

* Install a smart proxy to point to your pulp3_dev box

* ssh into the smart proxy

* sudo mkdir /etc/httpd/conf.d/05-foreman-ssl.d

* sudo vi /etc/httpd/conf/httpd.conf

* Add IncludeOptional "/etc/httpd/conf.d/05-foreman-ssl.d/*.conf to the end of the file and save

* In forklift > Run ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-proxy-devel

Next update your box following steps to enable pulp-certguard:

<pre>

sudo pip3 install pulp_certguard

sudo pip3 install python-dateutil

sudo pip3 install rhsm

</pre>

Ensure you have dynaconf >=3.1 on your proxy under /usr/local/lib/pulp/lib/python3.6/site-packages and update /etc/pulp/settings.py to add the following 2 settings:

<pre>

AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']

REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (

    'rest_framework.authentication.SessionAuthentication',

    'pulpcore.app.authentication.PulpRemoteUserAuthentication'

)

</pre>

* sudo systemctl stop pulpcore*

* sudo su - postgres

* dropdb pulp -p 7878

* createdb pulp -p 7878

* exit

* cd /usr/local/lib/pulp/src/pulpcore/

* sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' /usr/local/lib/pulp/bin/django-admin migrate --no-input

* sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' /usr/local/lib/pulp/bin/django-admin reset-admin-password --password password

* Copy /etc/httpd/conf.d/05-foreman-ssl.d/pulpcore.conf  from your dev box to proxy

* sudo systemctl restart pulpcore*

h2. Example for querying the Pulp 3 API in a development environment:

<pre>

curl https://`hostname`/pulp/api/v3/status/   --cert /etc/pki/katello/certs/pulp-client.crt  --key /etc/pki/katello/private/pulp-client.key

</pre>

h3. Other Resources

* pulp3 gap analysis wiki https://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis

* pulp3 gap analysis etherpad https://etherpad.net/p/pulp3_container

* pulp3 docker roadmap https://pulp.plan.io/projects/pulp/wiki/Pulp_Docker_Roadmap