Pulp 3 Integration » History » Revision 61
Revision 60 (Samir Jha, 09/23/2020 07:54 PM) → Revision 61/86 (Samir Jha, 09/23/2020 07:55 PM)
h1. Pulp 3 Integration
This page is meant to capture links to various resources around pulp3 integration
* Initial Integration Document: https://docs.google.com/document/d/1eFM4H9BteA_nTDbUkRJgEiqSttXfAtB9-51a_OuwVJw/edit?usp=sharing
* Initial Community Post: https://community.theforeman.org/t/pulp-3-migration/10474
* Youtube Videos: https://www.youtube.com/playlist?list=PL59WSIBtIpv7tbjnsEAWNcdiOKAmjjgqE
* Tasks List: https://docs.google.com/spreadsheets/d/1PFx-VAEXlivlqDYR9INMW26AUWtuMS_XtEnwhFVP5Rc/edit?usp=sharing
* Main Tracker: https://projects.theforeman.org/issues/26987
h2. Katello Dev box with pulp3:
* vagrant up katello-dev-box
* vagrant ssh katello-dev-box
h2. Adding the yum plugin:
<pre>
sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig3
sudo pip3 install scikit-build nose
sudo pip3 install pulp-rpm==3.0.0
cd /usr/lib/python3.6/site-packages/pulpcore
sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' python3-django-admin migrate --no-input
sudo systemctl restart pulpcore-resource-manager pulpcore-content pulpcore-api pulpcore-worker@1 pulpcore-worker@2
</pre>
h2. Adding the certguard plugin:
<pre>
sudo pip3 install pulp_certguard
sudo pip3 install python-dateutil
sudo pip3 install rhsm
sudo systemctl restart pulpcore*
</pre>
* Refresh the smart proxy
# login to katello/foreman
# navigate to infrastructure > smart proxies
# click 'refresh' in the list of actions beside your smart proxy
* Enable Katello Applicability
# Add the following to ~/foreman/config/settings.plugins.d/katello.yaml:
<pre>:katello_applicability: true</pre>
You should now see file, docker and yum as supported content type under pulpcore in the *Services tab*
h2. Katello Dev box with pulp3 on master:
On a devel box provisioned with steps above, run :
<pre>
sudo pip3 install scikit-build nose
sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig3
sudo pip3 install "git+https://github.com/pulp/pulpcore.git" \
"git+https://github.com/pulp/pulp_file.git" \
"git+https://github.com/pulp/pulp_rpm.git" \
"git+https://github.com/pulp/pulp_container.git" \
"git+https://github.com/pulp/pulp-2to3-migration.git"
cd /tmp
sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' /usr/local/bin/pulpcore-manager migrate
cd ~/foreman
bundle exec rails katello:reset
</pre>
Add the following to your /etc/pulp/settings.py
<pre>
AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']
REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (
'rest_framework.authentication.SessionAuthentication',
'pulpcore.app.authentication.PulpRemoteUserAuthentication'
)
</pre>
h2. Updating to a New Pulp Version in master & nightly:
1. Install new version locally ontop of existing version with a pip3 install command
2. Identify any installer changes, open installer PRs
3. Identify katello updates:
* Updating pulp client gem bindings in katello.gemspec
* re-record vcr cassettes
* Identify any katello code changes needed to fix test cases
* open katello PR with changes
4. Update client binding rpms at foreman-packaging
5. Request rpm builds email satellite6-build@redhat.com or open a pr ourselves: https://github.com/evgeni/pulpcore-packaging#build
h2. Old (ansible-based) way to deploy pulp3:
* Install a katello dev environment with forklift
* Install the following packages in your dev box:
<pre>sudo yum -y install https://people.redhat.com/iballou/libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>
<pre>sudo yum -y install https://people.redhat.com/iballou/python3-libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>
* checkout this forklift pr if its not merged: https://github.com/theforeman/forklift/pull/980
* setup ansible-pulp:
<pre>
cd forklift
git clone https://github.com/pulp/ansible-pulp.git
cd ansible-pulp
ansible-galaxy install -r requirements.yml -p ./roles
ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles
</pre>
* within forkflift run:
<pre>
ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-katello-devel
</pre>
* Refresh the smart proxy:
1. login to katello/foreman
2. navigate to infrastructure > smart proxies
3. click 'refresh' in the list of actions beside your smart proxy
You're now rockin' Pulp3
* *Known Issue:* Can't seem to use pulp3 on the box since pulp-api.service fails to start.
<pre>
sudo chmod -R 777 /etc/pki/pulp/
sudo systemctl restart pulpcore-api.service
</pre>
* Reset the Katello Pulp 3 Dev Environment
From within the Foreman directory:
<pre>bundle exec rake katello:reset</pre>
h2. Smart Proxy Setup:
* Install a smart proxy to point to your pulp3_dev box
* ssh into the smart proxy
* sudo mkdir /etc/httpd/conf.d/05-foreman-ssl.d
* sudo vi /etc/httpd/conf/httpd.conf
* Add IncludeOptional "/etc/httpd/conf.d/05-foreman-ssl.d/*.conf to the end of the file and save
* In forklift > Run ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-proxy-devel
Next update your box following steps to enable pulp-certguard: steps:
<pre>
sudo pip3 install pulp_certguard
sudo pip3 install python-dateutil
sudo pip3 install rhsm
</pre>
Ensure you have dynaconf >=3.1 on your proxy under /usr/local/lib/pulp/lib/python3.6/site-packages and update /etc/pulp/settings.py to add the following 2 settings:
<pre>
AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']
REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (
'rest_framework.authentication.SessionAuthentication',
'pulpcore.app.authentication.PulpRemoteUserAuthentication'
)
</pre>
* sudo systemctl stop pulpcore*
* sudo su - postgres
* dropdb pulp -p 7878
* createdb pulp -p 7878
* exit
* cd /usr/local/lib/pulp/src/pulpcore/
* sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' /usr/local/lib/pulp/bin/django-admin migrate --no-input
* sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' /usr/local/lib/pulp/bin/django-admin reset-admin-password --password password
* Copy /etc/httpd/conf.d/05-foreman-ssl.d/pulpcore.conf from your dev box to proxy
* sudo systemctl restart pulpcore*
h2. Example for querying the Pulp 3 API in a development environment:
<pre>
curl https://`hostname`/pulp/api/v3/status/ --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key
</pre>
h3. Other Resources
* pulp3 gap analysis wiki https://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis
* pulp3 gap analysis etherpad https://etherpad.net/p/pulp3_container
* pulp3 docker roadmap https://pulp.plan.io/projects/pulp/wiki/Pulp_Docker_Roadmap