Pulp 3 Integration » History » Revision 40
Revision 39 (Jonathon Turel, 02/20/2020 09:55 PM) → Revision 40/86 (Samir Jha, 02/21/2020 12:10 AM)
h1. Pulp 3 Integration This page is meant to capture links to various resources around pulp3 integration * Initial Integration Document: https://docs.google.com/document/d/1eFM4H9BteA_nTDbUkRJgEiqSttXfAtB9-51a_OuwVJw/edit?usp=sharing * Initial Community Post: https://community.theforeman.org/t/pulp-3-migration/10474 * Youtube Videos: https://www.youtube.com/playlist?list=PL59WSIBtIpv7tbjnsEAWNcdiOKAmjjgqE * Tasks List: https://docs.google.com/spreadsheets/d/1PFx-VAEXlivlqDYR9INMW26AUWtuMS_XtEnwhFVP5Rc/edit?usp=sharing * Main Tracker: https://projects.theforeman.org/issues/26987 h2. Katello Dev box with yum plugin using vagrant: +*Install and enable pulp_rpm plugin*+ * vagrant up katello-dev-box * vagrant ssh katello-dev-box * Execute commands below: <pre> sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel sudo pip3 install scikit-build nose sudo pip3 install pulp-rpm==3.0.0 sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' python3-django-admin migrate --no-input sudo systemctl restart pulpcore-resource-manager pulpcore-content pulpcore-api pulpcore-worker@1 pulpcore-worker@2 </pre> * Refresh the smart proxy # login to katello/foreman # navigate to infrastructure > smart proxies # click 'refresh' in the list of actions beside your smart proxy You should now see file, docker and yum as supported content type under pulpcore in the *Services tab* h2. Katello Dev environment with pulp 3 using ansible playbook: * Install a katello dev environment with forklift * Install the following packages in your dev box: <pre>sudo yum -y install https://people.redhat.com/iballou/libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre> <pre>sudo yum -y install https://people.redhat.com/iballou/python3-libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre> * checkout this forklift pr if its not merged: https://github.com/theforeman/forklift/pull/980 * setup ansible-pulp: <pre> cd forklift git clone https://github.com/pulp/ansible-pulp.git cd ansible-pulp ansible-galaxy install -r requirements.yml -p ./roles ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles </pre> * within forkflift run: <pre> ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-katello-devel </pre> * Refresh the smart proxy: 1. login to katello/foreman 2. navigate to infrastructure > smart proxies 3. click 'refresh' in the list of actions beside your smart proxy You're now rockin' Pulp3 * *Known Issue:* Can't seem to use pulp3 on the box since pulp-api.service fails to start. <pre> sudo chmod -R 777 /etc/pki/pulp/ sudo systemctl restart pulpcore-api.service </pre> * Reset the Katello Pulp 3 Dev Environment From within the Foreman directory: <pre>bundle exec rake katello:reset</pre> h2. Smart Proxy Setup: * Install a smart proxy to point to your pulp3_dev box * ssh into the smart proxy * sudo mkdir /etc/httpd/conf.d/05-foreman-ssl.d * sudo vi /etc/httpd/conf/httpd.conf * Add IncludeOptional "/etc/httpd/conf.d/05-foreman-ssl.d/*.conf to the end of the file and save * In forklift > Run ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-proxy-devel h3. Other Resources * pulp3 gap analysis wiki https://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis * pulp3 gap analysis etherpad https://etherpad.net/p/pulp3_container * pulp3 docker roadmap https://pulp.plan.io/projects/pulp/wiki/Pulp_Docker_Roadmap h3. Old way to deploy pulp3 on other box * <pre>yum install vagrant-sshfs</pre> * Checkout pulplift: https://github.com/pulp/pulplift * Follow pulplift readme instructions, copy the centos7-pulp3-github box definition from vagrant/boxes.d/99-local.yaml.example * vagrant up centos7-pulp3-github * On katello dev env, edit /etc/foreman-proxy/settings.d/pulp3.yml ** add/edit options (replacing dev.pulp3 with ip address or hostname of pulplift box): <pre> :enabled: true :pulp_url: http://dev.pulp3/pulp/ </pre> * restart foreman-proxy: systemctl restart foreman-proxy * refresh your smart proxy, from the Infrastructure > Smart Proxies page * Confirm that you see 'pulp3' show up in the list of features for that smart proxy h3. Deploying pulp3 on new dev boxes with postgres12: In forklift, on branch with https://github.com/theforeman/forklift/pull/980 apply patch https://gist.github.com/sjha4/5d4189eae804bd2b3229c10870f56ac2#file-forklift-patch cd ansible-pulp Apply patch https://gist.github.com/sjha4/5d4189eae804bd2b3229c10870f56ac2#file-0001-psql12-patch vagrant ssh into devel box run sudo ln -s /opt/rh/rh-postgresql12/root/usr/bin/pg_config /usr/sbin/pg_config sudo ln -s /opt/rh/rh-postgresql12/root/usr/lib64/libpq.so.rh-postgresql12-5 /usr/lib64/libpq.so.rh-postgresql12-5 exit Run the playbook. It might fail with postmaster.pid exists. In which case, ssh into dev box. sudo systemctl stop *postgresql* find / -name postmaster.pid rm -f ^^ re-run the playbook. Candlepin will probably stop working at this point. To fix this: sudo vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/pg_hba.conf in the file set all the auth from peer to trust. sudo systemctl restart *postgres* h3. Updating of pulp client binding Gems Every two weeks on a Friday, a PR should be open against katello to bump the client gems. A script is provided https://gist.githubusercontent.com/jlsherrill/836268d9ee9b33fc1fff06e9061db4f4/raw/7a1044725b00c44246fd241e9bed644c5fab6321/generate_pulp3_deps.rb to output the relevant requirements that can be put into katello.gemspec. Example PR: https://github.com/Katello/katello/pull/8294 If there are test failures, investigate and attempt to fix them. VCR re-recording may be necessary. If there is some expected change, this process can be done more often. We will ask for a volunteer the thursday prior to each friday. Current Schedule: Sept 6th Sept 20th Oct 4th Oct 18th