Project

General

Profile

Pulp 3 Integration » History » Version 53

James Jeffers, 04/15/2020 02:48 PM

1 1 Justin Sherrill
h1. Pulp 3 Integration
2
3
This page is meant to capture links to various resources around pulp3 integration
4
5
* Initial Integration Document: https://docs.google.com/document/d/1eFM4H9BteA_nTDbUkRJgEiqSttXfAtB9-51a_OuwVJw/edit?usp=sharing
6
* Initial Community Post: https://community.theforeman.org/t/pulp-3-migration/10474
7
* Youtube Videos: https://www.youtube.com/playlist?list=PL59WSIBtIpv7tbjnsEAWNcdiOKAmjjgqE
8
* Tasks List: https://docs.google.com/spreadsheets/d/1PFx-VAEXlivlqDYR9INMW26AUWtuMS_XtEnwhFVP5Rc/edit?usp=sharing
9 17 John Mitsch
* Main Tracker: https://projects.theforeman.org/issues/26987
10 2 Justin Sherrill
11 41 Justin Sherrill
h2. Katello Dev box with pulp3:
12 40 Samir Jha
13
* vagrant up katello-dev-box
14 1 Justin Sherrill
* vagrant ssh katello-dev-box
15
16 42 Justin Sherrill
h2. Adding the yum plugin:
17 40 Samir Jha
<pre>
18 53 James Jeffers
sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig3
19 40 Samir Jha
sudo pip3 install scikit-build nose
20 1 Justin Sherrill
sudo pip3 install pulp-rpm==3.0.0
21 50 Samir Jha
cd /usr/lib/python3.6/site-packages/pulpcore
22 40 Samir Jha
sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' python3-django-admin migrate --no-input
23
sudo systemctl restart pulpcore-resource-manager pulpcore-content pulpcore-api pulpcore-worker@1 pulpcore-worker@2
24
</pre>
25
26
* Refresh the smart proxy 
27
# login to katello/foreman
28
# navigate to infrastructure > smart proxies
29
# click 'refresh' in the list of actions beside your smart proxy
30
31 45 Samir Jha
You should now see file, docker and yum as supported content type under pulpcore in the *Services tab*
32 40 Samir Jha
33 44 Samir Jha
h2. Katello Dev box with pulp3 on master:
34 1 Justin Sherrill
35 44 Samir Jha
On a devel box provisioned with steps above, run :
36
37
<pre>
38
cd /usr/lib/python3.6/site-packages/
39
sudo pip3 install "git+https://github.com/pulp/pulpcore.git"
40
sudo pip3 install scikit-build nose
41 52 James Jeffers
sudo yum install -y gcc make cmake bzip2-devel expat-devel file-devel glib2-devel libcurl-devel libmodulemd2-devel ninja-build libxml2-devel python36-devel python36-gobject rpm-devel openssl-devel sqlite-devel xz-devel zchunk-devel zlib-devel swig swig3
42 44 Samir Jha
sudo pip3 install "git+https://github.com/pulp/pulp_file.git"
43
sudo pip3 install "git+https://github.com/pulp/pulp_rpm.git"
44 1 Justin Sherrill
sudo pip3 install "git+https://github.com/pulp/pulp_container.git"
45 45 Samir Jha
sudo pip3 install "git+https://github.com/pulp/pulp-2to3-migration.git"
46 44 Samir Jha
sudo rm -rf pulpcore*
47
sudo rm -rf pulp_*
48 47 Samir Jha
sudo rm -rf redis*
49 46 Samir Jha
sudo mv /usr/local/lib/python3.6/site-packages/pulp* /usr/lib/python3.6/site-packages/
50 47 Samir Jha
sudo mv /usr/local/lib/python3.6/site-packages/redis* /usr/lib/python3.6/site-packages/
51 44 Samir Jha
cd ~/foreman
52
bundle exec rails katello:reset
53
</pre>
54 40 Samir Jha
55 48 Samir Jha
56
Add the following to your /etc/pulp/settings.py
57
58
<pre>
59
AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']
60
61
REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (
62
    'rest_framework.authentication.SessionAuthentication',
63
    'pulpcore.app.authentication.PulpRemoteUserAuthentication'
64
)
65
66
</pre>
67
68
69 49 Justin Sherrill
h2. Updating to a New Pulp Version in master & nightly:
70
71
1.  Install new version locally ontop of existing version with a pip3 install command
72
2.  Identify any installer changes, open installer PRs
73
3.  Identify katello updates:
74
* Updating pulp client gem bindings in katello.gemspec 
75
* re-record vcr cassettes
76
* Identify any katello code changes needed to fix test cases
77
* open katello PR with changes
78
79
4. Update client binding rpms at foreman-packaging
80
5.  Request rpm builds email satellite6-build@redhat.com or open a pr ourselves: https://github.com/evgeni/pulpcore-packaging#build
81
82
83
84
85
86 48 Samir Jha
87
88 41 Justin Sherrill
h2. Old (ansible-based) way to deploy pulp3:
89 2 Justin Sherrill
90 15 Justin Sherrill
* Install a katello dev environment with forklift
91 32 Ian Ballou
* Install the following packages in your dev box: 
92 34 Ian Ballou
<pre>sudo yum -y install https://people.redhat.com/iballou/libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>
93
<pre>sudo yum -y install https://people.redhat.com/iballou/python3-libcomps-0.1.12-1.git.0.937a666.el7.x86_64.rpm</pre>
94 15 Justin Sherrill
* checkout this forklift pr if its not merged: https://github.com/theforeman/forklift/pull/980
95
* setup ansible-pulp:
96
<pre>
97
cd forklift
98
git clone https://github.com/pulp/ansible-pulp.git
99 1 Justin Sherrill
cd ansible-pulp
100 15 Justin Sherrill
ansible-galaxy install -r requirements.yml -p ./roles
101 29 Justin Sherrill
ansible-galaxy install pulp.pulp_rpm_prerequisites -p ./roles
102 15 Justin Sherrill
</pre>
103 24 Samir Jha
104
105 15 Justin Sherrill
* within forkflift run:
106
<pre>
107
ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-katello-devel
108
</pre>
109 1 Justin Sherrill
110 15 Justin Sherrill
* Refresh the smart proxy: 
111
112
  1. login to katello/foreman
113
  2. navigate to infrastructure > smart proxies
114
  3. click 'refresh' in the list of actions beside your smart proxy
115
You're now rockin' Pulp3
116 18 Partha Aji
117
118 22 Samir Jha
* *Known Issue:* Can't seem to use pulp3 on the box since pulp-api.service fails to start.
119
<pre>
120
sudo chmod -R 777 /etc/pki/pulp/
121 30 John Mitsch
sudo systemctl restart pulpcore-api.service
122 22 Samir Jha
</pre>
123 18 Partha Aji
124 33 Chris Roberts
* Reset the Katello Pulp 3 Dev Environment
125
From within the Foreman directory:
126
<pre>bundle exec rake katello:reset</pre>
127
128 23 Samir Jha
h2. Smart Proxy Setup:
129
130
* Install a smart proxy to point to your pulp3_dev box
131
* ssh into the smart proxy
132
* sudo mkdir /etc/httpd/conf.d/05-foreman-ssl.d
133
* sudo vi /etc/httpd/conf/httpd.conf
134
* Add IncludeOptional "/etc/httpd/conf.d/05-foreman-ssl.d/*.conf to the end of the file and save
135
* In forklift > Run ansible-playbook ./playbooks/katello_pulp3.yml -l centos7-proxy-devel
136
137 25 Justin Sherrill
h3. Other Resources
138
139
* pulp3 gap analysis wiki https://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis
140
* pulp3 gap analysis etherpad https://etherpad.net/p/pulp3_container
141
* pulp3 docker roadmap https://pulp.plan.io/projects/pulp/wiki/Pulp_Docker_Roadmap