Project

General

Profile

PulpV3GapAnalysis » History » Version 6

Brian Bouterse, 06/07/2018 06:51 PM
Notes from today's meeting

1 1 Brian Bouterse
h1. PulpV3GapAnalysis
2
3
h1.  Content Tab
4
5
h2.  Content -> Red Hat Repositories
6
7
Katello knows the content URLs from candlepin, matches on the CDN, presents them to the user, the user selects them
8
* Katello creates a Repo tracking this in Pulp with client certificates and CA certificate
9
* Katello specifies custom options from the 'Custom Repo Creation Page' but these use cases are covered in that section
10
11
Katello deletes a Repository
12
13
h2.  Content -> Products
14
15
h3.  Content -> Products -> New Product (used for things like CentOS, SLES, etc)
16
17
All data here is stored only in Katello since this is a Product not a Repository and Pulp doesn't have a concept of a Product
18
Sync Plans will *not* be handled inside of Pulp
19
20
h3.  Content -> Products -> {product_name} -> Repositories
21
22
The user selects a type and content-specific fields are shown.
23
24
h4. Debian:
25
26
h5. Sync Options
27
28
* Upstream URL (str)
29
* Releases (csv list)
30
* Components (csv list)
31
* Architectures (csv list)
32
* Verify SSL (boolean)
33
* Upstream username (str)
34
* Upstream password (str)
35
* Ignore Global http Proxy (bool)
36
37
h5. Publish Options
38
39
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
40
41
42
h4. Docker
43
44
* Sync Options
45
46
* Ustream URL (str)
47
48
* Upstream Repository Name (str)
49
50
* Verify SSL (bool)
51
52
* Upstream username (str)
53
54
* Upstream password (str)
55
56
* Ignore Global http Proxy (bool)
57
58
59
h4. File
60
61
h5. Sync Options
62
63
* Upstream URL (str)
64
65
* Verify SSL (boolean)
66
67
* Upstream username (str)
68
69
* Upstream password (str)
70
71
* Ignore Global http Proxy (bool)
72
73
h5. Publish Options
74
75
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
76
77
78
h4. OSTree
79
80
h5. Sync Options
81
82
* Upstream URL (str)
83
84
* Upstream Sync Policy (choice):  Latest Only, All History, Custom Depth (with a number specified)    <--- in Pulp2 also specified on distributor
85
86
* Verify SSL (boolean)
87
88
* Upstream username (str)
89
90
* Upstream password (str)
91
92
* Ignore Global http Proxy (bool)
93
94
95
h4. Puppet
96
97
h5. Sync Options
98
99
* Upstream URL (str)
100
101
* Verify SSL (boolean)
102
103
* Upstream username (str)
104
105
* Upstream password (str)
106
107
* Mirror on Sync (boolean)
108
109
* Ignore Global http Proxy (bool)
110
111
h5. Publish Options
112
113
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
114
115
116
h4. Yum
117
118
h5. General Fields   <------ not used by Pulp
119
120
* Restrict to Architecture (choice)
121
122
* GPG Key (str)  
123
124
h5. Sync Settings
125
126
* Upstream URL (str)
127
128
* Ignorable Content (multiselect): RPM, DRPM, SRPM, Errata, Distribution
129
130
* Verify SSL (boolean)
131
132
* Upstream username (str)
133
134
* Upstream password (str)
135
136
* Download Policy (choice): (On Demand, Background, Immediate)   <---- Background does not have a strong use case
137
138
* Mirror on Sync (bool)
139
140
* Ignore Global http Proxy (bool)
141
142
* SSL CA Cert (str)
143
144
* SSL Client Cert (str)
145
146
* SSL Client Key(str)
147
148
h5. Publish Settings
149
150
* Checksum: (choice) Default, sha256, sha1   <----- for all repodata including primary.xml
151
152
153 3 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories -> {repository_name}
154
155
This displays a created repository.
156
157
Katello allows the user to upload a package
158
159
* Receives the data from the user, sends it to Pulp
160
161 4 Brian Bouterse
* Relies on Pulp to fully parse the metadata and create the unit                <------- REQUIREMENT: must have Pulp determine all metadata
162 3 Brian Bouterse
163
* Associates the the unit with the repository
164
165
Katello Reads a content Summary on this page
166
167 4 Brian Bouterse
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Sync Now
168 1 Brian Bouterse
169 4 Brian Bouterse
Katello tells the remote associated with the repository to sync
170 3 Brian Bouterse
171 4 Brian Bouterse
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Advaced Sync
172 1 Brian Bouterse
173
Katello can peroform an 'Advnaced Sync':
174 3 Brian Bouterse
175 4 Brian Bouterse
Optimized Sync - Normal sync, presented
176 3 Brian Bouterse
177 4 Brian Bouterse
Complete Sync - force-full on sync and force-full on publish      <--------------------- GAP because we don't have force-full
178 1 Brian Bouterse
179 4 Brian Bouterse
Validate Content Sync - performs a checksum validation on all packages
180 1 Brian Bouterse
181 4 Brian Bouterse
* True Purpose:  Validate existing downloaded content and redownload if the file(s) are missing or corrupt, redownload them.    <-------- GAP
182 1 Brian Bouterse
183 4 Brian Bouterse
184
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Republish Repository Metadata
185
186
Republishes the metadata.
187
188
* Katello would create a new Publication and update the Distribution
189
190
191
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Delete a Repository
192
193
Deletes a repository
194
195
196 3 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories
197
198
This is the index view of all repositories
199
200
Repsitories in Katello can have the same name, but Pulp enforces a unique name on repositories globally      <--------- GAP
201
202
Katello takes a Product ID which resolves to a set of repos. Katello fetches this set of repos. For each repo we need to fetch:
203
204
* name (str)
205
206
* type (str), e.g. 'yum'
207
208
* sync status, e.g. 'Not synced, Pending, Error'     <-------------------------  GAP this would require a second call to load the data per Remote
209
210
* Content Summary, e.g. 2 packages, 5 errata, etc. Similarly for other types.
211
212
213
Katello can trigger a sync of one or more Repositories at once.
214
215
* Trigger the sync on one or more Remotes as independant calls
216
217
218
Katello can trigger a delete of one or more Repositories at once.
219
220
* Trigger the delete call to Pulp as independant calls
221
222
223
Search/Filtering of the list of Repositories, for Repository attributes
224
225
* content_type: the type of content
226
227
* content_view_id: the id of the content View              <-------- not in Pulp anywhere currently
228
229
* ignore_global_proxy                      <--------- GAP area, not currently in Pulp, but probably should be
230
231
* name
232
233
* product
234
235
* redhat          <----------   Anything added from Red Hat "Products" page in Katello gets Red Hat.
236
237
238
Search/Filtering of the list of Repositories, for content units
239
240
* distribution_arch: 
241
242
* distribution_bootable       <-----------    if Katello can detect if it has a vmlinuz init.rd it knows the distribution is bootable. Detected at the end of every sync.
243 1 Brian Bouterse
244 3 Brian Bouterse
* distribution_family
245
246
* distribution_uuid
247
248 1 Brian Bouterse
* distribution_variant
249
250 4 Brian Bouterse
* distribution_version
251 1 Brian Bouterse
252
253
*NOTE: Must not have to make a call for each item in a list page. Must be able to make one call.*
254
255
256 4 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Packages
257 1 Brian Bouterse
258 4 Brian Bouterse
Lists packages in a repository (the latest repository version)
259 1 Brian Bouterse
260 4 Brian Bouterse
Removing packages from the repository
261 1 Brian Bouterse
262 4 Brian Bouterse
* Can remove n packages from the repository
263
* Republish, Redistribute the repository
264
265
266
h2. Content -> Content Credentials
267
268
h3. Content -> Content Credentials -> GPG Keys
269
270
GPG keys can be created and stored by Katello
271
Pulp3 recommendation is to use pulp_file to hold the GPG keys hosted for clients to receive
272
273
274
h3. Content -> Content Credentials -> SSL Certificate  (GAP. This whole section is a GAP b/c Pulp doesn't "host" SSL certs, you have to manually install them on the filesystem first)
275
276
Stores SSL certificates for use by Pulp at sync time as CA cert, client cert, or client key
277
278
* name
279
* value
280
281
Supports updating them
282
Support deleting them
283
Support searching them (name, organization_id)
284
SSL Certs are per-product, so Katello needs some way to restrict the set of available SSL certs for the current "product"
285
286
287
h2. Content -> Sync Plans
288
289
Sync plans will not be handled by Pulp 3, Katello/Foreman will handle scheduling.
290
291
292
h2. Content -> Sync Status
293
294
Show the most-recent sync status from dynflow data. That data is populated by task status results from Pulp, which needs to contain at a minimum:
295
296
* start time
297
* create time
298
* end time
299
* state
300
* progress reports
301
* fatal errors
302
* non-fatal errors
303 1 Brian Bouterse
304
305
h2.  Content -> Lifecycle
306
307
h2.  Content -> Lifecycle Environments
308 5 Brian Bouterse
309
Creates a lifecycle environment
310
311
* Does *not* involve Pulp
312
313
314
h3. Content -> Lifecycle Environments -> {name} -> Details
315
316
Each lifecycle environment has a 'Registry Name Pattern'.      <------- GAP (specific to Docker only)
317
318
* Likely going to be on the Distributor
319
320
* Katello would use the template to produce a concrete value to set on the Distributor
321
322
* Important to ensure that two Distribution don't both receive the same concrete values
323
324
325
h3. Content -> Lifecycle Environments -> {name} -> Content Views
326
327
Filterable by:
328
329
* composite
330
* label
331
* name
332
* organization_id
333
334
335
h3. Content -> Lifecycle Environments -> {name} -> Yum Repositories
336
337
Content will come from CV section on Yum Repositories
338
339
340
h3. Content -> Lifecycle Environments -> {name} -> Errata
341
342
Content will come from CV section on Errata
343
344
345
h3. Content -> Lifecycle Environments -> {name} -> Packages
346
347
Content will come from CV section on Packages
348
349
350
h3. Content -> Lifecycle Environments -> {name} -> Puppet Modules
351
352
Content will come from CV section on Puppet Modules
353
354
355
h3. Content -> Lifecycle Environments -> {name} -> Container Image Tags
356
357
Content will come from CV section on Container Image Tags
358
359
360
h3. Content -> Lifecycle Environments -> {name} -> OSTree Branches
361
362
Content will come from CV section on OSTree Branches
363
364 6 Brian Bouterse
h2.  Content -> Content Views
365 1 Brian Bouterse
366 6 Brian Bouterse
h3. Content -> Content Views -> {name} -> Yum Repositories
367 1 Brian Bouterse
368 6 Brian Bouterse
List/Remove/Add one or more repositories to the Content View
369
* Does *not* involve Pulp
370 1 Brian Bouterse
371 6 Brian Bouterse
372
h3. Content -> Content Views -> {name} -> Yum Filters
373
374
h5. Package Filters
375
376
* Select RPMs using include or exclude filters to be included/disincluded from the content view.
377
* package name. Also supports wildcard. - an attribute of the metadata
378
* architecture. An attribute of the metadata
379
* version, lt, gt, range, etc. An attribute of the metadata
380
381
Checkbox with 'include all RPMs with no errata'. Solves a practical issue whereby packages that received no errata are not included in a content view when the user applies a filter that only includes packages referenced as errata.
382
383
384
h5. Package Group Filter
385
386
Select package groups to include or remove rpms
387
* name - an attribute of the metadata
388
* product - the katello stored attribute
389
* repository - the repo containing that unit
390
* description - an attribute of the metadata
391
392
393
h5. Errata by ID Filter                                                                                 <------ GAP: must be able to ask Pulp filter info and exclude 
394
395
filterable on errata attributes
396
* type (multiselect) i.e. security, enhancement, bugfix
397
* date either or choice:  i.e. updated on, Issued on w/ start/end date
398
* bug
399
* cve
400
* id
401
* issued
402
* package
403
* package_name
404
* reboot_suggested
405
* severity
406
* title
407
* type
408
* updated
409 1 Brian Bouterse
410
h2.  Content -> Activation Keys
411
412
413
h2.  Content -> Content Types
414
415
416
h2.  Content -> Deb Packages
417
418
419
h2.  Content -> Container Image Tags
420
421
422
h2.  Content -> Errata
423
424
425
h2.  Content -> Files
426
427
428
h2.  Content -> OSTree Branches
429
430
431
h2.  Content -> Packages
432
433
434
h2.  Content -> Puppet Modules
435
436
437
438
h1.  Hosts -> Content Hosts
439
440
441
h1.  Non UI things
442
443
* the API endpoint that clients upload their enabled repos
444
* the API endpoint that clients upload their package profiles
445
* the API endpoint that clients register
446
* the API endpoint that clients unregister
447
* speed throttling and other global settings?
448 2 Justin Sherrill
* Errata mailer
449
* smart proxy page/details
450 1 Brian Bouterse
451
452
453
h1.  Terminology
454
455
Candlepin Manifest - Defines Products, Subscriptions, and a Content Sets
456
Product - A collection of repositories. A repository can only belong to one product
457
Repository Set - Has a name, Label, and URL of the form:  /content/rhel/server/7/$RELVER/$BASEARCH/os/