Project

General

Profile

PulpV3GapAnalysis » History » Version 17

Justin Sherrill, 06/15/2018 03:33 PM

1 1 Brian Bouterse
h1. PulpV3GapAnalysis
2
3
h1.  Content Tab
4
5
h2.  Content -> Red Hat Repositories
6
7
Katello knows the content URLs from candlepin, matches on the CDN, presents them to the user, the user selects them
8
* Katello creates a Repo tracking this in Pulp with client certificates and CA certificate
9
* Katello specifies custom options from the 'Custom Repo Creation Page' but these use cases are covered in that section
10
11
Katello deletes a Repository
12
13
h2.  Content -> Products
14
15
h3.  Content -> Products -> New Product (used for things like CentOS, SLES, etc)
16
17
All data here is stored only in Katello since this is a Product not a Repository and Pulp doesn't have a concept of a Product
18
Sync Plans will *not* be handled inside of Pulp
19
20
h3.  Content -> Products -> {product_name} -> Repositories
21
22
The user selects a type and content-specific fields are shown.
23
24
h4. Debian:
25
26
h5. Sync Options
27
28
* Upstream URL (str)
29
* Releases (csv list)
30
* Components (csv list)
31
* Architectures (csv list)
32
* Verify SSL (boolean)
33
* Upstream username (str)
34
* Upstream password (str)
35
* Ignore Global http Proxy (bool)
36
37
h5. Publish Options
38
39
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
40
41
42
h4. Docker
43
44
* Sync Options
45
46
* Ustream URL (str)
47
48
* Upstream Repository Name (str)
49
50
* Verify SSL (bool)
51
52
* Upstream username (str)
53
54
* Upstream password (str)
55
56
* Ignore Global http Proxy (bool)
57
58
59
h4. File
60
61
h5. Sync Options
62
63
* Upstream URL (str)
64
65
* Verify SSL (boolean)
66
67
* Upstream username (str)
68
69
* Upstream password (str)
70
71
* Ignore Global http Proxy (bool)
72
73
h5. Publish Options
74
75
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
76
77
78
h4. OSTree
79
80
h5. Sync Options
81
82
* Upstream URL (str)
83
84
* Upstream Sync Policy (choice):  Latest Only, All History, Custom Depth (with a number specified)    <--- in Pulp2 also specified on distributor
85
86
* Verify SSL (boolean)
87
88
* Upstream username (str)
89
90
* Upstream password (str)
91
92
* Ignore Global http Proxy (bool)
93
94
95
h4. Puppet
96
97
h5. Sync Options
98
99
* Upstream URL (str)
100
101
* Verify SSL (boolean)
102
103
* Upstream username (str)
104
105
* Upstream password (str)
106
107
* Mirror on Sync (boolean)
108
109
* Ignore Global http Proxy (bool)
110
111
h5. Publish Options
112
113
* Publish via HTTP (bool)       <----------------------------- PROBLEM AREA
114
115
116
h4. Yum
117
118
h5. General Fields   <------ not used by Pulp
119
120
* Restrict to Architecture (choice)
121
122
* GPG Key (str)  
123
124
h5. Sync Settings
125
126
* Upstream URL (str)
127
128
* Ignorable Content (multiselect): RPM, DRPM, SRPM, Errata, Distribution
129
130
* Verify SSL (boolean)
131
132
* Upstream username (str)
133
134
* Upstream password (str)
135
136
* Download Policy (choice): (On Demand, Background, Immediate)   <---- Background does not have a strong use case
137
138
* Mirror on Sync (bool)
139
140
* Ignore Global http Proxy (bool)
141
142
* SSL CA Cert (str)
143
144
* SSL Client Cert (str)
145
146
* SSL Client Key(str)
147
148
h5. Publish Settings
149
150
* Checksum: (choice) Default, sha256, sha1   <----- for all repodata including primary.xml
151
152
153 3 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories -> {repository_name}
154
155
This displays a created repository.
156
157
Katello allows the user to upload a package
158
159
* Receives the data from the user, sends it to Pulp
160
161 4 Brian Bouterse
* Relies on Pulp to fully parse the metadata and create the unit                <------- REQUIREMENT: must have Pulp determine all metadata
162 3 Brian Bouterse
163
* Associates the the unit with the repository
164
165
Katello Reads a content Summary on this page
166
167 4 Brian Bouterse
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Sync Now
168 1 Brian Bouterse
169 4 Brian Bouterse
Katello tells the remote associated with the repository to sync
170 3 Brian Bouterse
171 4 Brian Bouterse
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Advaced Sync
172 1 Brian Bouterse
173
Katello can peroform an 'Advnaced Sync':
174 3 Brian Bouterse
175 4 Brian Bouterse
Optimized Sync - Normal sync, presented
176 3 Brian Bouterse
177 4 Brian Bouterse
Complete Sync - force-full on sync and force-full on publish      <--------------------- GAP because we don't have force-full
178 1 Brian Bouterse
179 4 Brian Bouterse
Validate Content Sync - performs a checksum validation on all packages
180 1 Brian Bouterse
181 4 Brian Bouterse
* True Purpose:  Validate existing downloaded content and redownload if the file(s) are missing or corrupt, redownload them.    <-------- GAP
182 1 Brian Bouterse
183 4 Brian Bouterse
184
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Republish Repository Metadata
185
186
Republishes the metadata.
187
188
* Katello would create a new Publication and update the Distribution
189
190
191
h5.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Select Action -> Delete a Repository
192
193
Deletes a repository
194
195
196 3 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories
197
198
This is the index view of all repositories
199
200
Repsitories in Katello can have the same name, but Pulp enforces a unique name on repositories globally      <--------- GAP
201
202
Katello takes a Product ID which resolves to a set of repos. Katello fetches this set of repos. For each repo we need to fetch:
203
204
* name (str)
205
206
* type (str), e.g. 'yum'
207
208
* sync status, e.g. 'Not synced, Pending, Error'     <-------------------------  GAP this would require a second call to load the data per Remote
209
210
* Content Summary, e.g. 2 packages, 5 errata, etc. Similarly for other types.
211
212
213
Katello can trigger a sync of one or more Repositories at once.
214
215
* Trigger the sync on one or more Remotes as independant calls
216
217
218
Katello can trigger a delete of one or more Repositories at once.
219
220
* Trigger the delete call to Pulp as independant calls
221
222
223
Search/Filtering of the list of Repositories, for Repository attributes
224
225
* content_type: the type of content
226
227
* content_view_id: the id of the content View              <-------- not in Pulp anywhere currently
228
229
* ignore_global_proxy                      <--------- GAP area, not currently in Pulp, but probably should be
230
231
* name
232
233
* product
234
235
* redhat          <----------   Anything added from Red Hat "Products" page in Katello gets Red Hat.
236
237
238
Search/Filtering of the list of Repositories, for content units
239
240
* distribution_arch: 
241
242
* distribution_bootable       <-----------    if Katello can detect if it has a vmlinuz init.rd it knows the distribution is bootable. Detected at the end of every sync.
243 1 Brian Bouterse
244 3 Brian Bouterse
* distribution_family
245
246
* distribution_uuid
247
248 1 Brian Bouterse
* distribution_variant
249
250 4 Brian Bouterse
* distribution_version
251 1 Brian Bouterse
252
253
*NOTE: Must not have to make a call for each item in a list page. Must be able to make one call.*
254
255
256 4 Brian Bouterse
h3.  Content -> Products -> {product_name} -> Repositories -> {repository_name} -> Packages
257 1 Brian Bouterse
258 4 Brian Bouterse
Lists packages in a repository (the latest repository version)
259 1 Brian Bouterse
260 4 Brian Bouterse
Removing packages from the repository
261 1 Brian Bouterse
262 4 Brian Bouterse
* Can remove n packages from the repository
263
* Republish, Redistribute the repository
264
265
266
h2. Content -> Content Credentials
267
268
h3. Content -> Content Credentials -> GPG Keys
269
270
GPG keys can be created and stored by Katello
271
Pulp3 recommendation is to use pulp_file to hold the GPG keys hosted for clients to receive
272
273
274
h3. Content -> Content Credentials -> SSL Certificate  (GAP. This whole section is a GAP b/c Pulp doesn't "host" SSL certs, you have to manually install them on the filesystem first)
275
276
Stores SSL certificates for use by Pulp at sync time as CA cert, client cert, or client key
277
278
* name
279
* value
280
281
Supports updating them
282
Support deleting them
283
Support searching them (name, organization_id)
284
SSL Certs are per-product, so Katello needs some way to restrict the set of available SSL certs for the current "product"
285
286
287
h2. Content -> Sync Plans
288
289
Sync plans will not be handled by Pulp 3, Katello/Foreman will handle scheduling.
290
291
292
h2. Content -> Sync Status
293
294
Show the most-recent sync status from dynflow data. That data is populated by task status results from Pulp, which needs to contain at a minimum:
295
296
* start time
297
* create time
298
* end time
299
* state
300
* progress reports
301
* fatal errors
302
* non-fatal errors
303 1 Brian Bouterse
304
h2.  Content -> Lifecycle Environments
305 5 Brian Bouterse
306
Creates a lifecycle environment
307
308
* Does *not* involve Pulp
309
310
311
h3. Content -> Lifecycle Environments -> {name} -> Details
312
313
Each lifecycle environment has a 'Registry Name Pattern'.      <------- GAP (specific to Docker only)
314
315
* Likely going to be on the Distributor
316
317
* Katello would use the template to produce a concrete value to set on the Distributor
318
319
* Important to ensure that two Distribution don't both receive the same concrete values
320
321
322
h3. Content -> Lifecycle Environments -> {name} -> Content Views
323
324
Filterable by:
325
326
* composite
327
* label
328
* name
329
* organization_id
330
331
332
h3. Content -> Lifecycle Environments -> {name} -> Yum Repositories
333
334
Content will come from CV section on Yum Repositories
335
336
337
h3. Content -> Lifecycle Environments -> {name} -> Errata
338
339
Content will come from CV section on Errata
340
341
342
h3. Content -> Lifecycle Environments -> {name} -> Packages
343
344
Content will come from CV section on Packages
345
346
347
h3. Content -> Lifecycle Environments -> {name} -> Puppet Modules
348
349
Content will come from CV section on Puppet Modules
350
351
352
h3. Content -> Lifecycle Environments -> {name} -> Container Image Tags
353
354
Content will come from CV section on Container Image Tags
355
356
357
h3. Content -> Lifecycle Environments -> {name} -> OSTree Branches
358
359
Content will come from CV section on OSTree Branches
360
361 6 Brian Bouterse
h2.  Content -> Content Views
362 1 Brian Bouterse
363 6 Brian Bouterse
h3. Content -> Content Views -> {name} -> Yum Repositories
364 1 Brian Bouterse
365 6 Brian Bouterse
List/Remove/Add one or more repositories to the Content View
366
* Does *not* involve Pulp
367 1 Brian Bouterse
368 6 Brian Bouterse
h3. Content -> Content Views -> {name} -> Yum Filters
369
370 9 Brian Bouterse
Katello filters combine together (whitelist/blacklist/etc), and can be heavily modified by users to ultimately produce a set of packages.    <------GAP:  Katello would have to store huge lists of packages/errata to maintain this design.
371
372
373
h5. Package Filters
374
375
* Select RPMs using include or exclude filters to be included/disincluded from the content view.
376
* package name. Also supports wildcard. - an attribute of the metadata
377
* architecture. An attribute of the metadata
378
* version, lt, gt, range, etc. An attribute of the metadata
379
380
Checkbox with 'include all RPMs with no errata'. Solves a practical issue whereby packages that received no errata are not included in a content view when the user applies a filter that only includes packages referenced as errata.
381
382
383
h5. Package Group Filter
384
385
Select package groups to include or remove rpms
386
* name - an attribute of the metadata
387
* product - the katello stored attribute
388
* repository - the repo containing that unit
389
* description - an attribute of the metadata
390
391
392
h5. Errata by ID Filter                                                                                 <------ GAP: must be able to ask Pulp filter info and exclude 
393
394
Filters to produce a list and then you can select from the list.
395
396
filterable on errata attributes
397
* type (multiselect) i.e. security, enhancement, bugfix
398
* date either or choice:  i.e. updated on, Issued on w/ start/end date
399
* bug
400
* cve
401
* id
402
* issued
403
* package
404
* package_name
405
* reboot_suggested
406
* severity
407
* title
408
* type
409
* updated
410
411
412
h5. Errata by Date
413
414
Filters to produce a list. You *cannot* select from the list.
415
416
filterable on errata attributes
417
* type (multiselect) i.e. security, enhancement, bugfix
418
* date either or choice:  i.e. updated on, Issued on w/ start/end date
419
420
421
h3. Content -> Content Views -> {name} -> Apt Repositories
422
423
No filtering. Add/remove Debian repositories from the content view.
424
425
426
h3. Content -> Content Views -> {name} ->  File Repositories
427
428
No filtering. Add/remove file repositories from the content view.
429
430
431
h3. Content -> Content Views -> {name} -> Puppet Modules
432
433
Each module can only be included once. Can't have 2+ versions of the same module in one content view.
434
435
Attributes:
436
437
* Name
438
* Author
439
* Version
440
441
442
h3. Content -> Content Views -> {name} ->  Container Images
443
444
Filtering is 'tag' based and used to produce a concrete set of image names.
445
446
447
h3. Content -> Content Views -> {name} -> OSTree Content
448
449
No filtering. Add/remove ostree repositories from the content view.
450
451
452
h3. Content -> Content Views -> {name} -> History
453
454
Not related to Pulp.
455
456
457
h3. Content -> Content Views -> {name} -> Tasks
458
459
Not related to Pulp.
460
461
462
h2. Content -> Content Views -> {name} -> Publish
463
464
The concrete content set from all filters is computed and those units are associated with the CV repositories.
465
Those content view repositories are then published via the Distributions that host Library.
466
467
468
h5. Promotion
469
470
Other "promotion" events will cause existing Publications to be exposed via existing/new Distributions associated with the lifecycle environment.
471
472
When delivering content to a capsule the "Force Yum Metadata Regernation" is used to cause Katello to inspect the published times of the repo on the main satellite server and the capsule. If nothing changed then don't "resync" the capsule's repo.
473
474
475
h5. Regenerate Repository Metadata
476
477
Causes Pulp to force-full publish.
478
479
480
h5. Incremental Update
481
482
Take an existing Content View and add/remove packages and errata w/ dependency resolution from the content set. Say a V1 exists, this would create a V1.1.
483
484
485
h5. Remove
486
487
Un-distributes and potentially delete the publications for one or more repos backing the content views.
488
489 1 Brian Bouterse
h2.  Content -> Deb Packages
490
491 12 Brian Bouterse
List:
492
493
* Name
494
* Version
495
* Architecture
496
497
Filter options:
498
    
499
* architecture
500
* checksum
501
* filename
502
* name
503
* version
504
505
506
h5. Details tab
507
508
Details of the Debian package
509
510
511
h5. Repositories tab
512
513
List the Debian repositories containing the package
514
515
Filtering by:
516
517
* Lifecycle Environment
518
* Organization
519 1 Brian Bouterse
520 13 Brian Bouterse
h2. Content -> Container Image Tags
521
522
List:
523
    
524
* name
525
* available schema versions
526
* product name
527
* repository name
528
529
h3. Content -> Container Image Tags -> {name}
530
531
Display info about a tag
532
533
Displays Container Image Management.
534
535
Manifest type
536
checksum
537
538
539
h5. Lifecycle Environments
540
541
For each LE:
542
543
* Environment: environment name
544
* Content View Version: the CV and version
545
* Published At: the link the user can fetch the image from
546
547 1 Brian Bouterse
h2.  Content -> Errata
548 15 Brian Bouterse
549
List Errata
550
* Errata ID
551
* Title
552
* Type
553
* Content Host Counts
554
* Updated, e.g. 1/27/12
555
556
Filtering booleans:
557
558
* applicable: An errata applies to a host, but it is not installable because all packages are not available via repos in its content views + lifecycle environment. This is computed against 'Library', which is the entire set of Errata in the system.
559
* installable: An errata applies to a host and all packages are available via its content views + lifecycle environments. This is filtered by the repos that the host is actually subscribed to.
560
561
Host filtering behavior:
562
* errata are only shown if they are applicable/installable to a host that I have access to via the Katello permission system
563
564
Filter attributes:
565
* bug
566
* cve
567
* id
568
* issued
569
* package
570
* package name
571
* reboot_suggested
572
* severity
573
* type, e.g. enhancement, security
574
* title
575
*updated
576
577
578
h3. Content -> Errata -> {errata_name}
579
580
h5. Details
581
582
Shows details of the erratum
583
584
585
h5. Content Hosts
586
587
List
588
589
* name
590
* OS
591
* environment
592
* content view
593
594
h5. Repositories
595
596
List
597
598
* name
599
* Product
600
* Last Sync
601
602
603
h3. Content -> Errata -> Select Content Hosts
604
605
Will publish an incremental update if the necessary packages are not. e.g. 2.1 built from the repoversion backing version 2
606
Also an option to trigger an update on the host after the publish is complete.
607
608
    In Pulp 2 terms: copy repo, copy errata with recursive=true
609
610
Shows the errata and packages that were installed.
611
612 1 Brian Bouterse
613
614
h2.  Content -> Files
615
616 16 Brian Bouterse
Filter by:
617 1 Brian Bouterse
618 16 Brian Bouterse
* checksum
619
* name
620
* repository
621 1 Brian Bouterse
622
623 16 Brian Bouterse
h3. Content -> Files -> {file_name}
624 1 Brian Bouterse
625 16 Brian Bouterse
h5. Details
626 1 Brian Bouterse
627 16 Brian Bouterse
Displays
628 1 Brian Bouterse
629 16 Brian Bouterse
* path
630
* checksum
631 1 Brian Bouterse
632
633 16 Brian Bouterse
h5. Library Repositories
634
635
List:
636
637
* name
638
* product
639
* last Sync, e.g. N/A, Not Synced, Success about 23 hours ago
640
641
Search: uses the same search syntax that all repository lists use
642
643
644
h5. Content Views
645
646
List:
647
648
* name
649
* environment
650
* version
651
652
Search: uses the same syntax as the content view version search i.e 
653
654
* content_view_id
655
* repositroy
656
* version
657
658
659
h2. Content -> OSTree Branches
660
661
List:
662
663
* name
664
* version
665
666
Search:
667
668
* commit
669
* created
670
* name
671
* repository
672
* uuid
673
* version
674
675
676
h3. Content -> OSTree Branches -> {branch_name}
677
678
h5. Details
679
680
List:
681
* version
682
* commit
683
* date
684
685
686
h5. Repositories
687
688
List:
689
690
* Name
691
* Product
692
* Last Sync
693
694
Search: uses the same search syntax that all repository lists use
695
696
697
h2. Content -> Packages
698
699
Filtering booleans:
700
701
* applicable: A package applies to a host, but it is not upgradable because the newer version is not available via repos in its content views + lifecycle environment. This is computed against 'Library', which is the entire set of Packages in the system.  A user only sees packages that are applicable to hosts the user has permission to read.
702
* upgradable: An older version of this package is installed on this host and a newer package is available via that host's content views + lifecycle environments. This is filtered by the repos that the host is actually subscribed to. A user only sees packages that are upgradable to hosts the user has permission to read.
703
704
List:
705
706
* rpm
707
* summary
708
* content host counts, i.e. X applicable, Y upgradable
709
710
Filter:
711
712
* arch
713
* checksum
714
* epoch
715
* filename
716
* name
717
* release
718
* sourcerpm
719
* version
720
721
722
h3. Content -> Packages -> {package_name}
723
724
h5. Details
725
726
Shows:
727
728
* installed on, i.e. host installed count
729
* Applicable to, i.e. host count
730
* Upgradeable for, i.e. host count
731
other package attributes
732
733
h5. Files
734
735
Lists the actual files
736
737
738
h5. Dependencies
739
740
List Requires packages
741
List Provides packages
742
743
h5. Repositories
744
745
List:
746
747
* name
748
* product
749
* last sync
750
751
Search: uses the same search syntax that all repository lists use
752
753
754
h2. Content -> Puppet Modules
755
756
List:
757
    
758
* author
759
* name
760
* summary
761
* version
762
763
Filter:
764
765
* author
766
* name
767
* summary
768
* version
769
770
771
h3. Content -> Puppet Modules -> {module_name}
772
773
h5. Details
774
775
Display:
776
777
* author
778
* version
779
* source
780
* Project Page, e.g. link
781
* license
782
* description
783
* summary
784
785
786
h5. Library Repositories
787
788
List:
789
790
* name
791
* product
792
* last Sync, e.g. N/A, Not Synced, Success about 23 hours ago
793
794
Search: uses the same search syntax that all repository lists use
795
796
797
h5. Content Views
798
799
List:
800
801
* name
802
* environment
803
* version
804
805
Search: uses the same syntax as the content view version search i.e 
806
807
* content_view_id
808
* repository
809
* version
810
811
812 1 Brian Bouterse
h1.  Hosts -> Content Hosts
813 16 Brian Bouterse
814
Applicability Info:
815
816
* counts of security errata
817
* counts of bugfix errata
818
* counts of enhancement errata
819
* count of package updates
820
821
Search by:
822
823
*applicable errata
824
* applicable rpms
825
* errata_status (up to date (green), non-security updates available (yellow), security updates available (red))
826
* installable errata
827
* installed package
828
* installed package name
829
* upgradeable rpms
830
831
832
h3. Hosts -> Content Hosts -> {host_name}
833
834
Displays Installable Errata data: counts for security, bugfix, enhancement
835 1 Brian Bouterse
836
837
h1.  Non UI things
838
839 17 Justin Sherrill
* Upgrades
840
* Repository, errata, and rpm Apis
841 1 Brian Bouterse
* the API endpoint that clients upload their enabled repos
842
* the API endpoint that clients upload their package profiles
843
* the API endpoint that clients register
844
* the API endpoint that clients unregister
845
* speed throttling and other global settings?
846 2 Justin Sherrill
* Errata mailer
847
* smart proxy page/details
848 1 Brian Bouterse
849
850
851
h1.  Terminology
852
853
Candlepin Manifest - Defines Products, Subscriptions, and a Content Sets
854
Product - A collection of repositories. A repository can only belong to one product
855
Repository Set - Has a name, Label, and URL of the form:  /content/rhel/server/7/$RELVER/$BASEARCH/os/