Project

General

Profile

InterServerSync » History » Version 9

Chris Duryee, 10/27/2015 12:44 PM

1 1 Chris Duryee
h1. InterServerSync
2
3 2 Chris Duryee
h3. What is ISS?
4
5
Inter-Server Sync is a feature designed to help users in two scenarios:
6
7
 * users who have a connected Foreman/Katello instance and disconnected Foreman/Katello, who want to propogate data from the connected side of their network to the disconnected side.
8
 * users who have a "main" Foreman/Katello and want to propogate some data (but not all data) to other instances. One example is for users who have "blessed" content views that are validated by the IT department, and want to propogate those down elsewhere.
9
10 5 Chris Duryee
More info is available at https://fedorahosted.org/spacewalk/wiki/InterSpacewalkServerSync.
11
12 1 Chris Duryee
h3. Goals
13
14 7 Chris Duryee
Roughly, phase 1 is the minimal "get it working" phase where we have a demoable end-to-end scenario that works via hammer. After that, we will collect feedback and move on to phase 2 which moves some of the grunt work done by hammer to the server, and adds web UI support. Phase 3 enhances the "connected" scenario, and Phase 4 will be for support of additional content types and new types of foreman/katello data.
15 1 Chris Duryee
16
h4. Phase 1 "get it working" goals
17
18
 * allow exporting and importing of products and Library content views
19
 * export/import can occur via hammer and web UI, but some filesystem-level access is needed for accessing yum repo exports (scp, http, etc)
20
 * only yum is supported
21
 * incremental dumps are supported
22
23 8 Chris Duryee
Note: phase 1 replaces katello-disconnected script, and solves the same problem that Spacewalk solves with Inter-Spacewalk Sync in a minimal fashion (notably, without the disk space usage optimization).
24 1 Chris Duryee
25 3 Chris Duryee
h4. Phase 2 "web UI and optimization" goals
26 1 Chris Duryee
27 3 Chris Duryee
 * API clean up to allow for bulk export/import with only a few server calls - this is needed for web UI support since we can't rely on hammer
28
 * support for import/export of products and library CV via web UI
29 1 Chris Duryee
 * allow export and import without filesystem access to machine (hammer or web ui can provide download of export and upload of import)
30
31 6 Chris Duryee
h4. Phase 3 "do it with less disk space" goals
32
33
 * stream export/import data from one Foreman/katello to another without needing to write a full export to disk
34 9 Chris Duryee
 * streaming must be able to be initiated from either the source or destination foreman/katello
35 6 Chris Duryee
36
h4. Phase 4+ "rinse and repeat" goals
37 1 Chris Duryee
38
 * allow exporting and importing of other data (environments, non-library content views, etc)
39
 * support for ostree, docker, puppet content
40
41
42
h3. Phase 1 Design
43
44
There are two areas that need to be updated for phase 1. We need to allow for exporting and importing of yum repo data (including incremental updates), and we need to allow for exporting and importing of product and library CV data. The former is largely provided by Pulp, and the latter is provided by hammer-cli-csv.
45
46
We will need to add the Pulp export distributor on newly created yum repos. Additionally, existing repos will need to have the export distributor added in via migration.
47
48 3 Chris Duryee
Phase 1 allows for hammer-cli-csv doing a lot of "grunt" work in loading the CSV data. This will need to be done server-side in later phases.
49
50
h4. Hammer design
51
52
(need to fill in)
53
54
h4. API modifications
55
56
(need to fill in)