InterServerSync » History » Revision 32
Revision 31 (Chris Duryee, 11/12/2015 08:32 PM) → Revision 32/34 (Thomas McKay, 11/13/2015 08:12 AM)
h1. InterServerSync h3. What is ISS? Inter-Server Sync is a feature designed to help users in two scenarios: * users who have a connected Foreman/Katello instance and disconnected Foreman/Katello, who want to propogate data from the connected side of their network to the disconnected side. * users who have a "main" Foreman/Katello and want to propogate some data (but not all data) to other instances. One example is for users who have "blessed" content views that are validated by the IT department, and want to propogate those down elsewhere. More info on the original ISS is available at https://fedorahosted.org/spacewalk/wiki/InterSpacewalkServerSync. h3. Goals For Katello 2.5 we want to be able to export a group of repos in a given CV or Environment in tgz, chunked iso, or as a set of directories, and import said export to another Katello/Foreman instance. Later iterations will allow for additional types of data besides yum repos or disk usage enhancements. h4. Katello 2.5 This goal replaces the katello-disconnected script, and solves the same problem that Spacewalk solves with Inter-Spacewalk Sync in a minimal fashion. Users will be able to export yum repos associated with an environment or CV to disk and import them on another server. We will optionally recreate any custom products during the import, but RH products will not be re-created since they must be created via manifest. We will support date-based incremental exports as well, via a ISO6601 "since" field on the export call. h3. Katello 2.5 design and stories Development steps # Step 1 - export > # As a user, I would like an API to export specific repositories. > > * "Feature #12446":http://projects.theforeman.org/issues/12446 : Add ability to export yum repositories (both custom and RH) to disk > > * Specify lifecycle environment and content view. > # As a user, I would like an API to export specific products. > # As an admin, I would like a role to limit which users may export. > > * Based on product read, repository read, export true? > # "Feature #12446" As an admin, I would like a setting to specify where exported files are stored on the server. > > * How are the files organized (per org, per user, per cv, per env)? > > * How are files cleaned up since they take significant space? > # As a user, I want to choose export format of iso, tgz, or dir # Step 2 - import > # "Feature #12459": http://projects.theforeman.org/issues/12459 : As a user, I want to temporarily set where to sync a repository from. > # As a user, I want to permanently set where to sync a repository from. > # As a user, I want to temporarily/permanently set sync location for a product. > # As a katello, I want to prevent import of Red Hat products into custom products. > > * If the Red Hat product already exists, it may sync from the location. A custom product may not sync Red Hat content. # Step 3 - repository enable > # As a user, I want to change the CDN to point to an export location. > # As a katello, I want the repository choices shown on the Red Hat Repositories page to be limited to what is in the export. > > * Only the Red Hat repos that were exported should be displayed as available for enable. > > * If a product was enabled previously, it should be shown as already enabled. h4. Katello 2.5 hammer design this section is getting reworked:) <pre> # export a repository to disk on the Katello server (need to do this for each repo but can be a repo in a CV) hammer repository export --id <id> # export products to CSV hammer csv products --csv-export --csv-file products.csv # replace URL with on-disk location for place to sync from on destination katello sed -i 's#https://repos.fedorapeople.org/#file:///mnt/imports/#g' /tmp/foo.csv # import steps # ensure export is available, then run hammer csv products --csv-import --csv-file products.csv # kick off syncs for imported products hammer repository synchronize --id <id> </pre> h4. Katello 2.6+ These features require additional thought, or possibly upstream RFEs for Pulp. * Push repo contents from one Katello/Foreman to another (if X can initiate connections to Y but not vice versa). * Allow exporting to local disk where hammer is being run * Allow exports/imports to occur without writing entire export to disk * Support for additional types of data (docker, ostree, puppet) * Support for additional types of metadata (users, etc?)