Plugins » Katello

h1. InterServerSync

h3. What is ISS?

Inter-Server Sync is a feature designed to help users in two scenarios:

 * users who have a connected Foreman/Katello instance and disconnected Foreman/Katello, who want to propogate data from the connected side of their network to the disconnected side.

 * users who have a "main" Foreman/Katello and want to propogate some data (but not all data) to other instances. One example is for users who have "blessed" content views that are validated by the IT department, and want to propogate those down elsewhere.

More info on the original ISS is available at https://fedorahosted.org/spacewalk/wiki/InterSpacewalkServerSync.

h3. Goals

For Katello 2.5 we want to be able to export a group of repos in a given CV or Environment in tgz, chunked iso, or as a set of directories, and import said export to another Katello/Foreman instance.

Later iterations will allow for additional types of data besides yum repos or disk usage enhancements.

h4. Katello 2.5

This goal replaces the katello-disconnected script, and solves the same problem that Spacewalk solves with Inter-Spacewalk Sync in a minimal fashion. Users will be able to export yum repos associated with an environment or CV to disk and import them on another server. We will optionally recreate any custom products during the import, but RH products will not be re-created since they must be created via manifest.

We will support date-based incremental exports as well, via a ISO6601 "since" field on the export call.

h3. Katello 2.5 design and stories

Development steps

# Step 1 - export

> # As a user, I would like an API to export specific repositories.

> > * "Feature #12446":http://projects.theforeman.org/issues/12446 : Add ability to export yum repositories (both custom and RH) to disk

> > * Specify lifecycle environment and content view.

> # As a user, I would like an API to export specific products.

> # As an admin, I would like a role to limit which users may export.

> > * Based on product read, repository read, export true?

> # "Feature #12446" As an admin, I would like a setting to specify where exported files are stored on the server.

> > * How are the files organized (per org, per user, per cv, per env)?

> > * How are files cleaned up since they take significant space?

> # As a user, I want to choose export format of iso, tgz, or dir

# Step 2 - import

> # "Feature #12459": http://projects.theforeman.org/issues/12459 : As a user, I want to temporarily set where to sync a repository from.

> # As a user, I want to permanently set where to sync a repository from.

> # As a user, I want to temporarily/permanently set sync location for a product.

> # As a katello, I want to prevent import of Red Hat products into custom products.

> > * If the Red Hat product already exists, it may sync from the location. A custom product may not sync Red Hat content.

# Step 3 - repository enable

> # As a user, I want to change the CDN to point to an export location.

> # As a katello, I want the repository choices shown on the Red Hat Repositories page to be limited to what is in the export.

> > * Only the Red Hat repos that were exported should be displayed as available for enable.

> > * If a product was enabled previously, it should be shown as already enabled.

h4. Katello 2.5 hammer design

this section is getting reworked:)

<pre>

# export a repository to disk on the Katello server (need to do this for each repo but can be a repo in a CV)

hammer repository export --id <id>

# export products to CSV

hammer csv products --csv-export --csv-file products.csv 

# replace URL with on-disk location for place to sync from on destination katello

sed -i 's#https://repos.fedorapeople.org/#file:///mnt/imports/#g' /tmp/foo.csv

# import steps

# ensure export is available, then run

hammer csv products --csv-import --csv-file products.csv 

# kick off syncs for imported products

hammer repository synchronize --id <id>

</pre>

h4. Katello to Katello syncing

This is useful for when you have a Katello instance that cannot connect to the Internet, but it can connect to another Katello with Internet access.

For this scenario, the players will be Katello #1 which has Internet access, and Katello #2 which only has access to #1. We will want Katello #2 to honor Katello #1's SSL server certificate, and we want Katello #1 to honor Katello #2's entitlement client certificate, even though Katello #2 got it from a manifest from access.redhat.com.

At a high level, we need to make the following changes:

 * Katello #1 needs to include the Red Hat Entitlement CA when determining if a client cert is valid

 * Katello #2 needs to include Katello #1's server CA when determining if the server is legitimate

 * Users will need to update the CDN location on Katello #2 to be Katello #1 instead of https://cdn.redhat.com.

The first change requires a few user stories:

 * as a Katello packager, I would like a copy of the Red Hat Entitlement CA installed.

 * as a Katello admin, I would like to optionally enable the Red Hat Entitlement CA as a valid CA when performing client certificate validation. This should not be enabled by default but should be set/updated via katello-installer. The easiest way to do this is to append both redhat-uep.pem and the RH Entitlement CA into "/etc/pki/katello/certs/katello-default-ca.crt". There may be a more elegant option though. Note that this is done via httpd settings and cannot be enabled on a per-org basis.

 * as a Katello administrator, I would like all instances of "SSLVerifyDepth" in my httpd configs to be at least 3.

 * As an org admin, I would like to allow an environment to be used as an ersatz CDN. In Pulp 2.7, this can be done by adding the org/env portion of the path on Katello #2's CDN url, and then adding this same string into repo_auth.conf on Katello #1. This lets Katello #1 "chop off" the org/env portion of the path before checking the entitlement certificate. Note that Pulp 2.6 does not have this functionality. There is a way around it but it's per-katello and not per-org.

The second change requires one additional story:

 * as a user, I would like to select a CA certficate to use when validating the upstream content source. Right now this is hardcoded to redhat-uep.pem. This change may need to be made in the Katello code that reads listing files as well as the Pulp repo setup code.

Finally, https://bugzilla.redhat.com/show_bug.cgi?id=1284683 will need to be addressed before this will work (aside from simple curl tests).

h4. Katello 2.6+

These features require additional thought, or possibly upstream RFEs for Pulp.

 * Push repo contents from one Katello/Foreman to another (if X can initiate connections to Y but not vice versa).

 * Allow exporting to local disk where hammer is being run

 * Allow exports/imports to occur without writing entire export to disk

 * Support for additional types of data (docker, ostree, puppet)

 * Support for additional types of metadata (users, etc?)