h1. Debian Packaging

{{toc}}

h2. Artifacts

The following artifacts are generated from the RPM build process and form part of any release:

* foreman, including database and compute resource subpackages

* foreman-installer

* foreman-proxy

* dependencies

h2. Tooling

h3. Jenkins

Jenkins is used to automate the package builds, using the jobs:

* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman_individual-DEB

* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman-proxy_individual-DEB

* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman_matrix

* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman-proxy_matrix

The matrix jobs only presently build the nightly packages, thus RC/Stable releases will be queued manually using the single jobs. The single jobs require you to sepcify:

* The target release (nightly/rc/stable)

* The target OS (wheezy/squeeze/precise)

* The target arch (32/64bit)

h3. Pbuilder

"Pbuilder":http://www.netfort.gr.jp/~dancer/software/pbuilder-doc/pbuilder-doc.html is the system used internally by jenkins to build the packages for each of the architectures. It should just work, but if there are problems, you can look at the pbuilder module in foreman-infra for more detail.

h2. Project sources

h3. foreman

Foreman(-proxy) is built from the packaging constructs in the foreman-rpms repo:

* https://github.com/theforeman/foreman-rpms/tree/deb/stable (or rc/nightly)

This repo is pulled in by Jenkins, thefore build preparation consists of appropriately updating this repo before queueing the Jenkins jobs.

h4. Initial tests

The nightly repo tracks the develop branch. At the point when a release candidate is forked, nightly will most likely be at the same commit, and will already be built (as it is triggered by a commit to develop). Thus, if the nightly packages work, one can conclude the packaging constructs in deb/nightly are sane.

h4. Release candidate

The following steps are used to step up a release candidate build from nightly

* Merge deb/nightly into deb/rc

** This _may_ require a force overwrite, but hopefully the merge will be clean as the only commits direct to the rc repo should be the <package>/build.sh updates

** If a force overwrite is used, revert the changes to the <package>/build.sh files as they should only contain the SHAsum lines for the package

* Update the SHAsums in <package>/build.sh to point to correct SHAsum for the release

* Add any new dependencies from nightly to rc (these will have been built during the previous release cycle for nightly, so at release it's just a matter of importing the debs)

** Log in to server2

** su -u freight -

** Use freight-add / freight cache to add new deps to rc repo, e.g:

*** freight add staged/apt/wheezy/nightly/*sinatra* apt/wheezy/rc                    

*** freight cache -v

* Update the changelog with a block announcing the release candidate, e.g:

<pre>

foreman (1.2.0+debian0.1) squeeze; urgency=low

  * Release 1.2.0 RC1

 -- Greg Sutcliffe <greg.sutcliffe@gmail.com>  Mon, 20 May 2013 16:31:00 +0100

</pre>

** Note the Debian changelog is _extremely_ sensitive to format and syntax. If in doubt, do this on a debian box and use the "dch -i" tool to add the entry

** Use <release-in-three-parts>+debian0.<rc-version> as the release to ensure the first stable release can use debian1

* PR / Commit the git changes to foreman-rpms:deb/rc

* Queue the Jenkins jobs!

* Test the resulting packages

h4. Stable release

The stable release is almost identical to the rc release, with good reason ;)

* Merge deb/rc into deb/stable

** This is far more likely to be clean, very little should be merged here without haven't already been through rc

** If required, update the SHAsums in <package>/build.sh again

* Add any new dependencies from rc to stable

** Same process on server2

* Update the changelog with a block announcing the release, e.g:

<pre>

foreman (1.2.0+debian1) squeeze; urgency=low

(as above for the rest)

</pre>

* PR / Commit the git changes to foreman-rpms:deb/stable

* Queue the Jenkins jobs!

* Test the resulting packages

Assuming a successful release, the changelog should be merged back to deb/nightly so that history is preserved during the next release cycle

h3. foreman-proxy

The exact same process above is used to release the proxy.

h3. foreman-installer

The installer is built using "fpm":https://github.com/jordansissel/fpm which provides a lightweight build mechanism that also supports Debian packages.  A small script wraps this:

* https://github.com/theforeman/foreman-rpms/blob/master/fpm/build_installer_pkgs.sh

The script currently assumes you are able to write to @/usr/share/foreman-installer@ rather than building in a temporary directory.  Patches gladly accepted.

A job on Jenkins can build this by checking out foreman-rpms and running the script:

* http://ci.theforeman.org/job/packaging_build_foreman-installer/

This can be triggered manually, where it takes the version number and the build type (rpm/deb) to generate the build artifact. Once built, test the deb on at least one Debian OS to ensure it works (it's a very simple package, so it will install on all of them if it installs on one), then upload it to freight:

** Log in to server2

** su -u freight -

** curl http://ci.theforeman.org/path/to/built/deb

** freight add ./installer.deb apt/wheezy/rc #(and the other repos)

** freight cache -v

(TODO: automate this properly along with the other packages)

h3. Dependencies

Foreman's Debian dependencies should all be present in the nightly repo in order to make the nightly builds work. Work is underway to store the appropriate packaging constructs in the foreman-rpms repo, such that they can be rebuilt if needed. The packages are built using pbuilder manually. Rough notes on performing this build are:

* Log in to server9

* Clone foreman-rpms:deb/nightly

* cd to the appropriate package

* run pdebuild-<os_arch> (eg. pdebuild-squeeze32) to generate a package for that OS/Arch

* Upload to serevr2 and use freight to add to the repo

TODO: Document this properly.

h2. Nightly Builds

Nightly builds are fully automated. Next time they break I'll document what I did to fix them. Full automation is possible because of the fact that the Debian packages use gem vendoring to ship a cache of prebuilt gems to the user; thus, they are far less susceptible to changes in the Gemfile (downside: they'll never be accepted upstream)