Project

General

Profile

Debian Packaging » History » Version 5

Greg Sutcliffe, 05/24/2013 07:25 AM

1 1 Greg Sutcliffe
h1. Debian Packaging
2
3
{{toc}}
4
5
h2. Artifacts
6
7
The following artifacts are generated from the RPM build process and form part of any release:
8
9
* foreman, including database and compute resource subpackages
10
* foreman-installer
11
* foreman-proxy
12 2 Greg Sutcliffe
* dependencies
13 1 Greg Sutcliffe
14
h2. Tooling
15
16
h3. Jenkins
17
18
Jenkins is used to automate the package builds, using the jobs:
19
20
* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman_individual-DEB
21
* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman-proxy_individual-DEB
22
* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman_matrix
23
* http://ci.theforeman.org/view/Packaging/job/packaging_build_foreman-proxy_matrix
24
25 3 Greg Sutcliffe
The matrix jobs only presently build the nightly packages, thus RC/Stable releases will be queued manually using the single jobs. The single jobs require you to sepcify:
26 1 Greg Sutcliffe
27
* The target release (nightly/rc/stable)
28
* The target OS (wheezy/squeeze/precise)
29
* The target arch (32/64bit)
30
31
h3. Pbuilder
32
33
"Pbuilder":http://www.netfort.gr.jp/~dancer/software/pbuilder-doc/pbuilder-doc.html is the system used internally by jenkins to build the packages for each of the architectures. It should just work, but if there are problems, you can look at the pbuilder module in foreman-infra for more detail.
34
35
h2. Project sources
36
37
h3. foreman
38
39
Foreman(-proxy) is built from the packaging constructs in the foreman-rpms repo:
40
41
* https://github.com/theforeman/foreman-rpms/tree/deb/stable (or rc/nightly)
42
43
This repo is pulled in by Jenkins, thefore build preparation consists of appropriately updating this repo before queueing the Jenkins jobs.
44
45
h4. Initial tests
46
47
The nightly repo tracks the develop branch. At the point when a release candidate is forked, nightly will most likely be at the same commit, and will already be built (as it is triggered by a commit to develop). Thus, if the nightly packages work, one can conclude the packaging constructs in deb/nightly are sane.
48
49
h4. Release candidate
50
51
The following steps are used to step up a release candidate build from nightly
52
53
* Merge deb/nightly into deb/rc
54 4 Greg Sutcliffe
** This _may_ require a force overwrite, but hopefully the merge will be clean as the only commits direct to the rc repo should be the <package>/build.sh updates
55
** If a force overwrite is used, revert the changes to the <package>/build.sh files as they should only contain the SHAsum lines for the package
56 1 Greg Sutcliffe
* Update the SHAsums in <package>/build.sh to point to correct SHAsum for the release
57
* Add any new dependencies from nightly to rc (these will have been built during the previous release cycle for nightly, so at release it's just a matter of importing the debs)
58 4 Greg Sutcliffe
** Log in to server2
59
** su -u freight -
60
** Use freight-add / freight cache to add new deps to rc repo, e.g:
61
*** freight add staged/apt/wheezy/nightly/*sinatra* apt/wheezy/rc                    
62
*** freight cache -v
63 1 Greg Sutcliffe
* Update the changelog with a block announcing the release candidate, e.g:
64
<pre>
65
foreman (1.2.0+debian0.1) squeeze; urgency=low
66
67
  * Release 1.2.0 RC1
68
69
 -- Greg Sutcliffe <greg.sutcliffe@gmail.com>  Mon, 20 May 2013 16:31:00 +0100
70
71
</pre>
72 4 Greg Sutcliffe
** Note the Debian changelog is _extremely_ sensitive to format and syntax. If in doubt, do this on a debian box and use the "dch -i" tool to add the entry
73
** Use <release-in-three-parts>+debian0.<rc-version> as the release to ensure the first stable release can use debian1
74 1 Greg Sutcliffe
* PR / Commit the git changes to foreman-rpms:deb/rc
75
* Queue the Jenkins jobs!
76
* Test the resulting packages
77
78
h4. Stable release
79
80
The stable release is almost identical to the rc release, with good reason ;)
81
82
* Merge deb/rc into deb/stable
83 4 Greg Sutcliffe
** This is far more likely to be clean, very little should be merged here without haven't already been through rc
84
** If required, update the SHAsums in <package>/build.sh again
85 1 Greg Sutcliffe
* Add any new dependencies from rc to stable
86 4 Greg Sutcliffe
** Same process on server2
87 1 Greg Sutcliffe
* Update the changelog with a block announcing the release, e.g:
88
<pre>
89
foreman (1.2.0+debian1) squeeze; urgency=low
90
(as above for the rest)
91
</pre>
92
* PR / Commit the git changes to foreman-rpms:deb/stable
93
* Queue the Jenkins jobs!
94
* Test the resulting packages
95
96
Assuming a successful release, the changelog should be merged back to deb/nightly so that history is preserved during the next release cycle
97
98
h3. foreman-proxy
99
100
The exact same process above is used to release the proxy.
101
102
h3. foreman-installer
103
104
The installer is built using "fpm":https://github.com/jordansissel/fpm which provides a lightweight build mechanism that also supports Debian packages.  A small script wraps this:
105
106
* https://github.com/theforeman/foreman-rpms/blob/master/fpm/build_installer_pkgs.sh
107
108
The script currently assumes you are able to write to @/usr/share/foreman-installer@ rather than building in a temporary directory.  Patches gladly accepted.
109
110
A job on Jenkins can build this by checking out foreman-rpms and running the script:
111
112
* http://ci.theforeman.org/job/packaging_build_foreman-installer/
113
114 5 Greg Sutcliffe
This can be triggered manually, where it takes the version number and the build type (rpm/deb) to generate the build artifact. Once built, test the deb on at least one Debian OS to ensure it works (it's a very simple package, so it will install on all of them if it installs on one), then upload it to freight:
115
116
** Log in to server2
117
** su -u freight -
118
** curl http://ci.theforeman.org/path/to/built/deb
119
** freight add ./installer.deb apt/wheezy/rc #(and the other repos)
120
** freight cache -v
121
122
(TODO: automate this properly along with the other packages)
123 1 Greg Sutcliffe
124
h3. Dependencies
125
126
Foreman's Debian dependencies should all be present in the nightly repo in order to make the nightly builds work. Work is underway to store the appropriate packaging constructs in the foreman-rpms repo, such that they can be rebuilt if needed. The packages are built using pbuilder manually. Rough notes on performing this build are:
127
128
* Log in to server9
129
* Clone foreman-rpms:deb/nightly
130
* cd to the appropriate package
131
* run pdebuild-<os_arch> (eg. pdebuild-squeeze32) to generate a package for that OS/Arch
132
* Upload to serevr2 and use freight to add to the repo
133
134
TODO: Document this properly.
135
136
h2. Nightly Builds
137
138
Nightly builds are fully automated. Next time they break I'll document what I did to fix them. Full automation is possible because of the fact that the Debian packages use gem vendoring to ship a cache of prebuilt gems to the user; thus, they are far less susceptible to changes in the Gemfile (downside: they'll never be accepted upstream)