Project

General

Profile

Release Process » History » Version 206

Ewoud Kohl van Wijngaarden, 07/17/2018 11:07 AM

1 1 Dominic Cleal
h1. Release Process
2
3 153 Dominic Cleal
{{toc}}
4
5 155 Dominic Cleal
For each major release (i.e. not patch releases), the project selects a release manager who's responsible for taking the develop branch through to release.  Their high level tasks and policies are described on [[Release_Management]], while this document covers specific steps (down to individual commands) that need to be run at each step of the release.
6 1 Dominic Cleal
7
Please amend these lists as you see free, and as you find what works and what doesn't work.
8
9 154 Dominic Cleal
These examples are all based around a 1.2 series release, so any reference to "1.2" should be replaced by the current release, "1.1" by your previous release, and "1.3" by the next release.
10
11 51 Lukas Zapletal
h2. Near-end of development phase
12 1 Dominic Cleal
13 200 Tomer Brisker
# Announce upcoming branching to discourse development category ("example":https://community.theforeman.org/t/1-19-branching-in-core-heads-up/10048) a month before
14 1 Dominic Cleal
# Change @$latest@ and @$next@ parameters on @web@ class to point to the new version numbers (see @foreman-infra/puppet/modules/web/manifests/init.pp@)
15 154 Dominic Cleal
# File a new release tracker on theforeman.org for "Foreman 1.2 manual" ("example":https://github.com/theforeman/theforeman.org/issues/494)
16 1 Dominic Cleal
# Refresh unattended templates from the community-templates to Foreman core, by running @script/sync_templates.sh@
17
# Create a new branch in community-templates for upcoming Foreman release (e.g. 1.2-stable)
18 154 Dominic Cleal
# Add new languages that are at a "reasonable completion on Transifex":https://www.transifex.com/foreman/foreman/foreman/ to *develop*
19 1 Dominic Cleal
# Generate a release GPG key using [[GPG_Keys]]
20
#* publish to keyserver
21 186 Ondřej Pražák
#* security.md on the website, add public key fingerprint to static/keys
22 112 Dominic Cleal
#* create @releases/1.2/RPM-GPG-KEY-foreman@ on yum.theforeman.org
23 196 Lukas Zapletal
# Ask plugin authors to start extracting i18n strings and pushing the changes into develop/master git branches so Transiflex can pick it up
24 153 Dominic Cleal
25
h2. Branching for series
26
27 1 Dominic Cleal
When ready to branch for release candidates.
28 153 Dominic Cleal
29
h3. Package build systems
30 55 Lukas Zapletal
31 194 Ondřej Pražák
# Clone tags and create build targets in "Koji":http://koji.katello.org/koji using koji/copy-tags-commands.sh in foreman-packaging/rpm/develop or use the tool_belt
32 201 Ewoud Kohl van Wijngaarden
#* Make sure you have a kkoji section in ~/.koji/config , you can find instructions in [[Koji]]
33 167 Daniel Lobato Garcia
#* This generates commands that create foreman-1.2-rhel7 etc. as clones of nightly tags and foreman-plugins-1.2-rhel7 etc. with inheritance from the new Foreman tags
34 165 Dominic Cleal
#* Untag all git/nightly builds: @for t in foreman-1.2-rhel7 foreman-1.2-nonscl-rhel7 foreman-1.2-fedora24; do kkoji list-tagged $t  | grep git | awk '{print $1}' | xargs kkoji untag-build  $t; done@
35 1 Dominic Cleal
# Create mash scripts and configuration on Koji
36 110 Dominic Cleal
#* copy /usr/local/bin/foreman-mash-split-1.*.py to foreman-mash-split-1.2.py, update the OSes and version numbers at the bottom
37 64 Lukas Zapletal
#* copy /etc/mash/foreman-1.*-*.mash and foreman-plugins-1.*-*.mash, update the version numbers, add/remove files for OS changes, set GPG key ID and ensure strict_keys=True
38
# Add new plugin tags (i.e. 1.2) to Koji plugins mash script (foreman-mash-split-plugins.py), remove old ones (keep three)
39 163 Dominic Cleal
# Add version (1.2) to these jobs in axes and/or combination filters, remove old ones (keep three)
40 174 Ondřej Pražák
## "foremanPluginsRelease":https://github.com/theforeman/foreman-infra/blob/master/puppet/modules/jenkins_job_builder/files/theforeman.org/pipelines/release/foremanPluginsRelease.groovy
41 188 Ondřej Pražák
## "packaging_build_deb_coreproject":https://github.com/theforeman/foreman-infra/blob/master/puppet/modules/jenkins_job_builder/files/theforeman.org/yaml/jobs/packaging_build_deb_coreproject.yaml
42
## "packaging_build_deb_dependency":https://github.com/theforeman/foreman-infra/blob/master/puppet/modules/jenkins_job_builder/files/theforeman.org/yaml/jobs/packaging_build_deb_dependency.yaml
43 163 Dominic Cleal
## "release_test":https://github.com/theforeman/foreman-infra/blob/master/puppet/modules/jenkins_job_builder/files/theforeman.org/yaml/jobs/release_test.yaml
44
## "release_push_deb":http://ci.theforeman.org/job/release_push_deb/
45 1 Dominic Cleal
## "release_push_rpm":http://ci.theforeman.org/job/release_push_rpm/
46 201 Ewoud Kohl van Wijngaarden
# Clone Debian nightly repos to 1.2 using [[Debian_Packaging#Project-sources|copy/freight instructions]]
47 164 Dominic Cleal
# Edit "foreman-packaging's PR template":https://github.com/theforeman/foreman-packaging/blob/master/.github/PULL_REQUEST_TEMPLATE.md to add 1.2 and remove the old release
48 8 Dominic Cleal
49 153 Dominic Cleal
h3. Main code repos
50 1 Dominic Cleal
51 40 Dominic Cleal
# Make releases of installer modules, usually new minor or major versions
52 169 Daniel Lobato Garcia
## Currently puppet-puppet, puppet-foreman, puppet-dhcp, puppet-dns, puppet-foreman_proxy, puppet-tftp, puppet-git
53 61 Lukas Zapletal
# In foreman *develop* run @make -C locale tx-update@
54 1 Dominic Cleal
# In foreman, smart-proxy, foreman-installer and foreman-selinux branch *develop* to *1.2-stable* (*from this point* cherry-picks into this branch only from this point using: @git cherry-pick -x SHA@)
55
# In foreman-installer, branch *1.2-stable*,
56 197 Ewoud Kohl van Wijngaarden
## run @bundle exec rake pin_modules@ to change Puppetfile from git references to specific version ranges (e.g. @>= 1.3.1 < 1.4.0@)
57
## run @sed -i '/Puppetfile.lock/d' .gitignore@ to remove Puppetfile.lock from .gitignore
58
## run @librarian-puppet install@
59
## commit Puppetfile, Puppetfile.lock and .gitignore changes
60 70 Dominic Cleal
# In foreman-packaging for RPMs:
61 132 Dominic Cleal
## branch *rpm/develop* to *rpm/1.2*
62
## on *rpm/1.2*
63
### in rel-eng/releasers.conf and rel-eng/tito.props, replace "nightly" with "1.2"
64
### change @foreman/foreman.repo@ URLs from @/nightly@ to @/releases/1.2@, change "nightly" in name to "1.2" and enable gpg checking
65
### change @foreman/foreman-plugins.repo@ URLs from @/plugins/nightly@ to @/plugins/1.2@, change "nightly" in name to "1.2" and DO NOT enable GPG checking
66 127 Dominic Cleal
### update @foreman/foreman.gpg@ with the release public key
67
## on *rpm/develop*
68 142 Dominic Cleal
### change foreman/foreman-proxy/foreman-installer/foreman-selinux version to 1.3.0, add %changelog entry
69 122 Dominic Cleal
### change tag_suffix to @.fm1_3@ in rel-eng/tito.props, and rel-eng/releasers.conf
70
# In foreman-packaging for debs:
71
## on *deb/develop*, update debian/*/*/changelog with entries from 1.1, commit with message "Sync 1.1.x releases into changelogs"
72 144 Dominic Cleal
## branch *deb/develop* to *deb/1.2*
73 100 Dominic Cleal
## on *deb/develop*, run @scripts/changelog.rb -v 1.3.0-1 -m "Bump changelog to 1.3.0 to match VERSION" debian/*/*/changelog@ 
74 106 Dominic Cleal
# In foreman *develop* commit with message "Bump version to 1.3-develop":
75 161 Dominic Cleal
## change VERSION to 1.3.0-develop
76
## change package.json version field to 1.3.0
77 1 Dominic Cleal
## run @extras/changelog@
78
# In smart-proxy *develop* commit with message "Bump version to 1.3-develop":
79
## change VERSION to 1.3.0-develop
80
## run @extra/changelog@
81
# In foreman-selinux *develop* commit with message "Bump version to 1.3-develop":
82
## change VERSION to 1.3.0-develop
83
## run @extras/changelog@
84
# In foreman-installer *develop* commit with message "Bump version to 1.3-develop":
85 161 Dominic Cleal
## change VERSION to 1.3.0-develop
86 162 Dominic Cleal
87
h3. Other systems
88
89
# Create release schedule page for next version (1.3) linked from [[Development_Resources]]
90 202 Tomer Brisker
# Add next version number (1.3.0) to Redmine under "Versions":https://projects.theforeman.org/projects/foreman/settings/versions and all relevant subprojects (installer, proxy, and selinux)
91 162 Dominic Cleal
# Add first patch release (1.2.1) to Redmine in the same way
92 195 Lukas Zapletal
# Update foreman-dev with "translations status":https://www.transifex.com/projects/p/foreman/ to encourage 100% translations before release
93
# Announce string freeze date on discourse and send announcement via https://www.transifex.com/foreman/foreman/announcements/
94 191 Ondřej Pražák
# Create test_1_2_stable.yaml and test_proxy_1_2_stable.yaml for JJB in foreman-infra, remove the oldest version to keep last 3
95 154 Dominic Cleal
# Ensure current Foreman deprecations for the next release are removed in *develop*
96 1 Dominic Cleal
97 187 Eric Helms
h3. Rails for RPMs
98
99
The Rails stack is built as a separate SCL in Copr but then mirrored for a given Foreman version on yum.theforeman.org. When branching, a version should be created for the Foreman version being released:
100
101 192 Eric Helms
<pre>
102 187 Eric Helms
# ssh yum.theforeman.org
103
# cd /var/www/vhosts/yum/htdocs/rails/
104
# cp -rf foreman-nightly foreman-<version>
105 192 Eric Helms
</pre>
106 187 Eric Helms
107
The foreman-release-scl RPM then needs updating to point at this new repository in SCL repo file.
108 192 Eric Helms
109
Koji needs to be updated to include a new external repository pointing to this new rails repository for builds.
110
111
112
  * ssh root@koji.katello.org
113
  * edit /etc/mrepo.conf and add foreman-<version> entry for new version
114 199 Tomer Brisker
  * Now run: <pre>mrepo -ug -f -v foreman-rails-<version></pre>
115 192 Eric Helms
  * Go back to developer machine
116
  * Add the new external repo: `koji add-external-repo foreman-rails-<version>-rhel-7 file:///mnt/koji/external-repos/www/foreman-rails-<version>-$arch/RPMS.ror51/`
117 187 Eric Helms
118 1 Dominic Cleal
h2. Preparing a release
119
120 100 Dominic Cleal
Version numbers for release candidates start with "1.2.0-RC1" (always hyphenated).  The first release after RC would be "1.2.0".  Change examples below as appropriate.
121
122 1 Dominic Cleal
h3. Writing release notes
123
124 179 Ondřej Pražák
# Copy website manual content ("theforeman.org repo":http://github.com/theforeman/theforeman.org) from previous version to this version ("example":https://github.com/theforeman/theforeman.org/pull/43)
125
#* add version number to documentation.md dropdown menu
126 153 Dominic Cleal
# Update manual if applicable for any additional installation steps
127
# Draft release notes in markdown ("example":https://gist.github.com/domcleal/5567450), with these sections (and do not use personal pronouns):
128
## Headline features: half a dozen important features with a few sentences description each
129
## Upgrade notes: all important notices that users must be aware of before upgrading
130 176 Ondřej Pražák
## Release notes: bullet point list by category of most changes, excluding bug fixes for issues introduced during the release cycle, include link to bug numbers. You can auto-generate changes using a script from "theforeman.org":https://github.com/theforeman/theforeman.org/blob/gh-pages/scripts/release_notes.rb or use the 'changelog' command in "tool_belt":https://github.com/theforeman/tool_belt
131 1 Dominic Cleal
## CLI release notes are taken from the hammer-cli and hammer-cli-foreman changelogs
132
## Link to installer changelogs and note versions being used
133 179 Ondřej Pražák
# Get the apipie doc and place it in api/$release_num
134
# Change links in _includes/version.html for 'latest' and 'latest stable'
135 153 Dominic Cleal
136
h3. Preparing code
137
138
# Request Hammer CLI releases from maintainers if desired
139 132 Dominic Cleal
# Make patch releases of installer modules that have important changes
140 1 Dominic Cleal
#* Branch to MAJ.MIN-stable if recent changes to the module aren't suitable for patch (x.y.z) release
141
# Compare tagged packages in nightly vs. release koji tag and re-tag any updated dependencies that are required
142 205 Tomer Brisker
# Add a new "Redmine version":https://projects.theforeman.org/projects/foreman/settings/versions for the next minor, unless the series is EOL
143 152 Dominic Cleal
# Remove/change release field for any open Redmine tickets assigned to the release still (next minor, unset it or reject)
144 205 Tomer Brisker
# Change Redmine version state to _Closed_
145 139 Dominic Cleal
# List all issues targeted at the release, order by Closed date ascending and use @git cherry-pick -x@ to cherry pick from develop to *1.2-stable* branch
146 1 Dominic Cleal
# Clone "tool_belt":https://github.com/theforeman/tool_belt (foreman branch) and run:
147
## @./tools.rb setup-environment configs/foreman_12.yaml@
148
## @./tools.rb cherry-picks --version 1.2.0 configs/foreman_12.yaml@
149 140 Dominic Cleal
## Verify tickets in the @cherry_picks_1.2.0@ file are accounted for or additional cherry pick them
150
151
h2. Tagging a release
152 1 Dominic Cleal
153 180 Ondřej Pražák
# In foreman *1.2-stable*:
154
## run @make -C locale tx-update@ (if Transifex has not switched to the next major release yet, usually after .2)
155
## run @script/sync_templates.sh@
156 1 Dominic Cleal
## change VERSION to 1.2.0
157
## change package.json version field to 1.2.0
158 161 Dominic Cleal
## run @extras/changelog@
159 180 Ondřej Pražák
## commit with message "Release 1.2.0"
160 104 Dominic Cleal
# In smart-proxy *1.2-stable* commit with message "Release 1.2.0":
161 1 Dominic Cleal
## change VERSION to 1.2.0
162 115 Dominic Cleal
## run @extra/changelog@
163 109 Dominic Cleal
# In foreman-selinux *1.2-stable* commit with message "Release 1.2.0":
164 1 Dominic Cleal
## change VERSION to 1.2.0
165
## run extras/changelog
166 115 Dominic Cleal
# In foreman-installer *1.2-stable* commit with message "Release 1.2.0":
167
## change VERSION to 1.2.0
168 168 Daniel Lobato Garcia
# Before tagging, make sure the test_xxxx_stable versions of each of the projects are green in http://ci.theforeman.org/
169 52 Dominic Cleal
# Tag commits in foreman, smart-proxy, foreman-installer and foreman-selinux: @git tag -m "Release 1.2.0" 1.2.0@
170 1 Dominic Cleal
# Push to the project repos: @git push project && git push project 1.2.0@
171
# Run the Jenkins "Release Pipeline":http://ci.theforeman.org/view/Release%20pipeline/ to create tarballs
172 115 Dominic Cleal
# Verify tarballs are present on downloads.theforeman.org, download, sign and upload detached signatures
173 132 Dominic Cleal
## @gpg --homedir gnupg/1.2 -b -u packages@theforeman.org foreman-1.2.0.tar.bz2@ (requires 1.2 release GPG key generated above)
174 185 Tomer Brisker
# If for some reason there was an issue with the tarballs that required uploading new tarballs, CDN cache should be invalidated so that the builders use the updated tarballs.
175 132 Dominic Cleal
176
h2. Packaging a release
177 1 Dominic Cleal
178 132 Dominic Cleal
# In foreman-packaging rpm/1.2 branch, change foreman.spec, foreman-proxy.spec, foreman-selinux.spec, foreman-installer.spec:
179
## set version to "1.2.0"
180
## For RCs: set release to "0.1%{?dotalpha....", uncomment alphatag* (replace # with %), change to RC1
181 52 Dominic Cleal
## For non-RCs: set release to "1%{?dotalpha....", comment alphatag* (replace % with #)
182 184 Ondřej Pražák
## in each package dir, remove the old tarball, run @spectool -g *.spec@ and @git annex add *.tar.bz2@
183 115 Dominic Cleal
## commit with message "Release 1.2.0"
184 132 Dominic Cleal
## perform RPM scratch build of each project (see [[RPM_Packaging]])
185 159 Dominic Cleal
## tag each project with @tito tag --keep-version --changelog "Release 1.2.0"@
186 178 Ondřej Pražák
# Create new branches as described in [[Debian_Packaging#foreman]]
187
# Update changelog files for debs:
188 132 Dominic Cleal
## @scripts/changelog.rb -v 1.2.0-1 -m "1.2.0 released" debian/*/*/changelog@
189 115 Dominic Cleal
# Trigger next step of release pipeline: @release_packages@
190 138 Dominic Cleal
# Use [[RPM_Packaging]] to sign RPMs
191 132 Dominic Cleal
# Trigger next step of release pipeline: @release_mash@
192 149 Dominic Cleal
# Trigger next step of release pipeline: @release_test@
193 115 Dominic Cleal
# Trigger next step of release pipeline: @release_push_deb@ and @release_push_rpm@
194 132 Dominic Cleal
195 41 Dominic Cleal
h2. Completion tasks
196 132 Dominic Cleal
197
If releasing the first non-RC release (i.e. 1.2.0), start with:
198 137 Dominic Cleal
# update @$stable@ class parameter on @web@ class to point to the new version number (see @foreman-infra/puppet/modules/web/manifests/init.pp@)
199
# deb.theforeman.org, changing stable:
200
## change web's /var/www/freight/apt/*/stable symlinks to the new version number
201
## @rm -rf /var/www/vhosts/deb/htdocs/pool/*/stable/@
202 132 Dominic Cleal
## re-run freight cache
203
# Update theforeman.org:
204
## @documentation.md@ change default version
205 166 Dominic Cleal
## @_config.yml@ foreman_version and quickstart link
206 1 Dominic Cleal
## @_layouts/manual.html@ latestversion
207
## @plugins/index.md@ version
208 158 Dominic Cleal
## add a blog post describing release highlights etc.
209 203 Tomer Brisker
# If there is any open version on https://projects.theforeman.org/projects/foreman/settings/versions for y-2 (e.g. 1.0.2 when releasing 1.2.0) that hasn't been released, close the version and move any issues assigned to it to the next y-1 release (e.g. 1.1.3).
210 132 Dominic Cleal
211
For all releases:
212 156 Dominic Cleal
213 154 Dominic Cleal
# Update theforeman.org:
214 173 Ewoud Kohl van Wijngaarden
## release notes from earlier steps (use scripts/release_notes.rb in theforeman.org repository)
215 154 Dominic Cleal
## @_includes/version.html@
216
## @_includes/latest_news.html@ entry
217
## @security.md@ for any released CVE fixes
218 182 Daniel Lobato Garcia
# Publish announcement notice in Release Announcements: (https://community.theforeman.org/c/release-announcements)
219 1 Dominic Cleal
# Link to announcement post on IRC, Twitter, Google+ and update IRC /topic
220 182 Daniel Lobato Garcia
# Update the release schedule on [[Development_Resources]] to enter the date and for the following release(s)
221 160 Dominic Cleal
# Complete [[Security Process]] for any released CVE fixes (oss-security list)
222 1 Dominic Cleal
# Final only: Update Wikipedia Foreman entry at: https://en.wikipedia.org/wiki/Foreman_(software)