Koji

We have a project Koji instance, which lives at http://koji.katello.org/koji/

Request Account

If you want to build in this Koji. You must send a request to tmlcoch (or use jblazek or dcleal as backup) and you'll be sent back a certificate - store it in ~/.katello.cert

Create ~/.koji/config

[kkoji]

;configuration for koji cli tool

;url of XMLRPC server
server = http://koji.katello.org/kojihub

;url of web interface
weburl = http://koji.katello.org/koji

;url of package download site
topurl = http://koji.katello.org/

;path to the koji top directory
;topdir = /mnt/koji

;configuration for SSL athentication

;client certificate
cert = ~/.katello.cert

;certificate of the CA that issued the client certificate
ca = ~/.katello-ca.cert

;certificate of the CA that issued the HTTP server certificate
serverca = ~/.katello-ca.cert

;use SSL instead of Kerberos default
authtype = ssl 

Create ~/.katello-ca.cert

-----BEGIN CERTIFICATE-----
MIIEDzCCAvegAwIBAgIJAJ/i4oLhK2xQMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNV
BAYTAlVTMQswCQYDVQQIEwJOQzEQMA4GA1UEBxMHUmFsZWlnaDEVMBMGA1UEChMM
S2F0ZWxsbyBLb2ppMQswCQYDVQQLEwJDQTEQMA4GA1UEAxMHa29qaSBDQTAeFw0x
NDA3MDgwOTM0MzBaFw0yNDA3MDUwOTM0MzBaMGIxCzAJBgNVBAYTAlVTMQswCQYD
VQQIEwJOQzEQMA4GA1UEBxMHUmFsZWlnaDEVMBMGA1UEChMMS2F0ZWxsbyBLb2pp
MQswCQYDVQQLEwJDQTEQMA4GA1UEAxMHa29qaSBDQTCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBAMvihJFUzcwvIjip+ACWYoVvOlAecSZnox1FSezBPiWm
280FrYf2wN+2s/FWvlDqUheAX03/DcZBPjHiB43v8AJwF3VNCGFu3CaqkQsg1T5s
vw6pvmL0D5gj/wy25OnSOQIGN1EPnb+paCSCN7BRN5DJTk/jDX4ebIbhKFWgC6IH
Jg3jmWvf3ROHXkSumMNMvLtCC91/Y6a28zBu91tL6uCTYBp7G/rVq0hG0doCMrrb
gShVuNq2uoPvQXasjE54Nsd4PnkFbcX+pQcDnhlORBHTVk/m98t+ArOnWBKKQlQh
88jPpVso/jKy6h8R3YHUZY3QY4G3lB7qPJRCI4SaRf8CAwEAAaOBxzCBxDAdBgNV
HQ4EFgQUWaJRvcA/+GaPABgzU0wydC1WDlgwgZQGA1UdIwSBjDCBiYAUWaJRvcA/
+GaPABgzU0wydC1WDlihZqRkMGIxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJOQzEQ
MA4GA1UEBxMHUmFsZWlnaDEVMBMGA1UEChMMS2F0ZWxsbyBLb2ppMQswCQYDVQQL
EwJDQTEQMA4GA1UEAxMHa29qaSBDQYIJAJ/i4oLhK2xQMAwGA1UdEwQFMAMBAf8w
DQYJKoZIhvcNAQEFBQADggEBAItZgucwDLFJI5U9ejzJXx6yw61/qlkqwNpN5608
PCuz4zdiIH6Ju8dCaOdnWF19aQdASUxc2xjyRs2X6qqZvaX9j4lZXUHxXrGpXTiu
BUJ4OsM3oDbyOvwj3EhK7+rSzjI3Cwx0V2n4xpnCLmfExlMt3Z/LreNUY4bKws0u
nRImQExCmyy+ubY5kGUPEOp8fLhoXVpNyWUX0eTjbiWeqwiFIBgpZrLYFnq0SVCF
lriG9rF2DdvfJFAxZOvbed3EFV/9FrEHhUD8cRdZZMteUh/LzNJZ7JVXTi5lC0Bz
fMKMQGtd6DocgHLpv+5uJg8rAz8bAA3TmnHHk0SCu0iI+tw=
-----END CERTIFICATE-----

Set up a kkoji alias

Using the "kkoji" section created above in the koji config file, aliasing "kkoji" to the koji binary will cause it to use this section.

mkdir ~/bin
ln -s /usr/bin/koji ~/bin/kkoji

If you have an old version of the koji client, you may need to use a bash alias instead. (Symlink approach is preferable as it works with xargs.)

Login to Koji WebUI

You do not need to be logged to WebUI. But it is useful if you want to cancel or resubmit task. Or set up notifications. Run:

openssl pkcs12 -export -inkey ~/.katello.cert -in ~/.katello.cert -CAfile ~/.katello-ca.cert -out katello_browser_cert.p12

When prompted for a password, either leave it blank or specify one which is used to secure the output file - you then need to supply it again on import.

And in Firefox do: {{{ Edit -> Preferences -> Advanced -> Encryption -> View Certificates -> Your Certificates -> Import... }}}

Foreman specifics

These live on RPM_Packaging and in https://github.com/theforeman/foreman-packaging/tree/rpm/develop#readme

Katello specifics: Building Packages

If using SCL

Install scl-utils and scl-utils-build

sudo yum install scl-utils scl-utils-build -y

Don't Forget To Tag Your Changes

tito tag --keep-version
git push && git push --tags

Using TITO

First, make sure you have in {{{~/.titorc}}} this line:

KOJI_OPTIONS=-c ~/.koji/katello-config build --nowait

Then in git directory with spec file (e.g ./agent/) run this:

tito release koji

This will build package on all configured platforms (i.e. EL6, F16, F17).

If you want to build only for one platform, do:

ONLY_TAGS="katello-nightly-rhel6" tito release koji

By hand

You can build directly using CLI of koji. This is mostly useful for scratch builds. Scratch build is build in the same environments as other packages, but the build is not tagged and preserved only few days and does not propagate to final yum repo.

 tito build --test --srpm --dist=.el6
 koji -c ~/.koji/katello-config build --scratch <name-of-tag> <name-of-src-rpm>

List of available tags - http://koji.katello.org/koji/tags
All supporting packages (i.e. those we are not upstream - eg. rubygems) should be build into thirdparty tags e.g. katello-thirdparty-fedora16. Our own packages, e.g. katello* should be build into nightly tags, e.g. katello-nightly-fedora16.

If you need some package in buildroot you need to tag build into *-override tag:

koji -c ~/.koji/katello-config tag-pkg katello-nightly-rhel6-override <name-of-build>

Tip: if you want to submit quick "scratch" builds more often, those aliases can help you. Just run them in the directory with spec:

alias scratchbuild='koji -c ~/.koji/katello-config build --scratch --nowait katello-nightly-rhel6 $(tito build --offline --test --srpm --dist=.el6 | tail -n1 | sed "s/^Wrote: //g")'
alias scratchbuild-f16='koji -c ~/.koji/katello-config build --scratch --nowait katello-nightly-fedora16 $(tito build --offline --test --srpm --dist=.fc16 | tail -n1 | sed "s/^Wrote: //g")'
alias scratchbuild-f17='koji -c ~/.koji/katello-config build --scratch --nowait katello-nightly-fedora17 $(tito build --offline --test --srpm --dist=.fc17 | tail -n1 | sed "s/^Wrote: //g")'
alias scratchbuild-f18='koji -c ~/.koji/katello-config build --scratch --nowait katello-nightly-fedora18 $(tito build --offline --test --srpm --dist=.fc18 | tail -n1 | sed "s/^Wrote: //g")'

Tag first-time package

Please first check whether the package is already present in Fedora or EPEL and only add it if package is not present there!

If you created new package and want to allow it to be build into some tag you have to tag it first:

koji -c ~/.koji/katello-config add-pkg --owner=msuchy TAG rubygem-rest-client

Adding new dependecy for Katello

When you are introducing new dependency for Katelo, you have to put spec file and tar.gz or gem into [#Katellodeps katello-thirdparty.git] and [#Tagfirst-timepackage tag if for first time]. For TAG use katello-thirdparty-rhel6, katello-thirdparty-fedora16 and katello-thirdparty-fedora17.
If your dependency needs to build only e.g for RHEL6, rhen open file rel-eng/tito.props in katello-thirdparty.git and add name of your package to blacklist in sections where you do not want to build your package - e.g.: katello-thirdparty-fedora17 and katello-thirdparty-fedora16.
Once done, run "tito release koji". Check if your package build successfully and if so, add it to [#Compsfiles comps files].
Do not forget to add to spec file of package, which requires new dependency either {{{Requires: newpackage}}} or {{{BuildRequires: newpackage}}} or both.

Repos

Repositories are generated automatically, to manually run it use '''/usr/local/bin/katello-mash-split.py''' on the koji box.

Final repositories can be accessed at http://koji.katello.org/releases/

And can be rsynced using:

rsync -alHvr rsync://koji.katello.org/releases /tmp/katello

Repos are now regulary (after each mashing) synced by lzap to http://fedorapeople.org/groups/katello/releases/ which is public faced. The cronjob script is:

40 7,13,18,21 * * * ssh fedorapeople.org 'cd /project/katello/bin && sh download-repos-from-koji'

If you want to get writte access to fedorahosted.org /project/katello directory, ask Fedora Infrastructure team for ACLs.

Comps files

What will end up in yum repository is determined by comps files (located in katello git repo in {{{./rel-eng/comps/}}} ). During process called meshing, koji will will lookup for latest builds of those packages listed in comps file and will put it in yum repo.
Note that nightly tag inherit everything from thirdparty tag. E.g: http://koji.katello.org/koji/taginfo?tagID=3

Once we will be close to release we will tag everything in nightly to e.g. katello-1.0-rhel6 and after release lock those tags so no one can retrospectively change already released packages by accident.

Troubleshoting

It may sometime happen that you get 404 error during build that package could not be downloaded (e.g python). This happen when package in external repo is updated and koji does not detect that.
In such case run:

koji -c ~/.koji/katello-config  regen-repo katello-nightly-fedora16-build

If you have some package which should be build only for e.g. rhel6 and you want to prevent building it for fedora when someone trigger {{{tito release koji}}}, then you have to add it to {{{./rel-eng/tito.props}}}

[katello-nightly-rhel6]
disttag = .el6
blacklist=

[katello-nightly-fedora16]
disttag = .fc16
blacklist=name-of-blacklisted-package

[katello-nightly-fedora17]
disttag = .fc17
blacklist=name-of-blacklisted-package

Helper scripts

We have some helper scripts in ./rel-eng/ directory. This is our own scripts and does not come with koji.

  • ./rel-eng/git-untagged-commits.pl - shows you which package are not tagged. You may want to "tito tag" this packages, to get new builds.
  • ./rel-eng/build-missing-builds-in-koji.sh - it will submit package to koji for building if git has newer evr of tagged package than koji.

If you do not know differences between package, build, tag, etc. I recommend you "ENG114 - Brew Build System" course in [https://rhsaba.redhat.com/Saba/Web/Employees Red Hat University] - it take 2 hours and is very helpful.

Infrastructure