The Foreman project maintains its own Jenkins instance for continuous integration at http://ci.theforeman.org/
Pull request testing¶
Core Foreman projects have GitHub pull requests tested on our Jenkins instance so it's identical to the way we test the primary development branches themselves. Less significant projects (such as installer submodules) may use Travis CI.
Every project that needs PR testing requires two Jenkins jobs. Taking core Foreman as an example:
- Test job for the main development branch (develop or master): test_develop
- Test job taking a single "pr_number" parameter that applies a patch and tests: test_develop_pull_request
The PR test job takes the PR number parameter, downloads the patch from GitHub (by appending a .patch extension to the PR URL), applies it to the local checkout of the project and then builds as normal. This process means PRs are effectively rebased onto the current development branch before tests are run, rather than testing the branch as-is. GitHub tracks "mergeability" so we don't test PRs that can't be merged cleanly.
The results from these PR jobs are only stored for a few weeks, sufficient for reviews.
To initiate the PR tests, the test-pull-requests script is used to scan for open PRs, check they're mergeable and then trigger the Jenkins job.