Upgrading from Puppet 3 to 4¶

This wiki page was written at the release of Foreman 1.12 and should be used at your own risk.

• Table of contents
• Upgrading from Puppet 3 to 4
  • Planning
  • 1. Beginning the upgrade
    • 1a. Install new PC1 packages
    • 1b. Environments, SSL and Apache
    • 1c. Choose foreman-installer or by hand
  • 2. Upgrading with foreman-installer
  • 3. Upgrading by hand
    • 3a. Puppet Config files
    • 3b. ENC files
    • 3c. Foreman settings
    • 3d. Smart proxy settings
    • 3e. Foreman installer
  • Further reading

This wiki page is a rough guide on how to upgrade from Puppet 3 to Puppet 4 when using Foreman. It doesn't replace Puppet's own documentation - you need to take both into account, and it's not for the faint-hearted.

Remember, Puppet is a separate piece of software to Foreman. Foreman integrates with Puppet in only a few places (e.g. reports, ENC and smart proxy class imports) but the Foreman installer may have set it up for you initially. Most of the work is changing the Puppet installation and then updating paths and configs in Foreman to suit.

If you're unfamiliar with how Puppet works, then you should consider setting up a new installation and migrating hosts instead.

Planning¶

Ensure you are running Foreman 1.12 or higher, previous versions are not compatible with Puppet 4, while 1.12 is compatible with both 3 and 4, so carry out that upgrade first. See Upgrading to Foreman 1.12 for more information.

Start with Puppet 3.x to 4.x: Get upgrade-ready - it has many excellent points, including:

• As with any upgrade, the smaller the step, the easier it will be. Ensure you've already upgraded to the latest 3.x release and fixed any deprecations from the release notes before moving to 4. This will save time later.
• Ensure your Puppet modules are going to be compatible with Puppet 4's new ("future") parser
• Plan to upgrade your masters before your agents, because the master can serve older agents, but not the other way around.
• Back up everything, especially SSL keys and certificates.
• Ensure you have enough RAM, Puppet Server defaults to requiring at least 2GB (or set --puppet-server-max-active-instances=1 --puppet-server-jvm-min-heap-size=1G --puppet-server-jvm-max-heap-size=1G to reduce it)

The guide will assume you're using regular 'puppet' packages either from your OS (or EPEL) or from Puppet Labs repositories. Puppet 4 packages are All-In-One (AIO) packages and work quite differently, introducing lots of new paths for config files and binaries. More information on these at:

• Welcome to Puppet Collections
• About Puppet collections and packages

1. Beginning the upgrade¶

These steps should be carried out whether using the installer or updating by hand.

1a. Install new PC1 packages¶

1. Configure the new PC1 repositories with the Using Puppet Collections instructions.

On EL:

1. run yum remove puppet-server to prevent later conflicts
2. run yum install puppetserver which should replace facter, puppet and puppet-server with puppetserver and puppet-agent

On Debian/Ubuntu:

1. run apt-get install puppetserver puppet-agent which should facter, puppet and puppetmaster with puppetserver and puppet-agent

1b. Environments, SSL and Apache¶

This section is based on Puppet 3.x to 4.x: Upgrade Puppet Server which goes into far more detail.

1. Move or copy any environments from /etc/puppet/environments to /etc/puppetlabs/code/environments
2. Move or copy all SSL keys and certificates from /var/lib/puppet/ssl to /etc/puppetlabs/puppet/ssl
3. Move or copy cached data (if there's any) from /var/lib/puppet/foreman_cache_data to /opt/puppetlabs/puppet/cache/foreman_cache_data
4. Remove the Puppet master VirtualHost from Apache at /etc/httpd/conf.d/25-puppet.conf (EL) or a2dissite 25-puppet (Debian/Ubuntu)
5. Remove 8140 from the Apache ports in /etc/httpd/conf/ports.conf or /etc/apache2/ports.conf
6. Update SSL paths in /etc/httpd/conf.d/05-foreman-ssl.conf or /etc/apache2/sites-available/05-foreman-ssl.conf, changing /var/lib/puppet/ssl to /etc/puppetlabs/puppet/ssl
7. Restart httpd/apache2 to free up the port

1c. Choose foreman-installer or by hand¶

Choose one of the two following sections - either update all configuration files using foreman-installer or by hand.

2. Upgrading with foreman-installer¶

If your installation is fairly standard, re-running the Foreman installer is the most reliable way to install and configure Puppet Server correctly. All parameters containing paths that change based on an AIO/non-AIO layout will be reset back to their defaults, which will match the paths files were moved to in the previous section.

First install the puppet-agent-oauth package to prevent warnings running in no-op mode.

foreman-installer --noop -v \
  --puppet-server-implementation=puppetserver \
  --reset-foreman-client-ssl-ca \
  --reset-foreman-client-ssl-cert \
  --reset-foreman-client-ssl-key \
  --reset-foreman-puppet-home \
  --reset-foreman-puppet-ssldir \
  --reset-foreman-server-ssl-ca \
  --reset-foreman-server-ssl-cert \
  --reset-foreman-server-ssl-chain \
  --reset-foreman-server-ssl-crl \
  --reset-foreman-server-ssl-key \
  --reset-foreman-websockets-ssl-cert \
  --reset-foreman-websockets-ssl-key \
  --reset-foreman-proxy-puppet-ssl-ca \
  --reset-foreman-proxy-puppet-ssl-cert \
  --reset-foreman-proxy-puppet-ssl-key \
  --reset-foreman-proxy-puppetca-cmd \
  --reset-foreman-proxy-puppetdir \
  --reset-foreman-proxy-ssl-ca \
  --reset-foreman-proxy-ssl-cert \
  --reset-foreman-proxy-ssl-key \
  --reset-foreman-proxy-ssldir \
  --reset-foreman-puppet-home \
  --reset-puppet-autosign \
  --reset-puppet-client-package \
  --reset-puppet-codedir \
  --reset-puppet-configtimeout \
  --reset-puppet-dir \
  --reset-puppet-logdir \
  --reset-puppet-rundir \
  --reset-puppet-ssldir \
  --reset-puppet-vardir \
  --reset-puppet-server-common-modules-path \
  --reset-puppet-server-default-manifest-path \
  --reset-puppet-server-dir \
  --reset-puppet-server-envs-dir \
  --reset-puppet-server-external-nodes \
  --reset-puppet-server-jruby-gem-home \
  --reset-puppet-server-package \
  --reset-puppet-server-puppetserver-dir \
  --reset-puppet-server-puppetserver-vardir \
  --reset-puppet-server-puppetserver-version \
  --reset-puppet-server-ruby-load-paths \
  --reset-puppet-server-ssl-dir

• Note: in noop mode, you can expect an error of Could not find a suitable provider for foreman_smartproxy to be logged, but this should be fixed on the live run, or install the puppet-agent-oauth package manually.

Keep the --noop -v flags on the first run to check if there are any unexpected changes, then remove it to perform the actual changes.

3. Upgrading by hand¶

If you have many manual customisations to your Foreman or Puppet installations that you wish to preserve, then you may wish to update config files by hand for new paths.

3a. Puppet Config files¶

1. mv /etc/puppet/autosign.conf /etc/puppetlabs/puppet/
2. cp /etc/puppet/puppet.conf /etc/puppetlabs/puppet/puppet.conf and change:
   • in the 'main' section:
     1. vardir = /opt/puppetlabs/puppet/cache
     2. logdir = /var/log/puppetlabs/puppet
     3. rundir = /var/run/puppetlabs
     4. ssldir = /etc/puppetlabs/puppet/ssl
     5. environmentpath = /etc/puppetlabs/code
     6. basemodulepath = /etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
   • in the 'agent' section:
     1. remove configtimeout
   • in the 'master' section:
     1. autosign = /etc/puppetlabs/puppet/autosign.conf { mode = 0644 }
     2. external_nodes = /etc/puppetlabs/puppet/node.rb
     3. ssldir = /etc/puppetlabs/puppet/ssl
3. edit /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf
   1. change master-var-dir to /opt/puppetlabs/puppet/cache
   2. uncomment/set use-legacy-auth-conf: false

If using a smart proxy to import classes, edit /etc/puppetlabs/puppetserver/conf.d/auth.conf, search for /puppet/v3/environments and add a new section below it:

        {
            match-request: {
                path: "/puppet/v3/resource_type" 
                type: path
                method: [get, post]
            }
            allow: "*" 
            sort-order: 500
            name: "puppetlabs resource type" 
        },

And if your Puppet server version is 4.4+, and you're using the proxy to import classes, add this section too:

        {
            match-request: {
                path: "/puppet/v3/environment_classes" 
                type: path
                method: get
            }
            allow: "*" 
            sort-order: 500
            name: "puppetlabs environment classes" 
        },

If you will still support Puppet 3 clients against the server running Puppet 4, see auth.conf rules for Puppet 3 and 4 agents for additional rules. Using foreman-installer (below) will also configure these by default.

Start and enable the puppetserver service with: /opt/puppetlabs/bin/puppet resource service puppetserver ensure=running enable=true

3b. ENC files¶

1. cp /etc/puppet/foreman.yaml /etc/puppetlabs/puppet/foreman.yaml and change:
   1. replace /var/lib/puppet/ssl with /etc/puppetlabs/puppet/ssl
   2. :puppetdir: /opt/puppetlabs/puppet/cache
2. mv /etc/puppet/node.rb /etc/puppetlabs/puppet/

3c. Foreman settings¶

1. edit /etc/foreman/settings.yaml and change :puppetssldir: /etc/puppetlabs/puppet/ssl
2. change websockets_* settings to use /etc/puppetlabs/puppet/ssl and also ssl_* if specified
3. restart Foreman by running touch ~foreman/tmp/restart.txt
4. check in Administer > Settings > Auth in the Foreman UI that SSL certificate, private key and CA file all use /etc/puppetlabs/puppet/ssl, else change them

3d. Smart proxy settings¶

1. edit /etc/foreman-proxy/settings.d/puppet.yml and set :puppet_version to the version of Puppet currently installed, e.g. 4.5.0
   • look up the version of puppet-agent (rpm -q puppet-agent or dpkg -l puppet-agent) and check Release contents
2. edit /etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml and change /var/lib/puppet/ssl to /etc/puppetlabs/puppet/ssl
3. edit /etc/foreman-proxy/settings.d/puppetca.yml and change:
   1. :ssldir: /etc/puppetlabs/puppet/ssl
   2. :puppetdir: /etc/puppetlabs/puppet
4. edit /etc/foreman-proxy/settings.yml and change /var/lib/puppet/ssl to /etc/puppetlabs/puppet/ssl
5. service foreman-proxy restart

3e. Foreman installer¶

Foreman installer stores many paths in its answers file, so if you plan to run it again in future, these need to be changed. See the earlier section on upgrading with Foreman installer to update them.

Further reading¶

• Puppet reference manual
• Puppet Server documentation
• Puppet Server vs Apache/Passenger Puppet master
• Where did everything go in Puppet 4.x?
• puppet-agent: What is it, and what's in it?