h1. Using iPXE in Foreman

{{toc}}

TFTP is a slow protocol on high-latency networks, but if your hardware is supported by iPXE (http://ipxe.org/appnote/hardware_drivers) or if UNDI driver of the NIC is compatible with iPXE, it is possible to configure PXELinux to chainboot iPXE and continue booting via HTTP protocol which is fast and reliable.

In this scenario, a system is PXE-booted into PXELinux which chainloads iPXE which continue booting via HTTP. The scenario is:

* hardware is turned on

* PXE driver gets network credentials from DHCP

* PXE driver gets PXELinux firmware from TFTP (pxelinux.0)

* PXELinux searches for configuration file on TFTP

* PXELinux chainloads iPXE (undionly-ipxe.0 or ipxe.lkrn)

* iPXE gets network credentials from DHCP again

* iPXE gets HTTP address from DHCP

* iPXE chainloads the iPXE template from Foreman

* iPXE loads kernel and init RAM disk of the installer

Requirements:

* hardware or VM is BIOS only (no UEFI)

* hardware is capable of PXE booting

* iPXE project has hardware driver for network card or hypervisor already uses iPXE as NIC firmware

* a host entry is created in Foreman

* MAC and IP addresses of the provisioning interface matches with Foreman inventory

* smart-proxy templates plugin is used when there is NAT between Foreman and client

* provisioning interface of the host has a valid DHCP reservation

These workflows won't work with discovery.

h2. Chainbooting virtual machines

Since most virtualization hypervisors use iPXE as the primary firmware for PXE booting, the above configuration will directly work without TFTP and PXELinux involved. This is known to work with libvirt, oVirt and RHEV. If the hypervisor is capable of replacing PXE firmware, it will work too (e.g. VMWare is documented at http://ipxe.org/howto/vmware).

This workflow will only work if *tokens are disabled*, to do that go to Administer - Settings and set "Token duration" to 0. There are two options.

h3. Foreman setup - iPXE templates

* Associate iPXE template 'Kickstart default iPXE' or 'Preseed default iPXE' depending on OS family with your Host or Hostgroup.

* Select the 'Kickstart/Preseed default iPXE' as the default template for an Operating System

* Set PXE Loader of the Host/Hostgroup must be set to "None"

h3. DHCP setup - managed server

In this case, Foreman manages DHCP and reservation (IP address) is known in advance, therefore iPXE script can be found in Foreman DB and returned to a client:

* VM (BIOS) is turned on

* iPXE gets network credentials from DHCP

* iPXE gets filename option from DHCP (URL)

* iPXE chainloads the iPXE template from Foreman

* Foreman renders iPXE template for given host found by remote IP

* iPXE executed the script, loads kernel and init RAM disk of the installer

In the /etc/dhcp/dhcpd.conf file change the "filename" global or subnet configuration as follows:

<pre><code>

option architecture code 93 = unsigned integer 16;

if exists user-class and option user-class = "iPXE" {

  filename "http://FOREMAN_OR_PROXY/unattended/iPXE";

} elsif option architecture = 00:06 {

  filename "grub2/shim.efi";

} elsif option architecture = 00:07 {

  filename "grub2/shim.efi";

} elsif option architecture = 00:09 {

  filename "grub2/shim.efi";

} else {

  filename "pxelinux.0";

}

</code></pre>

h3. DHCP setup - unmanaged server

In this case, Foreman cannot create DHCP reservation in advance. But an intermediate iPXE script can be deployed to report MAC address to find proper host:

* VM (BIOS) is turned on

* iPXE gets network credentials from DHCP

* iPXE gets filename option from DHCP (URL)

* iPXE loads the intermediate iPXE template from a HTTP server

* iPXE executes the intermediate script

* iPXE chainloads the iPXE template from Foreman with MAC address provided as a parameter

* Foreman renders iPXE template for given host found by remote IP

* iPXE executed the script, loads kernel and init RAM disk of the installer

Create the following script and put it somewhere on the network via HTTP so iPXE clients can access it.

<pre><code>

#!ipxe

# Intermediate iPXE script to report MAC address to Foreman

isset ${net0/mac} || goto no_nic

dhcp net0 || goto net1

chain http://FOREMAN_OR_PROXY/unattended/iPXE?mac=${net0/mac} || goto net1

exit 0

:net1

isset ${net1/mac} || goto no_nic

dhcp net1 || goto net2

chain http://FOREMAN_OR_PROXY/unattended/iPXE?mac=${net1/mac} || goto net2

exit 0

:net1

# Create as many copies as necessary (the example will work up to 2 NICs)

:no_nic

echo Failed to chainload from any network interface

sleep 30

exit 1

</code></pre>

Let's use httpd on Foreman server for this purpose:

<pre><code>

# scp intermediate.ipxe root@foreman:/var/www/htdocs/pub/

</code></pre>

On an *unmanaged DHCP server* change filename option to be "http://FOREMAN_OR_PROXY/pub/intermediate.ipxe". Instructions are different for various DHCP servers (like MS DHCP, Infoblox, Bluecoat) but if this was ISC DHCP, then simply change /etc/dhcp/dhcpd.conf file as follows:

<pre><code>

if exists user-class and option user-class = "iPXE" {

  filename "http://FOREMAN_OR_PROXY/pub/intermediate.ipxe";

} else {

  filename "pxelinux.0";

}

</code></pre>

h3. Remarks

* Configuration file dhcpd.conf is under puppet control, it can be overwritten.

* Use Foreman server IP address or smart-proxy IP address with port 8000 instead FOREMAN_OR_PROXY.

* Software iPXE shipped with Red Hat does not have HTTPS support compiled in, use HTTP instead.

* On isolated networks, a Smart Proxy Templates feature must be enabled must be enabled to work around NAT.

* Foreman 1.17 and older will not accept "mac" HTTP option, unless bootdisk plugin is installed, so make sure it is present. Foreman 1.18+ already accepts the "mac" option even if bootdisk plugin is not installed.

* Both managed and unmanaged setups are for BIOS VMs, it is possible to use UEFI VMs but iPXE EFI build must be downloaded separately as it's not part of RHEL package:

<pre><code>

$ wget http://boot.ipxe.org/ipxe.efi -O /var/lib/tftpboot/ipxe.efi

option architecture code 93 = unsigned integer 16;

if exists user-class and option user-class = "iPXE" {

  filename "http://FOREMAN_OR_PROXY/unattended/iPXE";

} elsif option architecture = 00:06 {

  filename "ipxe.efi";

} elsif option architecture = 00:07 {

  filename "ipxe.efi";

} elsif option architecture = 00:09 {

  filename "ipxe.efi";

} else {

  filename "pxelinux.0";

}

</code></pre>

h2. PXELinux chainboots iPXE

In this setup, iPXE uses build-in driver for network communication or UNDI interface. Therefore this will only work on supported cards. This setup will only work on BIOS systems.

h3. TFTP setup

Copy the iPXE firmware to the TFTP root directory:

  cp /usr/share/ipxe/ipxe.lkrn /var/lib/tftpboot/

  cp /usr/share/ipxe/undionly.kpxe /var/lib/tftpboot/undionly-ipxe.0

The source directory can be different on linux distributions, this is for Red Hats. The file is shipped in ipxe-bootimgs package.

Not all hardware is supported by iPXE drivers. In case of troubles, use latest development version build of iPXE:

  wget -O /var/lib/tftpboot/ http://boot.ipxe.org/ipxe.lkrn

Do not use symbolic links as TFTP runs in chroot. When using SELinux, remember to correct file contexts:

  restorecon -RvF /var/lib/tftpboot/

h3. Foreman setup - PXELinux template

* Associate iPXE template "Kickstart default iPXE" or "Preseed default iPXE" depending on OS family with an Operating System.

* Associate templates named "PXELinux chain iPXE" and "PXELinux chain iPXE UNDI" with the same Operating System.

* Select the "Kickstart/Preseed default iPXE" as default for the Operating System.

* Select the "PXELinux chain iPXE" as default for the Operating System.

* Set PXE Loader of the Host/Hostgroup must be set to "PXELinux BIOS"

h3. DHCP setup

The above configuration will lead to an endless loop of chainbooting iPXE firmware. To break this loop, configure DHCP server to hand over correct URL to iPXE to continue booting. In the /etc/dhcp/dhcpd.conf file change the "filename" global or subnet configuration as follows:

<pre><code>

option architecture code 93 = unsigned integer 16;

if exists user-class and option user-class = "iPXE" {

  filename "http://FOREMAN_OR_PROXY/unattended/iPXE";

} elsif option architecture = 00:06 {

  filename "grub2/shim.efi";

} elsif option architecture = 00:07 {

  filename "grub2/shim.efi";

} elsif option architecture = 00:09 {

  filename "grub2/shim.efi";

} else {

  filename "pxelinux.0";

}

</code></pre>

h3. Remarks

* Configuration file dhcpd.conf is under puppet control, it can be overwritten.

* Use Foreman server IP address or smart-proxy IP address with port 8000 instead FOREMAN_OR_PROXY.

* Software iPXE shipped with Red Hat does not have HTTPS support compiled in, use HTTP instead.

* This works only for BIOS, for UEFI hosts use UEFI HTTP booting instead.