Lukas Zapletal wrote:

Ubuntu 20.10 no longer supports the preseed installation but there is a new cloud-init mechanism instead.

What steps would be required to add this? We've used Foreman for a long while and have hit this with an LTS point release on 20.04.2, since they've dropped what they consider legacy debian-installer support for this patch release during their 5 year LTS support period (ugh).

From a quick look at the sources it looks like we'd need to add support for newer Ubuntu in `app/models/operatingsystems/ubuntu.rb` (or similar), and create a bunch of new provisioning templates to support the autoinstall method. Then add unit tests to ensure this works as intended.

Anything else this is likely to need?

I'm happy to have a go at adding basic support for Ubuntu 20.04+ if it would be helpful (and if I can make time to do so!)

Edit : And new Dell desktop hardware requires a newer kernel than provided by Ubuntu 20.04.1 to support the network interface, so we need to fix this to continue using Foreman for building our desktop PCs.

Hello,

the first step is to add token support for userdata_controller. It needs to work the same way as in unattended_controller, meaning /cloud-init/?token=XYZ must find a host with a valid token XYZ.

Then we need to create new set of PXE templates for Ubuntu (PXELinux, PXEGrub2, iPXE) that will use ds=http://foreman/cloud-init/?token=TOKEN (I got the syntax incorrect). Alternatively, we can use the current templates and do a check "if this is ubuntu and version is 20.04 and higher than use this instead preseed".

Finally, we need a change to allow Foreman to find PXE files (kernel/initramdisk) on the HTTP mirrors so these can be passed into TFTP Smart Proxy for download when new hosts are created.

Feel free to ping me on IRC (lzap) if you need assistance, review or anything.

the first step is to add token support for userdata_controller. It needs to work the same way as in unattended_controller, meaning /cloud-init/?token=XYZ must find a host with a valid token XYZ.

I'm keenly interested in this as well, as I need to provision ~3k+ machines with Ubuntu 20.04.3 and later releases, and the legacy d-i installer will no longer be used in any future Ubuntu releases. It has been deprecated for those releases.

Part of the challenge here is that the remote cloud-init data that is fetched by the UEFI boot image, must end in the name 'user-data' (or be referred to as such) and must also be preceded by an empty file called `meta-data` at the same URI as the `user-data` YAML.

The Foreman 2.3.1 I'm working with doesn't let me specify two cloud-init templates for a single host, nor the order they need to be made available in at boot time (eg, empty meta-data file first, user-data second).

It does also appear that even thought the path to the 'focal' repository is correct, the images being requested by default by Foreman's templates, are not what is needed to successfully UEFI/PXE boot the host.

Alternatively, we can use the current templates and do a check "if this is ubuntu and version is 20.04 and higher than use this instead preseed".

That means you'll have to create a very large, complex (and duplicated content) template for both d-i and another for cloud-init of the same type/parameters. I would prefer to keep those in separate templates, so hosts can be provisioned with 18.04 (d-i) or 20.04 (subiquity/cloud-init), and not have to decide with some "detection", which one it should use.

Finally, we need a change to allow Foreman to find PXE files (kernel/initramdisk) on the HTTP mirrors so these can be passed into TFTP Smart Proxy for download when new hosts are created.

I tried to hard-code this into a template with no macros or Jinja parsing, and it still results in a machine that attempts to boot, then goes to a black screen, no output whatsoever, and hangs there indefinitely, until the machine is power-cycled via the BMC/iDrac.

Feel free to ping me on IRC (lzap) if you need assistance, review or anything.

I pinged you there earlier today (not sure what TZ you're in), but I'll lurk until I can catch up with you real-time and work through this. This is a VERY high priority to resolve on my side.

Thanks!

In case you don't know, Foreman already supports /cloud-init endpoint, I am not sure what you refer to when you mention "must end with /cloud-init". Do you mean that the token parameter must not be used?

Anyway, please take my comments with grain of salt - I am just brainstorming here. I haven't tried neither did I test the new Ubuntu installation workflow.

Anyway, if you really need this and you don't have engineering capacity, I can recommend iRonin Poland who successfully delivered several features for our users. Kamil should get you connected: https://github.com/kamils-iRonin

Here's a how-to like document, which is practical for an initial setup:
https://louwrentius.com/understanding-the-ubuntu-2004-lts-server-autoinstaller.html

To give you guys a little update:

Foreman cannot download PXE files because Canonical no longer publishes them directly. A huge ISO file needs to be downloaded, mounted and PXE files extracted. This does not work well with how Foreman is currently designed. We are looking into maybe rearchitecting how PXE files are handled but this is not going to happen soon: https://community.theforeman.org/t/rfc-rearchitecting-how-pxe-files-are-deployed/27021/4

For now, we are working on some templates that could be a good starting point for ya all, there is a tutorial emerging here: https://community.theforeman.org/t/autoinstalling-ubuntu-server-20-04-3-from-the-live-iso/27050 and hopefully soon we will have official documentation on how to do Autoinstall provisioning.

For now, you will have to download PXE files (extract them from the ISO) and putting them to the TFTP/HTTP directory manually but other than that it should work. I will soon (today) create a PR with initial versions of templates so feel free to review the pull request and test them.