I just encountered this issue, and I know how to resolve it outside of The Foreman. Yet I do believe this is an issue that needs to be resolved in The Foreman. Yet, I would not rule out that this issue may be related to dependent host libraries.
I have provisioned many other systems successfully. CentOS 6.6 and 7.1 do not seem to have this issue, even when there is no hostname resolution setup in the DNS. Only CentOS 5.11 seems to exhibit this issue. But I have not tested this extensively.
Here are my details.
I experience this issue under this condition:
- The hostname assigned during kickstart/provision (which is also assigned in the foreman) does not resolve to an IP address because of misconfiguration in DNS.
- The host is named something like: system1.sub1.example.com 10.0.0.1
- But what is put into DNS is something like: system1.sub2.example.com 10.0.0.1
- /etc/resolv.conf has a search domain of "example.com", and only has the default loopback entries including: "127.0.0.1 localhost.localdomain"
- /etc/puppet/puppet.conf has "certname" set to "system1.sub1.example.com"
Log Output: /var/log/foreman/production.log
----
Started POST "/api/hosts/facts" for 10.1.0.1 at 2015-04-28 14:05:18 -0500
Processing by Api::V2::HostsController#facts as JSON
Parameters: {"facts"=>"[FILTERED]", "certname"=>"system1.sub1.example.com", "name"=>"localhost.localdomain", "apiv"=>"v2", :host=>{"name"=>"localhost.localdomain", "certname"=>"system1.sub1.example.com"}}
Import facts for 'localhost.localdomain' completed. Added: 96, Updated: 0, Deleted 0 facts
Saving sit0 NIC for host localhost.localdomain failed, skipping because:
MAC address can't be blank
Completed 201 Created in 1413ms (Views: 7.9ms | ActiveRecord: 0.0ms)
...
Started POST "/api/hosts/facts" for 10.1.0.1 at 2015-04-28 14:08:24 -0500
Processing by Api::V2::HostsController#facts as JSON
Parameters: {"facts"=>"[FILTERED]", "certname"=>"system1.sub1.example.com", "name"=>"system1.sub1.example.com", "apiv"=>"v2", :host=>{"name"=>"system1.sub1.example.com", "certname"=>"system1.sub1.example.com"}}
Import facts for 'localhost.localdomain' completed. Added: 4, Updated: 18, Deleted 0 facts
Saving sit0 NIC for host localhost.localdomain failed, skipping because:
MAC address can't be blank
Completed 201 Created in 284ms (Views: 5.1ms | ActiveRecord: 0.0ms)
...
Started GET "/unattended/built?token=5074d7f5-c5f2-44ce-b5dc-5aef0431a758" for 10.0.0.1 at 2015-04-28 14:12:09 - 0500
Processing by UnattendedController#built as */*
Parameters: {"token"=>"5074d7f5-c5f2-44ce-b5dc-5aef0431a758"}
Found system1.sub1.example.com
unattended: system1.sub1.example.com is Built!
Completed 409 Conflict in 745ms (ActiveRecord: 59.3ms)
----
- Note the error "MAC address can't be blank". Yet the MAC address is added to the foreman host entry. I confirmed this by searching foreman hosts with "mac = XX:XX..." and both "system1.sub1.example.com" and "localhost.localdomain" were in the search results, and both had the same MAC address.
- Also Note, interestingly, the first and second facts POST have identified the host name differently.
Resulting in the issue:
- `hostname` prints the correct FQDN
- /etc/hostname contains the correct FQDN
- `hostname -f` prints the message "hostname: Unknown host" instead of a FQDN
- `facter fqdn` prints the string "system1.example.com", an incorrect FQDN
- `puppet agent --config /etc/puppet/puppet.conf -o --tags no_such_tag <%= @host.puppetmaster.blank? ? '' : "--server #{@host.puppetmaster}" %> --no-daemonize` causes the foreman to register a new host with the hostname "localhost.localdomain", but having the certname "system1.sub1.example.com"
- the foreman host "system1.example.com" cannot finish the build because now there is a conflicting MAC Address in the system - both "system1.example.com" and "localhost.localdomain" have the same MAC address.
- end error is "409 Conflict" when the host notifies the foreman the build is done.
Resolution to fix:
1. To resolve this issue after it occurs:
- In The Foreman, delete the "localhost.localdomain" host entry
- In The Foreman, end the build for "system1.sub1.example.com"
- Note: the certificate for "system1.sub1.example.com" is already signed by the Puppet CA
- The Guest "system1.sub1.example.com" will already be working properly now that build is ended for the associated foreman host entry.
- Note: it is not required to fix the DNS entry to get this working - because Puppet uses Certs for identification, and not hostname resolution.
2. To resolve this issue before it occurs:
- Before provisioning, make sure the hostname resolves to the correct IP, and the IP resolves to the correct hostname. This will make `hostname -f` and `facter fqdn` report the host's FQDN.
- Not tested by me: during kickstart, write the hostname and IP to /etc/hosts
Note: Once the DNS is fixed properly, the following changes:
- `hostname -f` output changes from "hostname: Unknown host" to the true FQDN (e.g. "system1.sub1.example.com")
- `facter fqdn` output changes from e.g. "system1.example.com" to "system1.sub1.example.com"
OS Environment:
- CentOS 5.11 (Guest on a libvirt hypervisor - this does not matter)
- Foreman 1.7.2
- Puppet 3.7.5
- facter 2.4.3
- net-tools 1.60 / hostname 1.100 (2001-04-14)
- Experienced when provisioning the guest
- Provisioning with the Foreman using Static IP address
Infrastructure Environment:
- DNS is Bind 9, available on the local network, and not using foreman to manage DNS
- Not using DHCP, but kickstarting with static IP address