Project

General

Profile

Bug #7384

Foreman creates duplicate hosts based on incorrect facter 'fqdn' resolution

Added by Stephen Benjamin almost 7 years ago. Updated about 6 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Importers
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1129463
Description of problem:
When facter 'fqdn' fact resorts to resolving the domain name from resolv.conf (hostname -f fails), and resolv.conf searches a different domain, a duplicate host will be created, and the host will be stuck in a build loop because the token becomes associated to the new host created by the fact importer.

Version-Release number of selected component (if applicable):
6.0.4 Snap 4

How reproducible:
Always

Steps to Reproduce:
1. Create a new host in Satellite with a different domain than what the DHCP server returns for resolv.conf search/domain

2. Let the host build - when puppet reports for the first time, it will create a new host based on the report upload

3. foreman_built URL call fails

Actual results:

Given newhost123.unicorns.com, with resolv.conf containing:

search rainbows.net
nameserver 1.2.3.4
nameserver 1.2.3.4

And hostname -f returning (no DNS, no /etc/hosts record)

hostname: Unknown host

Foreman will create newhost123.rainbows.net, due to Facter's poor FQDN resolution techniques.

You will have newhost123.rainbows.net (dupe) and newhost123.unicorns.com (original). Foreman_built call will fail.

Expected results:
Foreman report upload relies on certname first. This seems to be a regression from upstream bug http://projects.theforeman.org/issues/1938

Additional info:

Possible fix ideas:

  • Report uploading relies on certname first - why is believing facter FQDN?
  • Default templates create a record in /etc/hosts (fixhosts snippet used for image providers)
foreman_double.png View foreman_double.png 7.08 KB foreman double Slava Bendersky, 11/21/2014 09:32 AM
Foreman double

Related issues

Related to Foreman - Bug #10366: Foreman allows to create invalid hosts upon facts uploadNew2015-05-05

History

#1 Updated by Slava Bendersky over 6 years ago

I am expiperence the same issue, but having problem "how to troubleshoot it"
I have two entries in /etc/resolv.conf and "hostanme -f" with "facter fqdn" return full name of the vm and in web ui get 2 records for for same vm. One shortname another full name.
See attached screenshot.

[root@ca01next14 ~]# facter fqdn
ca01next14.adm.lan
[root@ca01next14 ~]# hostname -f
ca01next14.adm.lan
[root@ca01next14 ~]# 

#2 Updated by Slava Bendersky over 6 years ago

Tried work arround for hosts /etc/hosts, but didn't worked.

#3 Updated by Russell Glaue over 6 years ago

I just encountered this issue, and I know how to resolve it outside of The Foreman. Yet I do believe this is an issue that needs to be resolved in The Foreman. Yet, I would not rule out that this issue may be related to dependent host libraries.

I have provisioned many other systems successfully. CentOS 6.6 and 7.1 do not seem to have this issue, even when there is no hostname resolution setup in the DNS. Only CentOS 5.11 seems to exhibit this issue. But I have not tested this extensively.

Here are my details.

I experience this issue under this condition:
  • The hostname assigned during kickstart/provision (which is also assigned in the foreman) does not resolve to an IP address because of misconfiguration in DNS.
  • The host is named something like: system1.sub1.example.com 10.0.0.1
  • But what is put into DNS is something like: system1.sub2.example.com 10.0.0.1
  • /etc/resolv.conf has a search domain of "example.com", and only has the default loopback entries including: "127.0.0.1 localhost.localdomain"
  • /etc/puppet/puppet.conf has "certname" set to "system1.sub1.example.com"

Log Output: /var/log/foreman/production.log
----
Started POST "/api/hosts/facts" for 10.1.0.1 at 2015-04-28 14:05:18 -0500
Processing by Api::V2::HostsController#facts as JSON
Parameters: {"facts"=>"[FILTERED]", "certname"=>"system1.sub1.example.com", "name"=>"localhost.localdomain", "apiv"=>"v2", :host=>{"name"=>"localhost.localdomain", "certname"=>"system1.sub1.example.com"}}
Import facts for 'localhost.localdomain' completed. Added: 96, Updated: 0, Deleted 0 facts
Saving sit0 NIC for host localhost.localdomain failed, skipping because:
MAC address can't be blank
Completed 201 Created in 1413ms (Views: 7.9ms | ActiveRecord: 0.0ms)

...

Started POST "/api/hosts/facts" for 10.1.0.1 at 2015-04-28 14:08:24 -0500
Processing by Api::V2::HostsController#facts as JSON
Parameters: {"facts"=>"[FILTERED]", "certname"=>"system1.sub1.example.com", "name"=>"system1.sub1.example.com", "apiv"=>"v2", :host=>{"name"=>"system1.sub1.example.com", "certname"=>"system1.sub1.example.com"}}
Import facts for 'localhost.localdomain' completed. Added: 4, Updated: 18, Deleted 0 facts
Saving sit0 NIC for host localhost.localdomain failed, skipping because:
MAC address can't be blank
Completed 201 Created in 284ms (Views: 5.1ms | ActiveRecord: 0.0ms)

...

Started GET "/unattended/built?token=5074d7f5-c5f2-44ce-b5dc-5aef0431a758" for 10.0.0.1 at 2015-04-28 14:12:09 - 0500
Processing by UnattendedController#built as */*
Parameters: {"token"=>"5074d7f5-c5f2-44ce-b5dc-5aef0431a758"}
Found system1.sub1.example.com
unattended: system1.sub1.example.com is Built!
Completed 409 Conflict in 745ms (ActiveRecord: 59.3ms)
----
  • Note the error "MAC address can't be blank". Yet the MAC address is added to the foreman host entry. I confirmed this by searching foreman hosts with "mac = XX:XX..." and both "system1.sub1.example.com" and "localhost.localdomain" were in the search results, and both had the same MAC address.
  • Also Note, interestingly, the first and second facts POST have identified the host name differently.
Resulting in the issue:
  • `hostname` prints the correct FQDN
  • /etc/hostname contains the correct FQDN
  • `hostname -f` prints the message "hostname: Unknown host" instead of a FQDN
  • `facter fqdn` prints the string "system1.example.com", an incorrect FQDN
  • `puppet agent --config /etc/puppet/puppet.conf -o --tags no_such_tag <%= @host.puppetmaster.blank? ? '' : "--server #{@host.puppetmaster}" %> --no-daemonize` causes the foreman to register a new host with the hostname "localhost.localdomain", but having the certname "system1.sub1.example.com"
  • the foreman host "system1.example.com" cannot finish the build because now there is a conflicting MAC Address in the system - both "system1.example.com" and "localhost.localdomain" have the same MAC address.
  • end error is "409 Conflict" when the host notifies the foreman the build is done.
Resolution to fix:
1. To resolve this issue after it occurs:
  • In The Foreman, delete the "localhost.localdomain" host entry
  • In The Foreman, end the build for "system1.sub1.example.com"
  • Note: the certificate for "system1.sub1.example.com" is already signed by the Puppet CA
  • The Guest "system1.sub1.example.com" will already be working properly now that build is ended for the associated foreman host entry.
  • Note: it is not required to fix the DNS entry to get this working - because Puppet uses Certs for identification, and not hostname resolution.
2. To resolve this issue before it occurs:
  • Before provisioning, make sure the hostname resolves to the correct IP, and the IP resolves to the correct hostname. This will make `hostname -f` and `facter fqdn` report the host's FQDN.
  • Not tested by me: during kickstart, write the hostname and IP to /etc/hosts
Note: Once the DNS is fixed properly, the following changes:
  • `hostname -f` output changes from "hostname: Unknown host" to the true FQDN (e.g. "system1.sub1.example.com")
  • `facter fqdn` output changes from e.g. "system1.example.com" to "system1.sub1.example.com"
OS Environment:
  • CentOS 5.11 (Guest on a libvirt hypervisor - this does not matter)
  • Foreman 1.7.2
  • Puppet 3.7.5
  • facter 2.4.3
  • net-tools 1.60 / hostname 1.100 (2001-04-14)
  • Experienced when provisioning the guest
  • Provisioning with the Foreman using Static IP address
Infrastructure Environment:
  • DNS is Bind 9, available on the local network, and not using foreman to manage DNS
  • Not using DHCP, but kickstarting with static IP address

#4 Updated by Dominic Cleal about 6 years ago

  • Category set to Importers

#5 Updated by Russell Glaue about 6 years ago

Regarding the previous comment:

Clarifications about the IPs in "Log Output: /var/log/foreman/production.log" section:

  • 10.0.0.1 is the "system1.sub1.example.com" system being provisioned
  • 10.1.0.1 is the foreman server. Is it the foreman-proxy calling the API with "name"=>"localhost.localdomain"?

Correction in the "Resulting in the issue" section (6th bullet item) - hostname was not correctly identified:

  • the foreman host "system1.sub1.example.com" cannot finish the build because now there is a conflicting MAC Address in the system - both "system1.sub1.example.com" and "localhost.localdomain" have the same MAC address.

#6 Updated by Marek Hulán about 6 years ago

  • Related to Bug #10366: Foreman allows to create invalid hosts upon facts upload added

Also available in: Atom PDF