I've also found that sometimes A records aren't created either. The problem for me seems to be that the smart proxy updates DNS via one DNS server, but the changes aren't replicated (via AD) to all the other DNS servers foreman has in its resolv.conf until several seconds to a few minutes later.

Reording the nameserver entries in resolv.conf has worked so far. I'm not sure what happens if DNS is temporarily down on the 'primary' server though. The resolver tries the DNS servers in the order they appear in the resolv.conf, but after it moves on to using the 2nd or 3rd entry does it ever switch back to using the first again?

Dominic Cleal wrote:

I think this might be the effect of #2668, where Foreman only uses the caching resolver to check for PTR records, but it uses the SOA for A records.

The rebuild will check if the record's already there before trying to create it, so it might see a cached record (even after deletion) and skip the step.

Just ran into this problem as well, when re-orchestrating a host the changes don't quite manage to propagate from the Infoblox master to the SOA NSes quick enough for foreman.
Which meant dropped PTR records, even losing several A and AAAA records for a few machines.

Is there any particular reason why Foreman doesn't use the naive workflow of "Delete, and then always create anew"?

In foreman/app/models/concerns/dns_interface.rb on line 37;

  def recreate_dns_record(type)
    return true if dns_record(type).nil? || dns_record(type).valid?
    set_dns_record(type)
  end

Wouldn't it be possible to just skip the second check? Since the function is only ever called right after - successfully - deleting the record in question.

Sorry for the activity on such an old bug, but I ran into this same situation today on 3.5.0-rc1 and it was thoroughly frustrating. In my case, the cause was (I think; haven't verified) replication lag between two FreeIPA DNS servers, causing a DNS record to appear present immediately after deleting it. The result was that each attempt at rebuilding caused a different, seemingly random set of records, both PTR and A, to be lost.

It doesn't help that there's nothing informative in either the Foreman or proxy logs - just the old records being deleted and no indication that re-creating them is being skipped or why.

I want to re-emphasize this suggestion (from 5 years ago!):

Alexander Olofsson wrote:

Wouldn't it be possible to just skip the second check? Since the function is only ever called right after - successfully - deleting the record in question.

I tried making this change locally, and it resolved the issue.

I always wondered about conflicts in Foreman. I share similar opinions as Alexander.

IMHO Foreman itself shouldn't do any DNS conflict detection. If anything, it should be delegated to the correct Smart Proxy which can have an accurate view. It can bypass DNS caches by going straight to the source.

Additionally the Smart Proxy should implement PUT /dns/:value/:type and Foreman should use that instead of DELETE + POST to ensures a certain state. Many DNS servers have an equivalent way, which can make updates more efficient. A single transaction also reduces the number of DNS zone updates, which helps with replication. #35087 is about that.

We should consider rollbacks, but a PUT method could make that more reliable as well.