Foreman » Smart Proxy

It turns out that ISC DHCP parser does not respect both expired and free leases. To give some basic background, ISC daemon is only appending into the file during runtime (it does reformat it explicitly from time to time, by default only during start). This means that leases are only appended and if a lease is released explicitly, new lease entry is added with the "free" binding state. Our codebase only skips this record, but if the lease was previously "active", it is not removed and Foreman sees the lease still as "active" which is causing conflicts during Discovery (OpenStack installer).

More than that, Foreman Proxy somehow incorrectly parse date. Leases expire also via "ends" entry, which is also not respected by the proxy. Adding the following in our test/fixtures/dhcp/dhcp.leases causes tests to fail:

# The following six leases are from booting a real-world DELL hardware with
# PXE using Foreman Discovery. Fist, a BIOS/UEFI DHCP request is sent along
# with an UID:
lease 192.168.122.81 {
  starts 4 2014/10/16 10:14:40;
  ends 4 2014/10/16 10:24:40;
  tstp 4 2014/10/16 10:24:40;
  cltt 4 2014/10/16 10:14:40;
  binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
}
lease 192.168.122.81 {
  starts 4 2014/10/16 12:53:16;
  ends 4 2014/10/16 13:03:16;
  cltt 4 2014/10/16 12:53:16;
  binding state active;
  next binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
  uid "\000DELLX\000\020W\200L\310\300O022";
}
# And it is released by the client.
lease 192.168.122.81 {
  starts 4 2014/10/16 12:53:16;
  ends 4 2014/10/16 12:54:03;
  tstp 4 2014/10/16 12:54:03;
  cltt 4 2014/10/16 12:53:16;
  binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
  uid "\000DELLX\000\020W\200L\310\300O022";
}
# Then Dracut acquires another lease, now without the UID:
lease 192.168.122.88 {
  starts 4 2014/10/16 12:54:03;
  ends 4 2014/10/16 13:04:03;
  cltt 4 2014/10/16 12:54:03;
  binding state active;
  next binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
}
# And then OS acquires another lease, now without the UID but with
# a client-hostname set:
lease 192.168.122.89 {
  starts 4 2014/10/16 12:55:22;
  ends 4 2014/10/16 13:05:22;
  cltt 4 2014/10/16 12:55:22;
  binding state active;
  next binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
  client-hostname "localhost";
}
lease 192.168.122.89 {
  starts 4 2014/10/16 12:59:40;
  ends 4 2199/01/01 00:00:01;
  cltt 4 2014/10/16 12:59:40;
  binding state active;
  next binding state free;
  hardware ethernet ec:f4:bb:c6:ca:fe;
  client-hostname "localhost";
}
# This section should be parsed as follows:
# .81 - free
# .88 - free (still active, but expired to this date)
# .89 - active (very long lease set with expiration in 2199)

As you can see, the test fails with returned value of:

  1) Failure:
DhcpApiTest#test_api_02_get_network [/home/lzap/work/smart-proxy/test/dhcp/dhcp_api_test.rb:65]:
<{"reservations"=>
  [{"hostname"=>"test.example.com",
    "ip"=>"192.168.122.1",
    "mac"=>"00:11:bb:cc:dd:ee"}],
 "leases"=>
  [{"ip"=>"192.168.122.2",
    "mac"=>"00:aa:bb:cc:dd:ee",
    "starts"=>"2014-07-12 10:08:29 UTC",
    "ends"=>nil,
    "state"=>"active"}]}> expected but was
<{"reservations"=>
  [{"hostname"=>"test.example.com",
    "ip"=>"192.168.122.1",
    "mac"=>"00:11:bb:cc:dd:ee"}],
 "leases"=>
  [{"ip"=>"192.168.122.2",
    "mac"=>"00:aa:bb:cc:dd:ee",
    "starts"=>"2014-07-12 10:08:29 UTC",
    "ends"=>nil,
    "state"=>"active"},
   {"ip"=>"192.168.122.81",
    "mac"=>"ec:f4:bb:c6:ca:fe",
    "starts"=>"2014-10-16 12:53:16 UTC",
    "ends"=>"2014-10-16 13:03:16 UTC",
    "state"=>"active"},
   {"ip"=>"192.168.122.88",
    "mac"=>"ec:f4:bb:c6:ca:fe",
    "starts"=>"2014-10-16 12:54:03 UTC",
    "ends"=>"2014-10-16 13:04:03 UTC",
    "state"=>"active"},
   {"ip"=>"192.168.122.89",
    "mac"=>"ec:f4:bb:c6:ca:fe",
    "starts"=>"2014-10-16 12:55:22 UTC",
    "ends"=>"2014-10-16 13:05:22 UTC",
    "state"=>"active"},
   {"ip"=>"192.168.122.89",
    "mac"=>"ec:f4:bb:c6:ca:fe",
    "starts"=>"2014-10-16 12:59:40 UTC",
    "ends"=>"2199-01-01 00:00:01 UTC",
    "state"=>"active"}]}>.

Both .81 and .88 are considered free by ISC DHCP (see dhcpd.leases man page for more details) but Foreman Proxy does report those as "active".

The workaround is to restart the dhcpd daemon from time to time.