Foreman » Smart Proxy

Also, is dhcpd server running on the same machine as smart-proxy?

Dmitri Dolguikh wrote:

Also, is dhcpd server running on the same machine as smart-proxy?

no. smart proxy and tftp are on the same server, but dhcp is an external server.

Dmitri Dolguikh wrote:

no. smart proxy and tftp are on the same server, but dhcp is an external server.

ISC dhcpd provider that comes with smart-proxy must be run on the same host as dhcpd process, as the provider monitors changes in leases file (the only complete source of information about the internal state of dhcpd).

So the smart proxy cant not manage an external lease file?

So the smart proxy cant not manage an external lease file?

Smart-proxy doesn't manage lease file, all access is read-only. All modifications to dhcpd state are made via omapi. At the moment smart-proxy must be run on the same host as dhcpd process.

There's https://github.com/theforeman/smart_proxy_dhcp_remote_isc that works with non-local leases file, but it relies on changes to smart-proxy that haven't been merged yet (https://github.com/theforeman/smart-proxy/pull/515). This provider fetches and parses leases file on each http request, therefore smart-proxy's performance when using it can be quite poor. I would suggest moving smart-proxy on the same host as dhcpd process.

Dmitri Dolguikh wrote:

So the smart proxy cant not manage an external lease file?

Smart-proxy doesn't manage lease file, all access is read-only. All modifications to dhcpd state are made via omapi. At the moment smart-proxy must be run on the same host as dhcpd process.

There's https://github.com/theforeman/smart_proxy_dhcp_remote_isc that works with non-local leases file, but it relies on changes to smart-proxy that haven't been merged yet (https://github.com/theforeman/smart-proxy/pull/515). This provider fetches and parses leases file on each http request, therefore smart-proxy's performance when using it can be quite poor. I would suggest moving smart-proxy on the same host as dhcpd process.

We just tested removing (manually) the entry from the lease file (on the external DHCP server) and then adding the same host back (via Foreman), which added an entry to the lease file, so which part does the read-only?

We just tested removing (manually) the entry from the lease file (on the external DHCP server) and then adding the same host back (via Foreman), which added an entry to the lease file, so which part does the read-only?

Smart-proxy never modifies leases file directly. All changes to dhcpd internal state are done via omapi; dhcpd controls how and when its state is persisted.

Dmitri Dolguikh wrote:

We just tested removing (manually) the entry from the lease file (on the external DHCP server) and then adding the same host back (via Foreman), which added an entry to the lease file, so which part does the read-only?

Smart-proxy never modifies leases file directly. All changes to dhcpd internal state are done via omapi; dhcpd controls how and when its state is persisted.

Does this limitation also apply to the TFTP server? In other words, can the TFTP server be a stand alone or does it have to be on the same server as the smart proxy?

I haven't tested tftp module with remote filesystems, it will probably work though.

Dmitri Dolguikh wrote:

I haven't tested tftp module with remote filesystems, it will probably work though.

When will the smart proxy changes and dhcp remote be available? Also, can you provide documentation on how to set it up?

It's available now in Smart-Proxy 1.15-RC1 and 1.16-develop. You'll need to install the provider, which is called smart_proxy_dhcp_remote_isc. The gem is available from rubygems, rpm package should be available in foreman nightlies repository. Please refer to http://projects.theforeman.org/projects/foreman/wiki/How_to_Install_a_Smart-Proxy_Plugin for detailed instructions on how to install a smart-proxy provider. Configuration options are the same as for the core isc dhcp provider, but should be placed in dhcp_remote_isc.yml file.

Dmitri Dolguikh wrote:

It's available now in Smart-Proxy 1.15-RC1 and 1.16-develop. You'll need to install the provider, which is called smart_proxy_dhcp_remote_isc. The gem is available from rubygems, rpm package should be available in foreman nightlies repository. Please refer to http://projects.theforeman.org/projects/foreman/wiki/How_to_Install_a_Smart-Proxy_Plugin for detailed instructions on how to install a smart-proxy provider. Configuration options are the same as for the core isc dhcp provider, but should be placed in dhcp_remote_isc.yml file.

I gave the provider a try, but I keep on hitting the following errors. I did make sure that the lease file does not contain this record, but I still hit these errors.

I, [2017-04-03T08:54:54.403396 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:54 -0700] "GET /tftp/serverName HTTP/1.1" 200 30 0.0059

E, [2017-04-03T08:54:54.521157 ] ERROR -- : No DHCP record for MAC 15.162.32.0/2c:44:fd:94:c1:d0 found
I, [2017-04-03T08:54:54.521696 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:54 -0700] "GET /dhcp/15.162.32.0/mac/2c:44:fd:94:c1:d0 HTTP/1.1" 404 58 0.0036

E, [2017-04-03T08:54:54.626358 ] ERROR -- : No DHCP records for IP 15.162.32.0/15.162.41.241 found
I, [2017-04-03T08:54:54.626818 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:54 -0700] "GET /dhcp/15.162.32.0/ip/15.162.41.241 HTTP/1.1" 404 54 0.0020

I, [2017-04-03T08:54:55.030517 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:55 -0700] "POST /dhcp/15.162.32.0 HTTP/1.1" 200 - 0.1175

I, [2017-04-03T08:54:55.106724 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:55 -0700] "POST /dns/ HTTP/1.1" 404 27 0.0011

W, [2017-04-03T08:54:55.219819 ] WARN -- : DELETE dhcp/:network/:record endpoint has been deprecated and will be removed in future versions. Please use DELETE dhcp/:network/mac/:mac_address or DELETE dhcp/:network/ip/:ip_address instead.
E, [2017-04-03T08:54:55.221339 ] ERROR -- : No DHCP record for 15.162.32.0/2c:44:fd:94:c1:d0 found
I, [2017-04-03T08:54:55.221773 ] INFO -- : 15.162.41.254 - - [03/Apr/2017:08:54:55 -0700] "DELETE /dhcp/15.162.32.0/2c:44:fd:94:c1:d0 HTTP/1.1" 404 54 0.0024

I can only think of two possible issues: the leases file used by dhcpd server is different from the one specified in dhcp_remote_isc provider's configuration file. Another possibility is that dhcp_remote_isc provider isn't being used (check smart-proxy log, it will have which modules and providers have been loaded and configured).

Dmitri Dolguikh wrote:

I can only think of two possible issues: the leases file used by dhcpd server is different from the one specified in dhcp_remote_isc provider's configuration file. Another possibility is that dhcp_remote_isc provider isn't being used (check smart-proxy log, it will have which modules and providers have been loaded and configured).

I, [2017-04-03T08:28:03.323034 ] INFO -- : Successfully initialized 'foreman_proxy'
I, [2017-04-03T08:28:03.323223 ] INFO -- : Successfully initialized 'tftp'
I, [2017-04-03T08:28:03.323318 ] INFO -- : Successfully initialized 'dhcp_remote_isc'
I, [2017-04-03T08:28:03.323377 ] INFO -- : Successfully initialized 'dhcp'
I, [2017-04-03T08:28:03.323430 ] INFO -- : Successfully initialized 'puppetca'
I, [2017-04-03T08:28:03.325035 ] INFO -- : Successfully initialized 'puppet_proxy_legacy'
I, [2017-04-03T08:28:03.325163 ] INFO -- : Successfully initialized 'puppet'
I, [2017-04-03T08:28:03.325249 ] INFO -- : Successfully initialized 'logs'
I, [2017-04-03T08:28:03.351172 ] INFO -- : WEBrick 1.3.1
I, [2017-04-03T08:28:03.351365 ] INFO -- : ruby 2.0.0 (2015-12-16) [x86_64-linux]

1. Enable DHCP management
2. Can be true, false, or http/https to enable just one of the protocols
   :enabled: true

1. valid providers:
2. - dhcp_isc (ISC dhcp server)
3. - dhcp_native_ms (Microsoft native implementation)
4. - dhcp_libvirt (dnsmasq via libvirt)
   :use_provider: dhcp_remote_isc

1. Redhat 5 #
   #:config: /etc/dhcpd.conf #
2. Settings for Ubuntu #
   #:config: /etc/dhcp3/dhcpd.conf
   #:leases: /var/lib/dhcp3/dhcpd.leases

1. Specifies TSIG key name and secret
   #:key_name: secret_key_name
   #:key_secret: secret_key

:omapi_port: 7911
:key_name: omapi_key
:key_secret:<key>

dhcp.yml looks ok to me.

dhcp_remote_isc.yml
:config: /etc/dhcp/dhcpd.conf
:leases: /var/lib/dhcpd/dhcpd.leases #

This looks like the default path to a dhcpd leases file. Are you sure it points to a remote file on the machine where dhcpd is running (i.e. /var/lib/dhcpd is mounted over NFS)?

Dmitri Dolguikh wrote:

dhcp.yml looks ok to me.

dhcp_remote_isc.yml
:config: /etc/dhcp/dhcpd.conf
:leases: /var/lib/dhcpd/dhcpd.leases #

This looks like the default path to a dhcpd leases file. Are you sure it points to a remote file on the machine where dhcpd is running (i.e. /var/lib/dhcpd is mounted over NFS)?

I pass the external dhcp server in the dhcp.yml file

:server: <ip>
:subnets: 15.162.32.0/255.255.224.0

Do I pass it in the dhcp_remote_isc.yml as well?

I pass the external dhcp server in the dhcp.yml file

Server address is required for creating and removing reservations in dhcpd. Dhcp isc provider also requires access to the leases file, the same one that is used by the (remote) dhcpd process. This file is used by dhcpd to persists its internal state, and by smart-proxy to detect all changes to dhcpd's internal state, including the ones smart-proxy initiated. If smart-proxy is running on a separate from dhcpd process machine, leases file must be shared over NFS, Gluster, or similar.

Dmitri Dolguikh wrote:

I pass the external dhcp server in the dhcp.yml file

Server address is required for creating and removing reservations in dhcpd. Dhcp isc provider also requires access to the leases file, the same one that is used by the (remote) dhcpd process. This file is used by dhcpd to persists its internal state, and by smart-proxy to detect all changes to dhcpd's internal state, including the ones smart-proxy initiated. If smart-proxy is running on a separate from dhcpd process machine, leases file must be shared over NFS, Gluster, or similar.

still seeing this message when trying to create a host:

Create Reverse IPv4 DNS record for dl380pg8-57.3pardata.com task failed with the following error: ERF12-2357 [ProxyAPI::ProxyException]: Unable to set DNS entry ([RestClient::ResourceNotFound]: 404 Resource Not Found) for proxy https://set-chef-vm.3pardata.com:8443/dns

Please check smart-proxy log for errors, including startup ones. Make sure dns module and provider have been successfully initialized. I would ask to use irc (freenode #theforeman) for support questions, redmine is mostly for tracking bugs.