Bug #31578
closedExternal IPAM now fails with "mac address cannot be nil" on Compute Resource deployment.
Description
I had successfully integrated phpipam with the new external ipam plugin however, since I upgraded to Foreman 2.3.1 I have noticed that "create host" now shows an error under the interfaces tab even though I am using a vmware compute resource and the mac dialog box is greyed out...
"ERF12-9889 [ProxyAPI::ProxyException]: Unable to retrieve the next available IP for subnet 1.2.3.0/24 External IPAM. ([RuntimeError]: mac address cannot be nil) for proxy https://foreman-proxy:8443/ipam" (values have been changed obviously)
If I change "Deploy On" to "Bare Metal" and and add a fake mac address I get a next_ip, then I can change it back to vmware and I can provision from a template again.
This is not ideal for other people less familiar with Foreman than me.
Not sure if it is related or not, but the IPAM Dashboard has also stopped working, now showing an error...
"ERF12-5974 [ProxyAPI::ProxyException]: Unable to obtain groups from External IPAM. ([TypeError]: no implicit conversion of String into Integer) for proxy https://foreman-proxy:8443/ipam"
I was able to see a list of subnets before.
No changes to the phpipam version or api user have been made since this was working, in fact I actually see a 200 in the phpipam apache log for the Dashboard request.
1.2.3.4 - $USER [05/Jan/2021:13:47:27 0800] "POST /phpipam/api/$USER/user/ HTTP/1.1" 200 116 "" "Ruby"
1.2.3.4 - - [05/Jan/2021:13:47:27 0800] "GET /phpipam/api/$USER/sections/ HTTP/1.1" 200 580 "" "Ruby"
Versions
foreman-2.3.1
foreman-proxy-2.3.1
smart_proxy_ipam (0.1.3)
Please let me know if you require any additional information.
Files
Updated by Lukas Zapletal almost 4 years ago
Hello, can you share the stacktrace? It's probably just some nil pointer exception if you say it was working previously. When using compute resources, MAC is blank, therefore it cannot be passed into IPAM.
Updated by Mark Gargiulo almost 4 years ago
Lukas,
unfortunately even in debug logging mode the log shows nothing more than the actual displayed messages I already listed (surrounded by a bunch of cache statements), there is no ruby stacktrace per se as this seems like an intentional validation error. If you think there is some hidden info I can provide them but they will be lengthy.
Updated by Mark Gargiulo almost 4 years ago
Additional info if it helps narrow things down, is that when this was working previously I was using smart_proxy_ipam 0.0.22 and foreman 2.2.1.
Updated by Lukas Zapletal almost 4 years ago
This looks like an error raised on the smart proxy, check proxy.log.
Updated by Mark Gargiulo almost 4 years ago
Lukas,
I already looked at the proxy.log in debug mode and there were no entries for the in initial "mac address cannot be nil" when first selecting "Create Host" I believe that comes from /usr/share/foreman/lib/proxy_api/external_ipam.rb which is part of the main foreman code.
rpm -qf /usr/share/foreman/lib/proxy_api/external_ipam.rb
foreman-2.3.1-1.el7.noarch
I verified that there was no communication with the proxy at this time by running a tcpdump on port 8443. I see traffic when I select "Bare Metal" and add a fake mac address but nothing when the vmware compute resource is selected and I click on the Interfaces tab.
Updated by Chris Smith almost 4 years ago
I will have a look into this. For the IPAM Dashboard issue, the below PR will fix this issue once merged.
Updated by Mark Gargiulo almost 4 years ago
Chris,
thanks for the update and creating a fix.
The IPAM Dashboard is of course not a critical function and was more informative than anything in case it was related to the more pressing issue (for me). If I can provide any further information let me know.
Updated by Mark Gargiulo almost 4 years ago
Chris,
I downloaded the external_ipam.rb from your PR to test and the error changed but still produces, "Oops, we're sorry but something went wrong no implicit conversion of String into Integer", I have attached the stacktrace for you.
Updated by Chris Smith almost 4 years ago
I forgot that the above PR fixes the issue, but only with the latest foreman_ipam plugin. Try upgrading it to 0.1.0 and that should resolve the dashboard issue. For the other more important issue, I will look into it and report back here.
Updated by Mark Gargiulo almost 4 years ago
Chris,
I can confirm your PR does indeed work with foreman_ipam 0.1.0.
Thanks.
Updated by The Foreman Bot almost 4 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/foreman/pull/8244 added
Updated by Chris Smith almost 4 years ago
Mark - I have found the issue and opened a PR. I'm unable to properly test this as I do not have access to any VMWare resources, however I'm fairly certain that this is the cause. Just need to remove one line of code.
https://github.com/theforeman/foreman/pull/8244
Basically, foreman is raising an Exception if the mac address is nil, which it should not be doing.
Let me know if this resolves the issue. My apologies for any inconvenience here!
Updated by Chris Smith almost 4 years ago
- Status changed from Ready For Testing to Closed
Applied in changeset foreman|2cbce2b6b73c17fe56a5d755a811a1788e708ea9.
Updated by Mark Gargiulo almost 4 years ago
Chris,
it appears that smart_proxy_ipam plugin is now passing back an error that a mac address is needed.
ERF12-9889 [ProxyAPI::ProxyException]: Unable to retrieve the next available IP for subnet 10.60.25.0/24 External IPAM. ([RestClient::BadRequest]: 400 Bad Request) for proxy https://foreman-proxy:8443/ipam
This seems to be coming from smart_proxy_ipam-0.1.3/lib/smart_proxy_ipam/ipam_api.rb
I saw in the PR you mentioned that the proxy will use a unique ID if the mac is not present for the IP cache key.
I checked and there does not appear to be an updated version available if this fix also requires a newer plugin. Currently smart_proxy_ipam (0.1.3)
Do I need to wait for the release of Foreman 2.3.2?
Updated by Chris Smith almost 4 years ago
Hi Mark - My apologies. The plugin will need to be updated as well. I thought I changed this in the last release. I will fix this later today.
Updated by Chris Smith almost 4 years ago
I have pushed a fix to the plugin in version 0.1.4. Please test and let me know if this resolves your issue.
Updated by Mark Gargiulo almost 4 years ago
Chris,
I can verify that this does indeed fix the external ipam issue, many thanks.