Project

General

Profile

Bug #31578

External IPAM now fails with "mac address cannot be nil" on Compute Resource deployment.

Added by Mark Gargiulo about 2 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Plugin integration
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

I had successfully integrated phpipam with the new external ipam plugin however, since I upgraded to Foreman 2.3.1 I have noticed that "create host" now shows an error under the interfaces tab even though I am using a vmware compute resource and the mac dialog box is greyed out...

"ERF12-9889 [ProxyAPI::ProxyException]: Unable to retrieve the next available IP for subnet 1.2.3.0/24 External IPAM. ([RuntimeError]: mac address cannot be nil) for proxy https://foreman-proxy:8443/ipam" (values have been changed obviously)

If I change "Deploy On" to "Bare Metal" and and add a fake mac address I get a next_ip, then I can change it back to vmware and I can provision from a template again.
This is not ideal for other people less familiar with Foreman than me.

Not sure if it is related or not, but the IPAM Dashboard has also stopped working, now showing an error...

"ERF12-5974 [ProxyAPI::ProxyException]: Unable to obtain groups from External IPAM. ([TypeError]: no implicit conversion of String into Integer) for proxy https://foreman-proxy:8443/ipam"

I was able to see a list of subnets before.
No changes to the phpipam version or api user have been made since this was working, in fact I actually see a 200 in the phpipam apache log for the Dashboard request.

1.2.3.4 - $USER [05/Jan/2021:13:47:27 0800] "POST /phpipam/api/$USER/user/ HTTP/1.1" 200 116 "" "Ruby"
1.2.3.4 - - [05/Jan/2021:13:47:27 0800] "GET /phpipam/api/$USER/sections/ HTTP/1.1" 200 580 "" "Ruby"

Versions
foreman-2.3.1
foreman-proxy-2.3.1
smart_proxy_ipam (0.1.3)

Please let me know if you require any additional information.

production_log_stacktrace.txt production_log_stacktrace.txt 16.1 KB Snippet from production.log Mark Gargiulo, 01/08/2021 09:11 PM

Associated revisions

Revision 2cbce2b6 (diff)
Added by Chris Smith about 2 months ago

Fixes #31578 - External IPAM now fails with 'mac address cannot be nil' on Compute Resource deployment

History

#1 Updated by Mark Gargiulo about 2 months ago

  • Category set to Plugin integration

#2 Updated by Lukas Zapletal about 2 months ago

Hello, can you share the stacktrace? It's probably just some nil pointer exception if you say it was working previously. When using compute resources, MAC is blank, therefore it cannot be passed into IPAM.

#3 Updated by Mark Gargiulo about 2 months ago

Lukas,
unfortunately even in debug logging mode the log shows nothing more than the actual displayed messages I already listed (surrounded by a bunch of cache statements), there is no ruby stacktrace per se as this seems like an intentional validation error. If you think there is some hidden info I can provide them but they will be lengthy.

#4 Updated by Mark Gargiulo about 2 months ago

Additional info if it helps narrow things down, is that when this was working previously I was using smart_proxy_ipam 0.0.22 and foreman 2.2.1.

#5 Updated by Lukas Zapletal about 2 months ago

This looks like an error raised on the smart proxy, check proxy.log.

#6 Updated by Mark Gargiulo about 2 months ago

Lukas,
I already looked at the proxy.log in debug mode and there were no entries for the in initial "mac address cannot be nil" when first selecting "Create Host" I believe that comes from /usr/share/foreman/lib/proxy_api/external_ipam.rb which is part of the main foreman code.

rpm -qf /usr/share/foreman/lib/proxy_api/external_ipam.rb
foreman-2.3.1-1.el7.noarch

I verified that there was no communication with the proxy at this time by running a tcpdump on port 8443. I see traffic when I select "Bare Metal" and add a fake mac address but nothing when the vmware compute resource is selected and I click on the Interfaces tab.

#7 Updated by Chris Smith about 2 months ago

I will have a look into this. For the IPAM Dashboard issue, the below PR will fix this issue once merged.

https://github.com/theforeman/foreman/pull/8193

#8 Updated by Mark Gargiulo about 2 months ago

Chris,
thanks for the update and creating a fix.
The IPAM Dashboard is of course not a critical function and was more informative than anything in case it was related to the more pressing issue (for me). If I can provide any further information let me know.

#9 Updated by Mark Gargiulo about 2 months ago

Chris,
I downloaded the external_ipam.rb from your PR to test and the error changed but still produces, "Oops, we're sorry but something went wrong no implicit conversion of String into Integer", I have attached the stacktrace for you.

#10 Updated by Chris Smith about 2 months ago

I forgot that the above PR fixes the issue, but only with the latest foreman_ipam plugin. Try upgrading it to 0.1.0 and that should resolve the dashboard issue. For the other more important issue, I will look into it and report back here.

#11 Updated by Mark Gargiulo about 2 months ago

Chris,
I can confirm your PR does indeed work with foreman_ipam 0.1.0.
Thanks.

#12 Updated by The Foreman Bot about 2 months ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/8244 added

#13 Updated by Chris Smith about 2 months ago

Mark - I have found the issue and opened a PR. I'm unable to properly test this as I do not have access to any VMWare resources, however I'm fairly certain that this is the cause. Just need to remove one line of code.

https://github.com/theforeman/foreman/pull/8244

Basically, foreman is raising an Exception if the mac address is nil, which it should not be doing.

Let me know if this resolves the issue. My apologies for any inconvenience here!

#14 Updated by The Foreman Bot about 2 months ago

  • Fixed in Releases 2.4.0 added

#15 Updated by Chris Smith about 2 months ago

  • Status changed from Ready For Testing to Closed

#16 Updated by Tomer Brisker about 2 months ago

  • Fixed in Releases 2.3.2 added

#17 Updated by Mark Gargiulo about 2 months ago

Chris,
it appears that smart_proxy_ipam plugin is now passing back an error that a mac address is needed.

ERF12-9889 [ProxyAPI::ProxyException]: Unable to retrieve the next available IP for subnet 10.60.25.0/24 External IPAM. ([RestClient::BadRequest]: 400 Bad Request) for proxy https://foreman-proxy:8443/ipam

This seems to be coming from smart_proxy_ipam-0.1.3/lib/smart_proxy_ipam/ipam_api.rb

I saw in the PR you mentioned that the proxy will use a unique ID if the mac is not present for the IP cache key.

I checked and there does not appear to be an updated version available if this fix also requires a newer plugin. Currently smart_proxy_ipam (0.1.3)

Do I need to wait for the release of Foreman 2.3.2?

#18 Updated by Chris Smith about 2 months ago

Hi Mark - My apologies. The plugin will need to be updated as well. I thought I changed this in the last release. I will fix this later today.

#19 Updated by Chris Smith about 2 months ago

I have pushed a fix to the plugin in version 0.1.4. Please test and let me know if this resolves your issue.

#20 Updated by Mark Gargiulo about 2 months ago

Chris,
I can verify that this does indeed fix the external ipam issue, many thanks.

#21 Updated by Chris Smith about 2 months ago

Glad to hear it is resolved!

Also available in: Atom PDF