Foreman » Smart Proxy

Thanks, Ohad!

Is this designed to be applied directly to dhcp_api.rb of 1.1 or I need to clone and install the latest off the the repo?

If the former, then I ran into a problem when tried to use this feature like this - http://dhcp:8443/dhcp/00:50:56:39:ac:40
Getting this error:

NoMethodError at /dhcp/00:50:56:39:ac:40
undefined method `accept?' for #<Sinatra::Request:0x7ffa35ece0f0>

file: dhcp_api.rb
    location: GET (?-mix:\/dhcp\/(([a-f0-9]{1,2}:){5}[a-f0-9]{1,2}))
    line: 61

I'm running on Ubuntu12.04 (in case I didn't mention before). I will try RH shortly as well.

Thanks again!

Unfortunately, I'm getting the same results on RH6 machine too...

With a couple of minor modifications to your suggested code, it works now with and without application/json header. Thank you.

get %r{\/dhcp\/(([a-f0-9]{1,2}:){5}[a-f0-9]{1,2})} do |mac, unused_match|
    begin
      log_halt 404, "No subnets found on this server" unless @subnets
      records = @subnets.map{|s| s.has_mac?(mac)}.compact.delete_if{|r| r == false }
      log_halt 404, mac + " not found" unless records[0]
    rescue => e
      log_halt 400, e
    end
    content_type :json
    records.to_json
  end

While I was browsing through the code though, I found that there's already a find_record method in proxy/dhcp/server.rb, which if I understand things correctly, could be used for any kind of searches, not just MACs. Couldn't it be used to expose a generic API like <proxy-host>:8443/dhcp/find/<hostname|IP|MAC> ?

Thanks!

which version of sinatra are you using? you might be hitting the problem in e21f675

regarding the method, yes, you are right, feel free to send a new PR instead :) (sorry, i hacked it in about 5 minutes).

I have sinatra 1.2.6 and "if request.accept? 'application/json'" does not work in it (gives me "undefined method `accept?' for #" error).
Since I'm planning on adding some additional BMC APIs, I already forked the repo and will migrate to sinatra 1.4.2 as well.

Thanks!

Konstantin Orekhov wrote:

I have sinatra 1.2.6 and "if request.accept? 'application/json'" does not work in it (gives me "undefined method `accept?' for #" error).

interesting.. didn't see that before, and i think we support sinatra 1.0+

Since I'm planning on adding some additional BMC APIs, I already forked the repo and will migrate to sinatra 1.4.2 as well.

1.42 has changed, so i was thinking you are using that (and that commit in develop branch address that) but it wasn't part of any official release just yet.

So, should I or should I not move to sinatra 1.4.2 to be used with Foreman smart-proxies?

Konstantin Orekhov wrote:

So, should I or should I not move to sinatra 1.4.2 to be used with Foreman smart-proxies?

it does not matter, however, if you do, you should use latest develop branch for it to work.

Hmm, still having some confusing issues. Here's what I did today.

- forked a development branch of smart-proxy repo and cloned it to my environment
- removed previously installed smart-proxy 1.1 and gems (sinatra 1.2.6, etc.) from my machines, the only gem left was rubyipmi 0.5.1
- installed sinatra 1.4.2 and dependencies off rubygems.org
- used forked repo above and used that to install latest smart-proxy on my machines (/version says 1.1 though, is that normal?)

Observations. Unless I revert the changes described here (e21f675) as per your note earlier, web pages do not get rendered at all and the only thing I get from proxy is JSON regardless if requested it or not. I had to go back to 'if request.accept.include?("application/json")' to fix that and now /dhcp and /features actually work in my browser.

Is that (JSON only) by design?

I actually "fixed" the above issue by moving back to sinatra 1.2.6 and now I have both JSON when I request that with a header and regular output when using browser to GET things.

New PR request has been submitted for DHCP global search by record (MAC, IP or hostname) - https://github.com/theforeman/smart-proxy/pull/87

However, while testing DHCP smart-proxy with dhcpd.conf|leases from real production, I ran into unacceptably poor performance.
In my tests on 24 vCPU bare-metal machine (Intel(R) Xeon(R) CPU X5675 @ 3.07GHz) with 72GB RAM and very small (just 5 subnets with 37 leases total), I get the following results:

$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp
<snip>
real 0m0.655s
user 0m0.011s
sys 0m0.004s

$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/&lt;network>
<snip>
real 0m1.878s
user 0m0.011s
sys 0m0.004s

$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/&lt;network&gt;/&lt;mac>
<snip>
real 0m1.849s
user 0m0.011s
sys 0m0.004s

$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/find/&lt;mac>
<snip>
real 0m3.491s
user 0m0.011s
sys 0m0.004s

The global search is understandably slower than a network search, but still even the latter takes 1.849s. I'm afraid that in real prod environment, this will become a huge issue. For example, I did a quick test on a 4 vCPU VM with 12GB RAM on dhcpd.conf|leases with 7649 subnets and who knows how many leases, just to get a listing of all subnets takes 1m45.582s.

What can be done here to improve the performance? IMHO, the majority of performance issues come from the fact that smart-proxy loads all subnet data on every request, creates a map of subnets and then does the search, for example with find_record method. To be honest, I don't quite understand what for as all the data is already available in dhcpd.leases so why to care which subnet it is in? Only things that are part of actual DHCP record matter (MAC, IP, hostname, options)... But I realize this is a logic re-thinking, which can take a significant time. I'm wondering if there's something else can be done in a short-term?

Thanks!