Feature #2482
openGlobal search in DHCP reservations/leases using MAC and/or hostname
Description
This feature request is in partly a duplicate of #1508 submitted a year ago, but has additional request in it.
When only a MAC address of the machine is known, in current implementation one would need to perform a search of that MAC in each of the reported subnets, which is very time consuming and a multistep process instead of a single API call like this: http://dhcp.vip:8443/dhcp/*/<MAC>
The same goes to a hostname - although a lower priority than a MAC search, it is still a useful feature.
Konstantin.
Updated by Ohad Levy over 11 years ago
- Status changed from New to Ready For Testing
Updated by Konstantin Orekhov over 11 years ago
Thanks, Ohad!
Is this designed to be applied directly to dhcp_api.rb of 1.1 or I need to clone and install the latest off the the repo?
If the former, then I ran into a problem when tried to use this feature like this - http://dhcp:8443/dhcp/00:50:56:39:ac:40
Getting this error:
NoMethodError at /dhcp/00:50:56:39:ac:40
undefined method `accept?' for #<Sinatra::Request:0x7ffa35ece0f0>
file: dhcp_api.rb
location: GET (?-mix:\/dhcp\/(([a-f0-9]{1,2}:){5}[a-f0-9]{1,2}))
line: 61
I'm running on Ubuntu12.04 (in case I didn't mention before). I will try RH shortly as well.
Thanks again!
Updated by Konstantin Orekhov over 11 years ago
Unfortunately, I'm getting the same results on RH6 machine too...
Updated by Konstantin Orekhov over 11 years ago
With a couple of minor modifications to your suggested code, it works now with and without application/json header. Thank you.
get %r{\/dhcp\/(([a-f0-9]{1,2}:){5}[a-f0-9]{1,2})} do |mac, unused_match|
begin
log_halt 404, "No subnets found on this server" unless @subnets
records = @subnets.map{|s| s.has_mac?(mac)}.compact.delete_if{|r| r == false }
log_halt 404, mac + " not found" unless records[0]
rescue => e
log_halt 400, e
end
content_type :json
records.to_json
end
While I was browsing through the code though, I found that there's already a find_record method in proxy/dhcp/server.rb, which if I understand things correctly, could be used for any kind of searches, not just MACs. Couldn't it be used to expose a generic API like <proxy-host>:8443/dhcp/find/<hostname|IP|MAC> ?
Thanks!
Updated by Ohad Levy about 11 years ago
which version of sinatra are you using? you might be hitting the problem in e21f675
regarding the method, yes, you are right, feel free to send a new PR instead :) (sorry, i hacked it in about 5 minutes).
Updated by Konstantin Orekhov about 11 years ago
I have sinatra 1.2.6 and "if request.accept? 'application/json'" does not work in it (gives me "undefined method `accept?' for #" error).
Since I'm planning on adding some additional BMC APIs, I already forked the repo and will migrate to sinatra 1.4.2 as well.
Thanks!
Updated by Ohad Levy about 11 years ago
Konstantin Orekhov wrote:
I have sinatra 1.2.6 and "if request.accept? 'application/json'" does not work in it (gives me "undefined method `accept?' for #" error).
interesting.. didn't see that before, and i think we support sinatra 1.0+
Since I'm planning on adding some additional BMC APIs, I already forked the repo and will migrate to sinatra 1.4.2 as well.
1.42 has changed, so i was thinking you are using that (and that commit in develop branch address that) but it wasn't part of any official release just yet.
Updated by Konstantin Orekhov about 11 years ago
So, should I or should I not move to sinatra 1.4.2 to be used with Foreman smart-proxies?
Updated by Ohad Levy about 11 years ago
Konstantin Orekhov wrote:
So, should I or should I not move to sinatra 1.4.2 to be used with Foreman smart-proxies?
it does not matter, however, if you do, you should use latest develop branch for it to work.
Updated by Konstantin Orekhov about 11 years ago
Hmm, still having some confusing issues. Here's what I did today.
- forked a development branch of smart-proxy repo and cloned it to my environment
- removed previously installed smart-proxy 1.1 and gems (sinatra 1.2.6, etc.) from my machines, the only gem left was rubyipmi 0.5.1
- installed sinatra 1.4.2 and dependencies off rubygems.org
- used forked repo above and used that to install latest smart-proxy on my machines (/version says 1.1 though, is that normal?)
Observations. Unless I revert the changes described here (e21f675) as per your note earlier, web pages do not get rendered at all and the only thing I get from proxy is JSON regardless if requested it or not. I had to go back to 'if request.accept.include?("application/json")' to fix that and now /dhcp and /features actually work in my browser.
Is that (JSON only) by design?
Updated by Konstantin Orekhov about 11 years ago
I actually "fixed" the above issue by moving back to sinatra 1.2.6 and now I have both JSON when I request that with a header and regular output when using browser to GET things.
Updated by Konstantin Orekhov about 11 years ago
New PR request has been submitted for DHCP global search by record (MAC, IP or hostname) - https://github.com/theforeman/smart-proxy/pull/87
However, while testing DHCP smart-proxy with dhcpd.conf|leases from real production, I ran into unacceptably poor performance.
In my tests on 24 vCPU bare-metal machine (Intel(R) Xeon(R) CPU X5675 @ 3.07GHz) with 72GB RAM and very small (just 5 subnets with 37 leases total), I get the following results:
$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp
<snip>
real 0m0.655s
user 0m0.011s
sys 0m0.004s
$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/<network>
<snip>
real 0m1.878s
user 0m0.011s
sys 0m0.004s
$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/<network>/<mac>
<snip>
real 0m1.849s
user 0m0.011s
sys 0m0.004s
$ time curl -H "Accept:application/json" -k -X GET https://dhcp:8443/dhcp/find/<mac>
<snip>
real 0m3.491s
user 0m0.011s
sys 0m0.004s
The global search is understandably slower than a network search, but still even the latter takes 1.849s. I'm afraid that in real prod environment, this will become a huge issue. For example, I did a quick test on a 4 vCPU VM with 12GB RAM on dhcpd.conf|leases with 7649 subnets and who knows how many leases, just to get a listing of all subnets takes 1m45.582s.
What can be done here to improve the performance? IMHO, the majority of performance issues come from the fact that smart-proxy loads all subnet data on every request, creates a map of subnets and then does the search, for example with find_record method. To be honest, I don't quite understand what for as all the data is already available in dhcpd.leases so why to care which subnet it is in? Only things that are part of actual DHCP record matter (MAC, IP, hostname, options)... But I realize this is a logic re-thinking, which can take a significant time. I'm wondering if there's something else can be done in a short-term?
Thanks!
Updated by Ohad Levy almost 11 years ago
- Is duplicate of Feature #1508: Run a wide search on DHCP for IP address added
Updated by Ohad Levy almost 11 years ago
- Status changed from Ready For Testing to Duplicate
Updated by Ohad Levy almost 11 years ago
- Description updated (diff)
- Status changed from Duplicate to Assigned
Updated by Dominic Cleal about 8 years ago
- Status changed from Assigned to New
https://github.com/theforeman/smart-proxy/pull/97 was opened, but is now closed due to inactivity.
Updated by The Foreman Bot over 7 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/smart-proxy/pull/478 added
Updated by Anonymous over 7 years ago
- Blocked by Refactor #17499: Leases by mac and hosts by mac mappings in subnet service do not need to be grouped by subnet added
Updated by Dominic Cleal over 7 years ago
- Status changed from Ready For Testing to New
- Pull request deleted (
https://github.com/theforeman/smart-proxy/pull/478)
PR closed.