Project

General

Profile

Actions

Bug #6075

closed

apipie significantly slower than naive API calls

Added by Roger K about 10 years ago. Updated almost 10 years ago.

Status:
Closed
Priority:
Low
Category:
Foreman commands (obsolete)
Target version:
-
Difficulty:
Triaged:
Team Backlog:
Fixed in Releases:
Found in Releases:
In Kanboard:

Description

I'm writing some tooling around TheForeman at work and needed the ability to enumerate Hosts. originally, I was writing all the API calls myself using HTTPI, but I did not necessarily want to manage all the resource types, argument validation, results processing, etc. at the suggestion of Dominic, I tried out apipie-bindings. however, it seemed massively slower than writing naive HTTP calls using an HTTP library. initial testing indicated that apipie was 10x slower or worse.

thomasmckay suggested I produce some code samples and file a bug report here regarding this potential performance issue. the below repo contains a simple benchmark I developed. feel free to modify it and run your own tests or suggest improvements to the benchmarking methodology.

https://github.com/neoice/theforeman-apipie-bugreport

Actions #1

Updated by Thomas McKay about 10 years ago

  • Bugzilla link set to https://bugzilla.redhat.com/show_bug.cgi?id=1105191
Actions #2

Updated by Martin Bacovsky about 10 years ago

  • Category set to Foreman commands (obsolete)
  • Status changed from New to Assigned
  • Assignee set to Martin Bacovsky

Thanks for detailed report and the benchmarks, it is realy appreciated!

The difference is caused by apipie-bindings, as all the three client implementations call the same (apipie documented) API on the server.
There is a couple of extra things that the binding does, e.g. apidoc cache checking, correct route evaluation based on parameters, parsing of the received JSON.

I'll check if the performance of apipie-bindings could be improved or fixed be more suitable configuration.

Actions #3

Updated by Roger K about 10 years ago

I expected a performance hit from apipie-bindings due to all the "magic" it performs. I was surprised to see that it was such a large magnitude of a performance hit. I was willing to make this trade for improved development speed. however, if simple test cases take 2+ minutes to run, it's probably faster for me to develop native HTTP calls instead of paying for the break in flow every time I want to test a piece of functionality :/

Actions #4

Updated by Martin Bacovsky about 10 years ago

On my foreman (1.5) instance I run in local VM the benchmark results for the bindings seems okay. I had to remove the kerberos support form the benchamrk since my instance is not setup to support it, but I assume it does not create the difference.

$ bundle exec ./benchmark/apipie.rb https://foreman.my.lan
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000249)
   0.020000   0.010000   0.030000 (  0.118600)
   0.030000   0.000000   0.030000 (  0.325904)
Hosts collected: 3

$ bundle exec ./benchmark/httpi-net_http.rb https://foreman.my.lan
D, [2014-06-11T11:50:25.627755 #10509] DEBUG -- : Net::NTLM is not available. Install via gem install rubyntlm.
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000003)
   0.000000   0.000000   0.000000 (  0.051564)
   0.020000   0.000000   0.020000 (  0.160803)
Hosts collected: 3

$ bundle exec ./benchmark/httpi-curb.rb https://foreman.my.lan                                                                                         
D, [2014-06-11T11:50:38.215713 #10573] DEBUG -- : Net::NTLM is not available. Install via gem install rubyntlm.
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000004)
   0.030000   0.050000   0.080000 (  0.117204)
   0.020000   0.000000   0.020000 (  0.209949)
Hosts collected: 3

Please check if your apipie cache is turned on (even in development mode). In config/initializers/apipie.rb there shoud be line

  config.use_cache = Rails.env.production? || File.directory?(config.cache_dir)

(I'm not sure if it was there before 1.5). Then run

$ foreman-rake apipi:cache

or while in development

$ bundle exec rake apipie:cache

This should create the apipie cache instead of re-computing it at runtime with every request which I believe is going on in your case.

If that didn't help, you could add logger to the bindings to see what is the bottle-neck and we can go on with investigation

    log = Logger.new(STDERR)
    log.level=Logger::DEBUG
    @api = ApipieBindings::API.new({:uri => base_url, :api_version => 2, :logger => log})
Actions #5

Updated by Roger K about 10 years ago

I was not even able to use apipie-bindings until after running foreman-rake apipie:cache

(today I discovered that my work zshrc does not have extendedhistory turned on)

root@theforeman:~# grep foreman-rake .histfile 
foreman-rake apipie:cache
root@theforeman:/usr/share/foreman/doc# ls -l
total 492
drwxr-xr-x.  3 foreman foreman   4096 Jun  2 19:48 ./
drwxr-xr-x. 13 root    root      4096 Jun  2 19:45 ../
drwxr-xr-x. 35 foreman foreman   4096 Jun  2 19:48 apidoc/
-rw-rw-r--.  1 foreman foreman 278580 Jun  2 19:48 apidoc-onepage.html
-rw-rw-r--.  1 foreman foreman 160137 Jun  2 19:48 apidoc-plain.html
-rw-rw-r--.  1 foreman foreman  35026 Jun  2 19:48 apidoc.html

I have re-run foreman-rake apipie:cache and repeated my tests with no change in outcome.

I added log output and it showed me that my Client was implemented extremely poorly, leading to excessive requests per host lookup. the commit for the fix is here: https://github.com/neoice/theforeman-apipie-bugreport/commit/5ab3294473b8f96af2db79185d599d77acce2501

in the plain HTTP version, I was getting a list of host names and then fetching details for each host.

in the apipie version, I was getting a list of host names and then searching for the host by walking the output of hosts/index. it was obvious when I noticed get_all_hosts and find_host both used the same .action() method.

I think the characteristic of this bug results in exponentially worse performance as node count increases (although it's a bit too early in the morning for me to formally examine this hypothesis). I ran my benchmark/apipie.rb script against a different Foreman instance: 15% increase in node count resulted in 90% increase in runtime.

my implementation change has caused run times to be roughly equivalent with plain HTTP API implementations.

sorry for causing such a commotion!

Actions #6

Updated by Martin Bacovsky almost 10 years ago

  • Status changed from Assigned to Closed

Thanks for your effort and explanation. I'll close this bug then.

Actions

Also available in: Atom PDF