Hammer CLI

I expected a performance hit from apipie-bindings due to all the "magic" it performs. I was surprised to see that it was such a large magnitude of a performance hit. I was willing to make this trade for improved development speed. however, if simple test cases take 2+ minutes to run, it's probably faster for me to develop native HTTP calls instead of paying for the break in flow every time I want to test a piece of functionality :/

On my foreman (1.5) instance I run in local VM the benchmark results for the bindings seems okay. I had to remove the kerberos support form the benchamrk since my instance is not setup to support it, but I assume it does not create the difference.

$ bundle exec ./benchmark/apipie.rb https://foreman.my.lan
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000249)
   0.020000   0.010000   0.030000 (  0.118600)
   0.030000   0.000000   0.030000 (  0.325904)
Hosts collected: 3

$ bundle exec ./benchmark/httpi-net_http.rb https://foreman.my.lan
D, [2014-06-11T11:50:25.627755 #10509] DEBUG -- : Net::NTLM is not available. Install via gem install rubyntlm.
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000003)
   0.000000   0.000000   0.000000 (  0.051564)
   0.020000   0.000000   0.020000 (  0.160803)
Hosts collected: 3

$ bundle exec ./benchmark/httpi-curb.rb https://foreman.my.lan                                                                                         
D, [2014-06-11T11:50:38.215713 #10573] DEBUG -- : Net::NTLM is not available. Install via gem install rubyntlm.
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000004)
   0.030000   0.050000   0.080000 (  0.117204)
   0.020000   0.000000   0.020000 (  0.209949)
Hosts collected: 3

Please check if your apipie cache is turned on (even in development mode). In config/initializers/apipie.rb there shoud be line

  config.use_cache = Rails.env.production? || File.directory?(config.cache_dir)

(I'm not sure if it was there before 1.5). Then run

$ foreman-rake apipi:cache

or while in development

$ bundle exec rake apipie:cache

This should create the apipie cache instead of re-computing it at runtime with every request which I believe is going on in your case.

If that didn't help, you could add logger to the bindings to see what is the bottle-neck and we can go on with investigation

    log = Logger.new(STDERR)
    log.level=Logger::DEBUG
    @api = ApipieBindings::API.new({:uri => base_url, :api_version => 2, :logger => log})

I was not even able to use apipie-bindings until after running foreman-rake apipie:cache

(today I discovered that my work zshrc does not have extendedhistory turned on)

root@theforeman:~# grep foreman-rake .histfile 
foreman-rake apipie:cache

root@theforeman:/usr/share/foreman/doc# ls -l
total 492
drwxr-xr-x.  3 foreman foreman   4096 Jun  2 19:48 ./
drwxr-xr-x. 13 root    root      4096 Jun  2 19:45 ../
drwxr-xr-x. 35 foreman foreman   4096 Jun  2 19:48 apidoc/
-rw-rw-r--.  1 foreman foreman 278580 Jun  2 19:48 apidoc-onepage.html
-rw-rw-r--.  1 foreman foreman 160137 Jun  2 19:48 apidoc-plain.html
-rw-rw-r--.  1 foreman foreman  35026 Jun  2 19:48 apidoc.html

I have re-run foreman-rake apipie:cache and repeated my tests with no change in outcome.

I added log output and it showed me that my Client was implemented extremely poorly, leading to excessive requests per host lookup. the commit for the fix is here: https://github.com/neoice/theforeman-apipie-bugreport/commit/5ab3294473b8f96af2db79185d599d77acce2501

in the plain HTTP version, I was getting a list of host names and then fetching details for each host.

in the apipie version, I was getting a list of host names and then searching for the host by walking the output of hosts/index. it was obvious when I noticed get_all_hosts and find_host both used the same .action() method.

I think the characteristic of this bug results in exponentially worse performance as node count increases (although it's a bit too early in the morning for me to formally examine this hypothesis). I ran my benchmark/apipie.rb script against a different Foreman instance: 15% increase in node count resulted in 90% increase in runtime.

my implementation change has caused run times to be roughly equivalent with plain HTTP API implementations.

sorry for causing such a commotion!