Bug #19742

API call for fact_values is slow when user is non-admin , Foreman 1.15.0

Added by Karen Kalinux 6 months ago. Updated 3 months ago.

Status:Closed
Priority:Normal
Assigned To:Marek Hulán
Category:API
Target version:-
Difficulty: Bugzilla link:
Found in release: Pull request:https://github.com/theforeman/foreman/pull/4818
Story points-
Velocity based estimate-
Release1.15.4Release relationshipAuto

Description

Hi Gents,

Seems the same user with admin rights can have a respond from API call within 0.1s and with limited access about 10.5s,

Logs from Foreman below:

limited user
2017-06-01 15:39:42 5cb09216 [app] [I] Started GET "/api/hosts/345/facts?per_page=999" for myhost.domain.com at 2017-06-01 15:39:42 +0200
2017-06-01 15:39:42 5cb09216 [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 15:39:42 5cb09216 [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"345", "fact_value"=>{}}
2017-06-01 15:39:42 5cb09216 [app] [I] Authorized user karen(karen)
2017-06-01 15:39:53 5cb09216 [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.6ms)
2017-06-01 15:39:53 5cb09216 [app] [I] Completed 200 OK in 10555ms (Views: 54.3ms | ActiveRecord: 190.0ms)

as admin
2017-06-01 15:40:46 3588293e [app] [I] Started GET "/api/hosts/345/facts?per_page=999" for myhost.domain.com at 2017-06-01 15:40:46 +0200
2017-06-01 15:40:46 3588293e [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 15:40:46 3588293e [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"345", "fact_value"=>{}}
2017-06-01 15:40:46 3588293e [app] [I] Authorized user karen(karen)
2017-06-01 15:40:46 3588293e [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.6ms)
2017-06-01 15:40:46 3588293e [app] [I] Completed 200 OK in 776ms (Views: 11.5ms | ActiveRecord: 18.3ms)

as user without Organization or Location assigned
2017-06-01 17:41:11 bcf71b47 [app] [I] Started GET "/api/hosts/645/facts?per_page=999" for myhost.domain.com at 2017-06-01 17:41:11 +0200
2017-06-01 17:41:11 bcf71b47 [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 17:41:11 bcf71b47 [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"645", "fact_value"=>{}}
2017-06-01 17:41:11 bcf71b47 [app] [I] Authorized user karen(karen)
2017-06-01 17:41:15 bcf71b47 [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.3ms)
2017-06-01 17:41:15 bcf71b47 [app] [I] Completed 200 OK in 3569ms (Views: 19.0ms | ActiveRecord: 68.9ms)


Related issues

Related to Foreman - Bug #11996: fact_values API with non-admin user and search on host.ho... Closed 09/29/2015

Associated revisions

Revision e12885c7
Added by Marek Hulán 3 months ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

Revision 44072648
Added by Marek Hulán 3 months ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

Revision 82ab8c84
Added by Marek Hulán 3 months ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

History

#1 Updated by Daniel Lobato Garcia 6 months ago

  • Release changed from 1.15.1 to 1.15.2

#2 Updated by Karen Kalinux 5 months ago

Any updates?

#3 Updated by Matteo Castellarin 5 months ago

Hi
This is somehow blocking, as REST retrieving data for a pool of VMs (i.e. their facts) really takes hours for just some hundred of VMs, while it can be performed in a matter of seconds with admin rights.

#4 Updated by Daniel Lobato Garcia 5 months ago

  • Release changed from 1.15.2 to 1.15.3

#5 Updated by Daniel Lobato Garcia 4 months ago

  • Release changed from 1.15.3 to 1.15.4

#6 Updated by Marek Hulán 3 months ago

  • Related to Bug #11996: fact_values API with non-admin user and search on host.hostgroup throws SQL error added

#7 Updated by Marek Hulán 3 months ago

  • Status changed from New to Assigned
  • Assigned To changed from Daniel Lobato Garcia to Marek Hulán

#8 Updated by The Foreman Bot 3 months ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/4818 added

#9 Updated by Marek Hulán 3 months ago

Steps to reproduce

  1. create a user in some org and loc, have a host with facts in the same org and loc
  2. compare SQL log from hammer host facts --id $host_id --per-page 100 ran under admin and non-admin user

For non-admin user it takes much longer since for every fact we load host and verify its taxonomies. For admin we don't check taxonomies, admin users can access all so we skip it. Therefore there's no n+1 issue for admins.

The same issue is present for the endpoint without specific host, could be reproduced with hammer fact list --per-page 1000, with the patch I was able to list 1000 facts for set on 10 hosts without any issue.

#10 Updated by Anonymous 3 months ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF