Project

General

Profile

Bug #19742

API call for fact_values is slow when user is non-admin , Foreman 1.15.0

Added by Karen Kalinux almost 6 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Marek Hulán
Category:
API
Target version:
1.15.4
Difficulty:
Triaged:
Bugzilla link:
Pull request:
https://github.com/theforeman/foreman/pull/4818
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Hi Gents,

Seems the same user with admin rights can have a respond from API call within 0.1s and with limited access about 10.5s,

Logs from Foreman below:

limited user
2017-06-01 15:39:42 5cb09216 [app] [I] Started GET "/api/hosts/345/facts?per_page=999" for myhost.domain.com at 2017-06-01 15:39:42 +0200
2017-06-01 15:39:42 5cb09216 [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 15:39:42 5cb09216 [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"345", "fact_value"=>{}}
2017-06-01 15:39:42 5cb09216 [app] [I] Authorized user karen(karen)
2017-06-01 15:39:53 5cb09216 [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.6ms)
2017-06-01 15:39:53 5cb09216 [app] [I] Completed 200 OK in 10555ms (Views: 54.3ms | ActiveRecord: 190.0ms)

as admin
2017-06-01 15:40:46 3588293e [app] [I] Started GET "/api/hosts/345/facts?per_page=999" for myhost.domain.com at 2017-06-01 15:40:46 +0200
2017-06-01 15:40:46 3588293e [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 15:40:46 3588293e [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"345", "fact_value"=>{}}
2017-06-01 15:40:46 3588293e [app] [I] Authorized user karen(karen)
2017-06-01 15:40:46 3588293e [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.6ms)
2017-06-01 15:40:46 3588293e [app] [I] Completed 200 OK in 776ms (Views: 11.5ms | ActiveRecord: 18.3ms)

as user without Organization or Location assigned
2017-06-01 17:41:11 bcf71b47 [app] [I] Started GET "/api/hosts/645/facts?per_page=999" for myhost.domain.com at 2017-06-01 17:41:11 +0200
2017-06-01 17:41:11 bcf71b47 [app] [I] Processing by Api::V2::FactValuesController#index as JSON
2017-06-01 17:41:11 bcf71b47 [app] [I] Parameters: {"per_page"=>"999", "apiv"=>"v2", "host_id"=>"645", "fact_value"=>{}}
2017-06-01 17:41:11 bcf71b47 [app] [I] Authorized user karen(karen)
2017-06-01 17:41:15 bcf71b47 [app] [I] Rendered api/v2/fact_values/index.json.rabl within api/v2/layouts/index_layout (1.3ms)
2017-06-01 17:41:15 bcf71b47 [app] [I] Completed 200 OK in 3569ms (Views: 19.0ms | ActiveRecord: 68.9ms)

Related issues

Related to Foreman - Bug #11996: fact_values API with non-admin user and search on host.hostgroup throws SQL errorClosed2015-09-29

Associated revisions

Revision e12885c7 (diff)
Added by Marek Hulán over 5 years ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

Revision 44072648 (diff)
Added by Marek Hulán over 5 years ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

Revision 82ab8c84 (diff)
Added by Marek Hulán over 5 years ago

Fixes #19742 - cache hosts for fact values

When we generate fact values hash we always loaded a host for a given
fact. This in combination with taxonomies queryies that are
automatically generated caused roughly 6N+1 issue. We can preload all
hosts that are required and use this in-memory cache for generating the
hash.

History

#1 Updated by Daniel Lobato Garcia almost 6 years ago

  • Legacy Backlogs Release (now unused) changed from 248 to 266

#2 Updated by Karen Kalinux almost 6 years ago

Any updates?

#3 Updated by Matteo Castellarin almost 6 years ago

Hi
This is somehow blocking, as REST retrieving data for a pool of VMs (i.e. their facts) really takes hours for just some hundred of VMs, while it can be performed in a matter of seconds with admin rights.

#4 Updated by Daniel Lobato Garcia over 5 years ago

  • Legacy Backlogs Release (now unused) changed from 266 to 276

#5 Updated by Daniel Lobato Garcia over 5 years ago

  • Legacy Backlogs Release (now unused) changed from 276 to 287

#6 Updated by Marek Hulán over 5 years ago

  • Related to Bug #11996: fact_values API with non-admin user and search on host.hostgroup throws SQL error added

#7 Updated by Marek Hulán over 5 years ago

  • Status changed from New to Assigned
  • Assignee changed from Daniel Lobato Garcia to Marek Hulán

#8 Updated by The Foreman Bot over 5 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/4818 added

#9 Updated by Marek Hulán over 5 years ago

Steps to reproduce

  1. create a user in some org and loc, have a host with facts in the same org and loc
  2. compare SQL log from hammer host facts --id $host_id --per-page 100 ran under admin and non-admin user

For non-admin user it takes much longer since for every fact we load host and verify its taxonomies. For admin we don't check taxonomies, admin users can access all so we skip it. Therefore there's no n+1 issue for admins.

The same issue is present for the endpoint without specific host, could be reproduced with hammer fact list --per-page 1000, with the patch I was able to list 1000 facts for set on 10 hosts without any issue.

#10 Updated by Anonymous over 5 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF