Project

General

Profile

Bug #18995

Puppet class import removes existing environments when orgs/locations are disabled

Added by Dominic Cleal over 4 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Category:
Puppet integration
Target version:
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

In a default installation where organisations and locations are disabled (and taxable_taxonomies is unpopulated), then a Puppet class import will attempt to remove environments that are actually present, then fail class import silently:

2017-03-23 05:05:59 3a426e5e [app] [I] Started POST "/api/smart_proxies/1/import_puppetclasses" for 127.0.0.1 at 2017-03-23 05:05:59 -0400
2017-03-23 05:06:00 3a426e5e [app] [I] Processing by Api::V2::SmartProxiesController#import_puppetclasses as JSON
2017-03-23 05:06:00 3a426e5e [app] [I]   Parameters: {"dryrun"=>false, "apiv"=>"v2", "id"=>"1", "smart_proxy"=>{}}
2017-03-23 05:06:00 3a426e5e [sql] [D]   ActiveRecord::SchemaMigration Load (0.5ms)  SELECT "schema_migrations".* FROM "schema_migrations" 
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (0.5ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "authorize_login_delegation"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   User Load (0.4ms)  SELECT  "users".* FROM "users" WHERE "users"."lower_login" = $1 LIMIT 1  [["lower_login", "admin"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   AuthSource Load (0.4ms)  SELECT  "auth_sources".* FROM "auth_sources" WHERE "auth_sources"."id" = $1 LIMIT 1  [["id", 1]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT  "users".* FROM "users" WHERE "users"."lower_login" = $1 LIMIT 1  [["lower_login", "admin"]]
2017-03-23 05:06:00 3a426e5e [sql] [D] Authenticated user admin against INTERNAL authentication source
2017-03-23 05:06:00 3a426e5e [sql] [D]   User Load (10.9ms)  SELECT  "users".* FROM "users" WHERE "users"."lower_login" = $1 LIMIT 1  [["lower_login", "foreman_admin"]]
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to foreman_admin
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to nil
2017-03-23 05:06:00 3a426e5e [sql] [D] Post-login processing for admin
2017-03-23 05:06:00 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT  "users".* FROM "users" WHERE "users"."lower_login" = $1 LIMIT 1  [["lower_login", "foreman_admin"]]
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to foreman_admin
2017-03-23 05:06:00 3a426e5e [sql] [D]   SQL (21.7ms)  UPDATE "users" SET "last_login_on" = '2017-03-23 09:06:00.587729' WHERE "users"."id" = $1  [["id", 3]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Role Load (0.2ms)  SELECT  "roles".* FROM "roles" WHERE "roles"."builtin" = $1 LIMIT 1  [["builtin", 2]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Role Exists (0.4ms)  SELECT  1 AS one FROM "roles" INNER JOIN "user_roles" ON "roles"."id" = "user_roles"."role_id" WHERE "user_roles"."owner_id" = $1 AND "user_roles"."owner_type" = $2 AND "roles"."id" = $3 LIMIT 1  [["owner_id", 3], ["owner_type", "User"], ["id", 7]]
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to nil
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to admin
2017-03-23 05:06:00 3a426e5e [sql] [D]    (0.3ms)  SELECT auth_sources.id FROM "auth_sources" WHERE "auth_sources"."type" IN ('AuthSourceHidden')
2017-03-23 05:06:00 3a426e5e [sql] [D]   User Load (0.2ms)  SELECT  "users".* FROM "users" WHERE ("users"."auth_source_id" NOT IN (2)) AND "users"."lower_login" = $1 LIMIT 1  [["lower_login", "admin"]]
2017-03-23 05:06:00 3a426e5e [app] [I] Authorized user admin(Admin User)
2017-03-23 05:06:00 3a426e5e [app] [D] Setting current user thread-local variable to admin
2017-03-23 05:06:00 3a426e5e [sql] [D]    (0.2ms)  BEGIN
2017-03-23 05:06:00 3a426e5e [sql] [D]    (0.1ms)  COMMIT
2017-03-23 05:06:00 3a426e5e [sql] [D]   ActiveRecord::SessionStore::Session Load (0.4ms)  SELECT  "sessions".* FROM "sessions" WHERE "sessions"."session_id" = $1  ORDER BY "sessions"."id" ASC LIMIT 1  [["session_id", "691cad1dae9acf9df1545818c9955522"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (0.2ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "idle_timeout"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   SmartProxy Load (0.3ms)  SELECT  "smart_proxies".* FROM "smart_proxies" WHERE "smart_proxies"."id" = $1  ORDER BY smart_proxies.name LIMIT 1  [["id", 1]]
2017-03-23 05:06:00 3a426e5e [sql] [D]    (0.8ms)  SELECT smart_proxies.id FROM "smart_proxies" INNER JOIN "features_smart_proxies" ON "features_smart_proxies"."smart_proxy_id" = "smart_proxies"."id" INNER JOIN "features" ON "features"."id" = "features_smart_proxies"."feature_id" WHERE "features"."name" = 'Puppet'  ORDER BY smart_proxies.name
2017-03-23 05:06:00 3a426e5e [app] [D] Resource not found
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (0.3ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "proxy_request_timeout"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (10.6ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "ssl_certificate"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (0.2ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "ssl_ca_file"]]
2017-03-23 05:06:00 3a426e5e [sql] [D]   Setting Load (0.2ms)  SELECT  "settings".* FROM "settings" WHERE "settings"."name" = $1  ORDER BY "settings"."name" ASC LIMIT 1  [["name", "ssl_priv_key"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]    (0.3ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]    (0.5ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]    (0.4ms)  SELECT DISTINCT "environments"."name" FROM "environments" INNER JOIN "taxable_taxonomies" ON "taxable_taxonomies"."taxable_id" = "environments"."id" AND "taxable_taxonomies"."taxable_type" = $1 WHERE 1=0  [["taxable_type", "Environment"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]    (0.6ms)  SELECT "environments"."name" FROM "environments" WHERE "environments"."name" IN ('common', 'production', 'development')  ORDER BY environments.name
2017-03-23 05:06:01 3a426e5e [sql] [D]   Environment Load (0.1ms)  SELECT  "environments".* FROM "environments" WHERE "environments"."name" = $1  ORDER BY environments.name LIMIT 1  [["name", "common"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT  "environments".* FROM "environments" WHERE "environments"."name" = $1  ORDER BY environments.name LIMIT 1  [["name", "common"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]    (0.6ms)  SELECT environments.name FROM "environments"  ORDER BY environments.name
2017-03-23 05:06:01 3a426e5e [sql] [D]   Environment Load (0.2ms)  SELECT  "environments".* FROM "environments" WHERE "environments"."name" = $1  ORDER BY environments.name LIMIT 1  [["name", "development"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT  "environments".* FROM "environments" WHERE "environments"."name" = $1  ORDER BY environments.name LIMIT 1  [["name", "development"]]
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:01 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT DISTINCT "environments"."name" FROM "environments" INNER JOIN "taxable_taxonomies" ON "taxable_taxonomies"."taxable_id" = "environments"."id" AND "taxable_taxonomies"."taxable_type" = $1 WHERE 1=0  [["taxable_type", "Environment"]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "environments"."name" FROM "environments" WHERE "environments"."name" IN ('common', 'production', 'development')  ORDER BY environments.name
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Organization')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "taxonomies"."id" FROM "taxonomies" WHERE "taxonomies"."type" IN ('Location')  ORDER BY "taxonomies"."title" ASC
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT DISTINCT "environments"."name" FROM "environments" INNER JOIN "taxable_taxonomies" ON "taxable_taxonomies"."taxable_id" = "environments"."id" AND "taxable_taxonomies"."taxable_type" = $1 WHERE 1=0  [["taxable_type", "Environment"]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   CACHE (0.0ms)  SELECT "environments"."name" FROM "environments" WHERE "environments"."name" IN ('common', 'production', 'development')  ORDER BY environments.name
2017-03-23 05:06:02 3a426e5e [sql] [D]   SmartProxy Load (0.7ms)  SELECT  "smart_proxies".* FROM "smart_proxies" INNER JOIN "features_smart_proxies" ON "features_smart_proxies"."smart_proxy_id" = "smart_proxies"."id" INNER JOIN "features" ON "features"."id" = "features_smart_proxies"."feature_id" WHERE "features"."name" = 'Puppet'  ORDER BY smart_proxies.name LIMIT 1
2017-03-23 05:06:02 3a426e5e [sql] [D]   Environment Load (0.2ms)  SELECT  "environments".* FROM "environments" WHERE "environments"."name" = $1  ORDER BY environments.name LIMIT 1  [["name", "production"]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   Puppetclass Load (0.3ms)  SELECT "puppetclasses".* FROM "puppetclasses" WHERE "puppetclasses"."name" = '_destroy_'  ORDER BY puppetclasses.name
2017-03-23 05:06:02 3a426e5e [sql] [D]   HostClass Load (0.6ms)  SELECT "host_classes".* FROM "host_classes" INNER JOIN "hosts" ON "hosts"."id" = "host_classes"."host_id" AND "hosts"."type" IN ('Host::Managed') WHERE "hosts"."environment_id" = $1 AND "host_classes"."puppetclass_id" IN (SELECT "puppetclasses"."id" FROM "puppetclasses" WHERE "puppetclasses"."name" = '_destroy_'  ORDER BY puppetclasses.name)  [["environment_id", 1]]
2017-03-23 05:06:02 3a426e5e [sql] [D]    (0.1ms)  BEGIN
2017-03-23 05:06:02 3a426e5e [sql] [D]   TaxableTaxonomy Load (0.1ms)  SELECT "taxable_taxonomies".* FROM "taxable_taxonomies" WHERE "taxable_taxonomies"."taxable_id" = $1 AND "taxable_taxonomies"."taxable_type" = $2  [["taxable_id", 1], ["taxable_type", "Environment"]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   Host::Managed Load (0.2ms)  SELECT "hosts".* FROM "hosts" WHERE "hosts"."type" IN ('Host::Managed') AND "hosts"."environment_id" = $1  [["environment_id", 1]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   Permission Load (0.3ms)  SELECT "permissions".* FROM "permissions" WHERE "permissions"."name" = $1  [["name", "view_hosts"]]
2017-03-23 05:06:02 3a426e5e [sql] [D]   Host::Managed Load (0.3ms)  SELECT "hosts".* FROM "hosts" WHERE "hosts"."type" IN ('Host::Managed')
2017-03-23 05:06:02 3a426e5e [sql] [D]   Hostgroup Load (0.3ms)  SELECT "hostgroups".* FROM "hostgroups" WHERE "hostgroups"."environment_id" = $1  [["environment_id", 1]]
2017-03-23 05:06:02 3a426e5e [app] [E] You may not destroy production as it is in use!
2017-03-23 05:06:02 3a426e5e [sql] [D]    (0.1ms)  ROLLBACK
2017-03-23 05:06:02 3a426e5e [sql] [D]   SQL (0.4ms)  SELECT "puppetclasses"."id" AS t0_r0, "puppetclasses"."name" AS t0_r1, "puppetclasses"."created_at" AS t0_r2, "puppetclasses"."updated_at" AS t0_r3, "environment_classes"."puppetclass_id" AS t1_r0, "environment_classes"."environment_id" AS t1_r1, "environment_classes"."id" AS t1_r2, "environment_classes"."puppetclass_lookup_key_id" AS t1_r3 FROM "puppetclasses" LEFT OUTER JOIN "environment_classes" ON "environment_classes"."puppetclass_id" = "puppetclasses"."id" WHERE "puppetclasses"."name" = '_destroy_' AND "environment_classes"."environment_id" IS NULL  ORDER BY puppetclasses.name
2017-03-23 05:06:02 3a426e5e [app] [I]   Rendered api/v2/import_puppetclasses/index.json.rabl within api/layouts/import_puppetclasses_layout (1.9ms)
2017-03-23 05:06:02 3a426e5e [app] [D] Body: {
 |   "message": "Successfully updated environment and puppetclasses from the on-disk puppet installation",
 |   "environments_with_new_puppetclasses": 0,
 |   "environments_updated_puppetclasses": 0,
 |   "environments_obsolete": 1,
 |   "environments_ignored": 0,
 |   "results": [{"name":"production","actions":["obsolete"],"removed_environment":"production"}]
 | }
 |
2017-03-23 05:06:02 3a426e5e [app] [I] Completed 200 OK in 2332ms (Views: 7.1ms | ActiveRecord: 161.6ms)

The error "You may not destroy production as it is in use!" and API response shows the production environment being marked as obsolete.

User#visible_environments attempts to join environments to taxable_taxonomies irrespective of which organisation and/or location features are enabled. It should work correctly for all permutations.

There should not be queries to the taxonomies or taxable_taxonomies tables in a default installation, but there are many in the above log file.

Since #17463.


Related issues

Related to Foreman - Bug #17463: Importing classes for a new org fails when the environment already exist in a different orgClosed2016-11-23
Related to Foreman - Bug #19483: User with inherited admin flag cannot see environments outside org/locs during importClosed2017-05-09
Has duplicate Foreman - Bug #19455: 1.15.0-RC2 Scanning for puppet classes - Foreman attempts to delete environments that existDuplicate2017-05-03

Associated revisions

Revision f4b7c52f (diff)
Added by Daniel Lobato Garcia over 4 years ago

Fixes #18995, #18996 - visible_environments works w/o taxonomies

Revision d733a555 (diff)
Added by Daniel Lobato Garcia over 4 years ago

Fixes #18995, #18996 - visible_environments works w/o taxonomies

History

#1 Updated by Dominic Cleal over 4 years ago

  • Related to Bug #17463: Importing classes for a new org fails when the environment already exist in a different org added

#2 Updated by Dominic Cleal over 4 years ago

  • Has duplicate Bug #19455: 1.15.0-RC2 Scanning for puppet classes - Foreman attempts to delete environments that exist added

#3 Updated by The Foreman Bot over 4 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Daniel Lobato Garcia
  • Pull request https://github.com/theforeman/foreman/pull/4510 added

#4 Updated by Anonymous over 4 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#5 Updated by Dominic Cleal over 4 years ago

  • Related to Bug #19483: User with inherited admin flag cannot see environments outside org/locs during import added

Also available in: Atom PDF