While working on Issue #9815, I found that the way that the API does a "Hostgroup clone" operation, does not actually clone the parameters. Rather, it "moves" them over to the new Hostgroup. (changes the reference_id)

This leads to two hostgroups using the same backend parameter in the database, and the possibility of serious confusion and chaos. When I changed the one line of code to actually dup(licate) the parameters for #9815, it caused the API test to fail, because of the reference_id in the "new" parameters.

The test is currently succeeding, but should be failing because the parameters were not properly cloned.

As the fix for #9815 breaks this, this becomes a blocker.

This will require model changes.

Here is some debug output demonstrating the information above:

[vagrant@devforeman db]$ git checkout develop
error: You have local changes to 'app/models/hostgroup.rb'; cannot switch branches.
[vagrant@devforeman db]$ git checkout -- ..
[vagrant@devforeman db]$ cd ..
[vagrant@devforeman foreman]$ git checkout develop
Switched to branch 'develop'
[vagrant@devforeman foreman]$ rails console
The Apipie cache is turned off. Enable it and run apipie:cache rake task to speed up API calls.
Workaround for RbVmomi may not work as ComputeResource is already loaded: ComputeResource
/home/vagrant/foreman/config/initializers/rbvmomi.rb:20: warning: toplevel constant ComputeResource referenced by RbVmomi::VIM::ComputeResource
For some operations a user must be set, try User.current = User.first
Loading development environment (Rails 3.2.21)
irb(main):001:0> User.current = User.first
2015-03-18 18:28:57 [D]   User Load (0.2ms)  SELECT "users".* FROM "users" ORDER BY firstname LIMIT 1

** Hirb Warning: Too many fields for the current width. Configure your width and/or fields to avoid this error. Defaulting to a vertical table. **
********************************** 1. row **********************************
                     id: 2
                  login: foreman_api_admin
              firstname: API
               lastname: Admin
                   mail: 
                  admin: true
          last_login_on: 
         auth_source_id: 2
             created_at: 2014-11-25 10:00:08 UTC
             updated_at: 2014-11-25 10:00:08 UTC
          password_hash: 
          password_salt: 
                 locale: 
            avatar_hash: 
default_organization_id: 
    default_location_id: 
            lower_login: foreman_api_admin
           mail_enabled: true
               timezone: 
1 row in set
irb(main):002:0> hg=Hostgroup.find(1)
2015-03-18 18:29:14 [D]   Hostgroup Load (0.3ms)  SELECT "hostgroups".* FROM "hostgroups" WHERE "hostgroups"."id" = ? ORDER BY hostgroups.title LIMIT 1  [["id", 1]]
2015-03-18 18:29:14 [D]   Operatingsystem Load (0.1ms)  SELECT "operatingsystems".* FROM "operatingsystems" WHERE "operatingsystems"."id" = 1 ORDER BY title LIMIT 1

** Hirb Warning: Too many fields for the current width. Configure your width and/or fields to avoid this error. Defaulting to a vertical table. **
*************************** 1. row ***************************
                id: 1
              name: Testing
        created_at: 2014-11-25 10:05:23 UTC
        updated_at: 2014-11-25 10:36:26 UTC
    environment_id: 1
operatingsystem_id: 1
   architecture_id: 1
         medium_id: 1
         ptable_id: 7
         root_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.
puppet_ca_proxy_id: 
         use_image: 
        image_file: 
          ancestry: 
       vm_defaults: 
         subnet_id: 
         domain_id: 1
   puppet_proxy_id: 
             title: Testing
          realm_id: 
compute_profile_id: 1
         grub_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.
1 row in set
irb(main):003:0> hg.group_parameters
2015-03-18 18:29:26 [D]   GroupParameter Load (0.2ms)  SELECT "parameters".* FROM "parameters" WHERE "parameters"."type" IN ('GroupParameter') AND "parameters"."reference_id" = 1 ORDER BY parameters.name
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| id | name      | value   | refere... | create... | updated_at | type      | priority | hidden... |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| 1  | TestPa... | MyValue | 1         | 2015-0... | 2015-03... | GroupP... | 3        | *****     |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
1 row in set

1. This is the method that the API Clone controller calls, this indicates its not an API specific problem, but a model problem.

irb(main):004:0> clone_hg=hg.clone("MyNewClonedGroup")
2015-03-18 18:29:56 [D]   Puppetclass Load (0.1ms)  SELECT "puppetclasses".* FROM "puppetclasses" INNER JOIN "hostgroup_classes" ON "puppetclasses"."id" = "hostgroup_classes"."puppetclass_id" WHERE "hostgroup_classes"."hostgroup_id" = 1 ORDER BY puppetclasses.name
2015-03-18 18:29:56 [D]   Taxonomy Load (0.1ms)  SELECT "taxonomies".* FROM "taxonomies" INNER JOIN "taxable_taxonomies" ON "taxonomies"."id" = "taxable_taxonomies"."taxonomy_id" WHERE "taxable_taxonomies"."taxable_id" = 1 AND "taxable_taxonomies"."taxable_type" = 'Hostgroup' AND (taxonomies.type='Location') ORDER BY title
2015-03-18 18:29:56 [D]   Taxonomy Load (5.3ms)  SELECT "taxonomies".* FROM "taxonomies" INNER JOIN "taxable_taxonomies" ON "taxonomies"."id" = "taxable_taxonomies"."taxonomy_id" WHERE "taxable_taxonomies"."taxable_id" = 1 AND "taxable_taxonomies"."taxable_type" = 'Hostgroup' AND (taxonomies.type='Organization') ORDER BY title
2015-03-18 18:29:56 [D]   ConfigGroup Load (0.2ms)  SELECT "config_groups".* FROM "config_groups" INNER JOIN "host_config_groups" ON "config_groups"."id" = "host_config_groups"."config_group_id" WHERE "host_config_groups"."host_id" = 1 AND "host_config_groups"."host_type" = 'Hostgroup' ORDER BY config_groups.name
2015-03-18 18:29:56 [D]   Operatingsystem Load (0.1ms)  SELECT "operatingsystems".* FROM "operatingsystems" WHERE "operatingsystems"."id" = 1 ORDER BY title LIMIT 1

** Hirb Warning: Too many fields for the current width. Configure your width and/or fields to avoid this error. Defaulting to a vertical table. **
*************************** 1. row ***************************
              name: MyNewClonedGroup
        created_at: 
        updated_at: 
    environment_id: 1
operatingsystem_id: 1
   architecture_id: 1
         medium_id: 1
         ptable_id: 7
         root_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.
puppet_ca_proxy_id: 
         use_image: 
        image_file: 
          ancestry: 
       vm_defaults: 
         subnet_id: 
         domain_id: 1
   puppet_proxy_id: 
             title: Testing
          realm_id: 
compute_profile_id: 1
         grub_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.
1 row in set
irb(main):005:0> clone_hg.save
2015-03-18 18:30:06 [D]    (0.0ms)  begin transaction
2015-03-18 18:30:06 [D]   Hostgroup Exists (0.2ms)  SELECT 1 AS one FROM "hostgroups" WHERE (LOWER("hostgroups"."name") = LOWER('MyNewClonedGroup') AND "hostgroups"."ancestry" IS NULL) LIMIT 1
2015-03-18 18:30:06 [D]   Hostgroup Exists (1.0ms)  SELECT 1 AS one FROM "hostgroups" WHERE "hostgroups"."title" = 'MyNewClonedGroup' LIMIT 1
2015-03-18 18:30:06 [D]   GroupParameter Exists (0.1ms)  SELECT 1 AS one FROM "parameters" WHERE "parameters"."type" IN ('GroupParameter') AND ("parameters"."name" = 'TestParam1' AND "parameters"."id" != 1 AND "parameters"."reference_id" = 1) LIMIT 1
2015-03-18 18:30:06 [D]   Hostgroup Load (0.2ms)  SELECT "hostgroups".* FROM "hostgroups" WHERE 1=0 ORDER BY (case when hostgroups.ancestry is null then 0 else 1 end), hostgroups.ancestry
2015-03-18 18:30:06 [D]   SQL (0.6ms)  INSERT INTO "hostgroups" ("ancestry", "architecture_id", "compute_profile_id", "created_at", "domain_id", "environment_id", "grub_pass", "image_file", "medium_id", "name", "operatingsystem_id", "ptable_id", "puppet_ca_proxy_id", "puppet_proxy_id", "realm_id", "root_pass", "subnet_id", "title", "updated_at", "use_image", "vm_defaults") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)  [["ancestry", nil], ["architecture_id", 1], ["compute_profile_id", 1], ["created_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00], ["domain_id", 1], ["environment_id", 1], ["grub_pass", "$6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3."], ["image_file", nil], ["medium_id", 1], ["name", "MyNewClonedGroup"], ["operatingsystem_id", 1], ["ptable_id", 7], ["puppet_ca_proxy_id", nil], ["puppet_proxy_id", nil], ["realm_id", nil], ["root_pass", "$6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3."], ["subnet_id", nil], ["title", "MyNewClonedGroup"], ["updated_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00], ["use_image", nil], ["vm_defaults", nil]]
2015-03-18 18:30:06 [D]   TaxableTaxonomy Exists (0.1ms)  SELECT 1 AS one FROM "taxable_taxonomies" WHERE ("taxable_taxonomies"."taxonomy_id" = 1 AND "taxable_taxonomies"."taxable_id" = 4 AND "taxable_taxonomies"."taxable_type" = 'Hostgroup') LIMIT 1
2015-03-18 18:30:06 [D]   SQL (0.3ms)  INSERT INTO "taxable_taxonomies" ("created_at", "taxable_id", "taxable_type", "taxonomy_id", "updated_at") VALUES (?, ?, ?, ?, ?)  [["created_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00], ["taxable_id", 4], ["taxable_type", "Hostgroup"], ["taxonomy_id", 1], ["updated_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00]]
2015-03-18 18:30:06 [D]   TaxableTaxonomy Exists (0.1ms)  SELECT 1 AS one FROM "taxable_taxonomies" WHERE ("taxable_taxonomies"."taxonomy_id" = 2 AND "taxable_taxonomies"."taxable_id" = 4 AND "taxable_taxonomies"."taxable_type" = 'Hostgroup') LIMIT 1
2015-03-18 18:30:06 [D]   SQL (0.1ms)  INSERT INTO "taxable_taxonomies" ("created_at", "taxable_id", "taxable_type", "taxonomy_id", "updated_at") VALUES (?, ?, ?, ?, ?)  [["created_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00], ["taxable_id", 4], ["taxable_type", "Hostgroup"], ["taxonomy_id", 2], ["updated_at", Thu, 19 Mar 2015 00:30:06 UTC +00:00]]
2015-03-18 18:30:06 [D]   Architecture Load (0.3ms)  SELECT "architectures".* FROM "architectures" WHERE "architectures"."id" = 1 LIMIT 1
2015-03-18 18:30:06 [D]   SQL (0.3ms)  UPDATE "architectures" SET "hostgroups_count" = COALESCE("hostgroups_count", 0) + 1 WHERE "architectures"."id" = 1
2015-03-18 18:30:06 [D]   Environment Load (0.1ms)  SELECT "environments".* FROM "environments" WHERE "environments"."id" = 1 ORDER BY environments.name LIMIT 1
2015-03-18 18:30:06 [D]   SQL (0.2ms)  UPDATE "environments" SET "hostgroups_count" = COALESCE("hostgroups_count", 0) + 1 WHERE "environments"."id" IN (SELECT "environments"."id" FROM "environments" WHERE "environments"."id" = 1 ORDER BY environments.name)
2015-03-18 18:30:06 [D]   SQL (0.1ms)  UPDATE "operatingsystems" SET "hostgroups_count" = COALESCE("hostgroups_count", 0) + 1 WHERE "operatingsystems"."id" IN (SELECT "operatingsystems"."id" FROM "operatingsystems" WHERE "operatingsystems"."type" IN ('Redhat') AND "operatingsystems"."id" = 1 ORDER BY title)
2015-03-18 18:30:07 [D]   Hostgroup Load (0.2ms)  SELECT "hostgroups".* FROM "hostgroups" WHERE "hostgroups"."id" = 4 ORDER BY hostgroups.title LIMIT 1
2015-03-18 18:30:07 [D]    (19.1ms)  SELECT MAX("audits"."version") AS max_id FROM "audits" WHERE "audits"."auditable_id" = 4 AND "audits"."auditable_type" = 'Hostgroup'
2015-03-18 18:30:07 [D]   SQL (8.8ms)  INSERT INTO "audits" ("action", "associated_id", "associated_name", "associated_type", "auditable_id", "auditable_name", "auditable_type", "audited_changes", "comment", "created_at", "remote_address", "user_id", "user_type", "username", "version") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)  [["action", "create"], ["associated_id", nil], ["associated_name", nil], ["associated_type", nil], ["auditable_id", 4], ["auditable_name", "MyNewClonedGroup"], ["auditable_type", "Hostgroup"], ["audited_changes", "---\nname: MyNewClonedGroup\nenvironment_id: 1\noperatingsystem_id: 1\narchitecture_id: 1\nmedium_id: 1\nptable_id: 7\nroot_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.\npuppet_ca_proxy_id: \nuse_image: \nimage_file: \nancestry: \nvm_defaults: \nsubnet_id: \ndomain_id: 1\npuppet_proxy_id: \nrealm_id: \ncompute_profile_id: 1\ngrub_pass: $6$tfESPO6K$Ku5wTFTUiYwmAPwQxh1IJOr9oFukQIOUAN.DxP6husJ.seki6/lUf3kLIN3jMbJRFr6AKvuTZ99ItWwlGp0N3.\n"], ["comment", nil], ["created_at", Thu, 19 Mar 2015 00:30:07 UTC +00:00], ["remote_address", nil], ["user_id", 2], ["user_type", nil], ["username", "API Admin"], ["version", 1]]
2015-03-18 18:30:07 [D]   Parameter Load (0.2ms)  SELECT "parameters".* FROM "parameters" WHERE "parameters"."id" = 1 ORDER BY parameters.name LIMIT 1
2015-03-18 18:30:07 [D]    (0.1ms)  SELECT MAX("audits"."version") AS max_id FROM "audits" WHERE "audits"."auditable_id" = 1 AND "audits"."auditable_type" = 'Parameter'
2015-03-18 18:30:07 [D]   SQL (0.2ms)  INSERT INTO "audits" ("action", "associated_id", "associated_name", "associated_type", "auditable_id", "auditable_name", "auditable_type", "audited_changes", "comment", "created_at", "remote_address", "user_id", "user_type", "username", "version") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)  [["action", "update"], ["associated_id", 4], ["associated_name", "MyNewClonedGroup"], ["associated_type", "Hostgroup"], ["auditable_id", 1], ["auditable_name", "TestParam1"], ["auditable_type", "Parameter"], ["audited_changes", "---\nreference_id:\n- 1\n- 4\n"], ["comment", nil], ["created_at", Thu, 19 Mar 2015 00:30:07 UTC +00:00], ["remote_address", nil], ["user_id", 2], ["user_type", nil], ["username", "API Admin"], ["version", 2]]
2015-03-18 18:30:07 [D]    (0.1ms)  UPDATE "parameters" SET "reference_id" = 4, "updated_at" = '2015-03-19 00:30:07.267996' WHERE "parameters"."type" IN ('GroupParameter') AND "parameters"."id" = 1
2015-03-18 18:30:07 [D]   Domain Load (0.2ms)  SELECT "domains".* FROM "domains" WHERE "domains"."id" = 1 ORDER BY domains.name LIMIT 1
2015-03-18 18:30:07 [D]   SQL (0.1ms)  UPDATE "domains" SET "hostgroups_count" = COALESCE("hostgroups_count", 0) + 1 WHERE "domains"."id" IN (SELECT "domains"."id" FROM "domains" WHERE "domains"."id" = 1 ORDER BY domains.name)
2015-03-18 18:30:07 [D]   Hostgroup Load (0.1ms)  SELECT "hostgroups".* FROM "hostgroups" WHERE (ancestry IS NOT NULL) ORDER BY hostgroups.title
2015-03-18 18:30:08 [D]   SQL (0.2ms)  UPDATE "lookup_values" SET "match" = 'hostgroup=MyNewClonedGroup' WHERE "lookup_values"."match" = 'hostgroup='
2015-03-18 18:30:08 [D]    (74.6ms)  commit transaction
=> true
irb(main):006:0> clone_hg.group_parameters
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| id | name      | value   | refere... | create... | updated_at | type      | priority | hidden... |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| 1  | TestPa... | MyValue | 4         | 2015-0... | 2015-03... | GroupP... | 3        | *****     |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
1 row in set

1. NOTICE (above) how the reference_id has changed to "4" but its still parameter_id "1" ... but hg (hostgroup_id=1) is still pointing to parameter_id 1*this is bad* :)

irb(main):009:0> hg.group_parameters
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| id | name      | value   | refere... | create... | updated_at | type      | priority | hidden... |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
| 1  | TestPa... | MyValue | 4         | 2015-0... | 2015-03... | GroupP... | 3        | *****     |
+----+-----------+---------+-----------+-----------+------------+-----------+----------+-----------+
1 row in set
irb(main):010:0> 

This was found originally in 1.7.3 and reproduced in the "develop" branch as of today.