Bug #11375

Smart class parameters turn YAML to string when using a multi line yaml

Added by Ori Rabin over 2 years ago. Updated almost 2 years ago.

Status:Closed
Priority:Normal
Assigned To:Dominic Cleal
Category:Smart Variables
Target version:-
Difficulty: Bugzilla link:
Found in release: Pull request:https://github.com/theforeman/foreman/pull/2755
Story points-
Velocity based estimate-
Release1.9.3Release relationshipAuto

Description

Adding an override to a smart class parameter of type YAML with the value:
test:
example: hello

results in the host edit form showing the parameter as: --- ! "test: \nexample: hello\n"

deep_merge_exception.txt Magnifier (24.3 KB) Anthony Lapenna, 10/27/2015 07:10 AM


Related issues

Related to Foreman - Tracker #4470: Usability of parameters and overrides New 02/27/2014
Related to Foreman - Bug #11782: Puppet override with invalid hash array doesn't give error Closed 09/11/2015
Related to Foreman - Bug #12095: Smart Class Parameters mangle erb interpolated variables Closed 10/07/2015
Related to Foreman - Bug #12284: host group parameter overrides reformatting YAML values t... New 10/23/2015
Duplicated by Foreman - Bug #11830: Different results when adding a parameter override from a... Duplicate 09/15/2015
Duplicated by Foreman - Bug #11962: Override of Host-Parameter is saved malformed Duplicate 09/25/2015
Duplicated by Foreman - Bug #12191: Array-typed smart parameters changed to broken JSON Duplicate 10/15/2015
Duplicated by Foreman - Bug #12324: Hash and array smart parameters are getting corrupted aft... Duplicate 10/27/2015

Associated revisions

Revision 00335956
Added by Dominic Cleal over 2 years ago

fixes #11375, #11782 - validate a host's lookup_values

Enabling validation ensures the casting and validation within
LookupValue runs during host/hostgroup save. Tweaks to hostgroup
cloning ensure that the newly cloned hostgroup now passes validation
when it has associated lookup values.

Don't show any LookupValue#match errors in the host/hostgroup forms
as the value is autogenerated by the nested attribute setter.

Revision e476ae3a
Added by Dominic Cleal over 2 years ago

fixes #11375, #11782 - validate a host's lookup_values

Enabling validation ensures the casting and validation within
LookupValue runs during host/hostgroup save. Tweaks to hostgroup
cloning ensure that the newly cloned hostgroup now passes validation
when it has associated lookup values.

Don't show any LookupValue#match errors in the host/hostgroup forms
as the value is autogenerated by the nested attribute setter.

(cherry picked from commit 003359564b4ab5fa8198cc9b035df405e7e600cd)

History

#1 Updated by Ori Rabin over 2 years ago

  • Related to Tracker #4470: Usability of parameters and overrides added

#2 Updated by Dominic Cleal over 2 years ago

  • Category set to Smart Variables

#3 Updated by Dominic Cleal over 2 years ago

  • Release set to 1.9.3

#4 Updated by Dominic Cleal over 2 years ago

  • Duplicated by Bug #11830: Different results when adding a parameter override from a hostgroup vs puppet class directly added

#5 Updated by Dominic Cleal over 2 years ago

This seems to be because of the :validate => false on the association in host_common to lookup_value. The validation's necessary to cast the value correctly - and also to fix issues such as #11782.

#6 Updated by Dominic Cleal over 2 years ago

  • Related to Bug #11782: Puppet override with invalid hash array doesn't give error added

#7 Updated by Dominic Cleal over 2 years ago

  • Status changed from New to Assigned
  • Assigned To set to Dominic Cleal
  • Release deleted (1.9.3)

#8 Updated by The Foreman Bot over 2 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/2755 added

#9 Updated by Dominic Cleal over 2 years ago

  • Duplicated by Bug #11962: Override of Host-Parameter is saved malformed added

#10 Updated by Julien Pivotto over 2 years ago

Why did you delete Release (1.9.3)? this is a very annoying bug and the PR has lots of conflicts with 1.9 branch so it would be great to see this backported properly in 1.9. Thanks :)

#11 Updated by Julien Pivotto over 2 years ago

The blocking problem is that each time you save the host (e.g change anything on the host and click save) the smart parameters are malformed

#12 Updated by Dominic Cleal over 2 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#13 Updated by Dominic Cleal over 2 years ago

  • Release set to 1.10.0

#14 Updated by Dominic Cleal over 2 years ago

Julien Pivotto wrote:

Why did you delete Release (1.9.3)?

I mistakenly thought it was a new, 1.9 regression, but it wasn't.

this is a very annoying bug and the PR has lots of conflicts with 1.9 branch so it would be great to see this backported properly in 1.9. Thanks :)

Yeah, I will investigate this but at the moment I don't have the time to make sure it works properly. The implementation of overrides between 1.10 and 1.9 is different in a few places, so I'm not sure if the fix will introduce further bugs in 1.9 - it's going to need some testing. For now I'm placing it on 1.10.

#15 Updated by Dominic Cleal over 2 years ago

  • Related to Bug #12095: Smart Class Parameters mangle erb interpolated variables added

#16 Updated by Anthony Lapenna over 2 years ago

+1 really annoying bug, would be nice if the fix could be release in 1.9.

#17 Updated by Vladimir Stackov over 2 years ago

+1 for 1.9 backport.

#18 Updated by Dominic Cleal over 2 years ago

  • Release changed from 1.10.0 to 1.9.3

No promises, but I'll move it to make sure it appears on the right TODO list. Hopefully the backport's possible.

#19 Updated by Michael Moll over 2 years ago

  • Duplicated by Bug #12191: Array-typed smart parameters changed to broken JSON added

#20 Updated by Julien Pivotto over 2 years ago

The problem is the same for hostgroups. You can not save hostgroup without breaking the smart variables.

#21 Updated by larry campbell over 2 years ago

Is this fix found in 1.10.0-RC1?

#22 Updated by Dominic Cleal over 2 years ago

larry campbell wrote:

Is this fix found in 1.10.0-RC1?

Yes.

#23 Updated by Dominic Cleal over 2 years ago

  • Related to Bug #12284: host group parameter overrides reformatting YAML values to strings added

#24 Updated by Anthony Lapenna over 2 years ago

Btw, once your parameters are 'corrupted' you'll not be able to edit your host or even access the YML dump.

See the attached file for more details about the stacktrace.

#25 Updated by Dominic Cleal over 2 years ago

  • Duplicated by Bug #12324: Hash and array smart parameters are getting corrupted after multiple saves added

#26 Updated by Anonymous over 2 years ago

Dominic Cleal wrote:

No promises, but I'll move it to make sure it appears on the right TODO list. Hopefully the backport's possible.

Will this make it to 1.9.3?

#27 Updated by Dominic Cleal over 2 years ago

Alyssa H wrote:

Will this make it to 1.9.3?

Yep, seems to be working fine! I hope to get .3 out early next week as we've got pretty much everything I hoped for.

#28 Updated by Vladimir Stackov over 2 years ago

Hallelujah! Thanks Dominic!

#29 Updated by Tim Verhoeven over 2 years ago

I've just upgrade my 1.9.2 install to 1.9.3. But it seems that the issue isn't fixed in 1.9.3. I'm still having the issue. Could somebody else test and verify?

#30 Updated by Anthony Lapenna over 2 years ago

We're still having this issue but with array type parameters only.

#31 Updated by Brad Cowie over 2 years ago

It seems like 1.9.3 fixed it for me when using arrays on the host/group override page.

I did initially have trouble updating overrides that had previously been escaped and stored in the db. Try fixing your overrides manually in the puppet classes page and then when the correct unescaped version of the override is stored in the db try again from the host/group override page.

#32 Updated by Vincent Miszczak over 2 years ago

I still have the issue in 1.10 for a host belonging to a nested group.
Group have original multi line yaml->NestedGroup inherits->Host inherits->broken yaml

#33 Updated by Ben Bettridge almost 2 years ago

This also impacts me on 1.11.1.

Also available in: Atom PDF