Project

General

Profile

Bug #11375

Smart class parameters turn YAML to string when using a multi line yaml

Added by Ori Rabin over 7 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Dominic Cleal
Category:
Parameters
Target version:
1.9.3
Difficulty:
Triaged:
Bugzilla link:
Pull request:
https://github.com/theforeman/foreman/pull/2755
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Adding an override to a smart class parameter of type YAML with the value:
test:
example: hello

results in the host edit form showing the parameter as: --- ! "test: \nexample: hello\n"

deep_merge_exception.txt deep_merge_exception.txt 24.3 KB Anthony Lapenna, 10/27/2015 07:10 AM

Related issues

Related to Foreman - Tracker #4470: Usability of parameters and overridesNew

Related to Foreman - Bug #11782: Puppet override with invalid hash array doesn't give errorClosed2015-09-11
Related to Foreman - Bug #12095: Smart Class Parameters mangle erb interpolated variablesClosed2015-10-07
Related to Foreman - Bug #12284: host group parameter overrides reformatting YAML values to stringsNew2015-10-23
Has duplicate Foreman - Bug #11830: Different results when adding a parameter override from a hostgroup vs puppet class directlyDuplicate2015-09-15
Has duplicate Foreman - Bug #11962: Override of Host-Parameter is saved malformedDuplicate2015-09-25
Has duplicate Foreman - Bug #12191: Array-typed smart parameters changed to broken JSONDuplicate2015-10-15
Has duplicate Foreman - Bug #12324: Hash and array smart parameters are getting corrupted after multiple savesDuplicate2015-10-27

Associated revisions

Revision 00335956 (diff)
Added by Dominic Cleal over 7 years ago

fixes #11375, #11782 - validate a host's lookup_values

Enabling validation ensures the casting and validation within
LookupValue runs during host/hostgroup save. Tweaks to hostgroup
cloning ensure that the newly cloned hostgroup now passes validation
when it has associated lookup values.

Don't show any LookupValue#match errors in the host/hostgroup forms
as the value is autogenerated by the nested attribute setter.

Revision e476ae3a (diff)
Added by Dominic Cleal over 7 years ago

fixes #11375, #11782 - validate a host's lookup_values

Enabling validation ensures the casting and validation within
LookupValue runs during host/hostgroup save. Tweaks to hostgroup
cloning ensure that the newly cloned hostgroup now passes validation
when it has associated lookup values.

Don't show any LookupValue#match errors in the host/hostgroup forms
as the value is autogenerated by the nested attribute setter.

(cherry picked from commit 003359564b4ab5fa8198cc9b035df405e7e600cd)

History

#1 Updated by Ori Rabin over 7 years ago

  • Related to Tracker #4470: Usability of parameters and overrides added

#2 Updated by Dominic Cleal over 7 years ago

  • Category set to Parameters

#3 Updated by Dominic Cleal over 7 years ago

  • Legacy Backlogs Release (now unused) set to 91

#4 Updated by Dominic Cleal over 7 years ago

  • Has duplicate Bug #11830: Different results when adding a parameter override from a hostgroup vs puppet class directly added

#5 Updated by Dominic Cleal over 7 years ago

This seems to be because of the :validate => false on the association in host_common to lookup_value. The validation's necessary to cast the value correctly - and also to fix issues such as #11782.

#6 Updated by Dominic Cleal over 7 years ago

  • Related to Bug #11782: Puppet override with invalid hash array doesn't give error added

#7 Updated by Dominic Cleal over 7 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal
  • Legacy Backlogs Release (now unused) deleted (91)

#8 Updated by The Foreman Bot over 7 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/2755 added
  • Pull request deleted ()

#9 Updated by Dominic Cleal over 7 years ago

  • Has duplicate Bug #11962: Override of Host-Parameter is saved malformed added

#10 Updated by Julien Pivotto over 7 years ago

Why did you delete Release (1.9.3)? this is a very annoying bug and the PR has lots of conflicts with 1.9 branch so it would be great to see this backported properly in 1.9. Thanks :)

#11 Updated by Julien Pivotto over 7 years ago

The blocking problem is that each time you save the host (e.g change anything on the host and click save) the smart parameters are malformed

#12 Updated by Dominic Cleal over 7 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#13 Updated by Dominic Cleal over 7 years ago

  • Legacy Backlogs Release (now unused) set to 63

#14 Updated by Dominic Cleal over 7 years ago

Julien Pivotto wrote:

Why did you delete Release (1.9.3)?

I mistakenly thought it was a new, 1.9 regression, but it wasn't.

this is a very annoying bug and the PR has lots of conflicts with 1.9 branch so it would be great to see this backported properly in 1.9. Thanks :)

Yeah, I will investigate this but at the moment I don't have the time to make sure it works properly. The implementation of overrides between 1.10 and 1.9 is different in a few places, so I'm not sure if the fix will introduce further bugs in 1.9 - it's going to need some testing. For now I'm placing it on 1.10.

#15 Updated by Dominic Cleal over 7 years ago

  • Related to Bug #12095: Smart Class Parameters mangle erb interpolated variables added

#16 Updated by Anthony Lapenna over 7 years ago

+1 really annoying bug, would be nice if the fix could be release in 1.9.

#17 Updated by Vladimir Stackov over 7 years ago

+1 for 1.9 backport.

#18 Updated by Dominic Cleal over 7 years ago

  • Legacy Backlogs Release (now unused) changed from 63 to 91

No promises, but I'll move it to make sure it appears on the right TODO list. Hopefully the backport's possible.

#19 Updated by Anonymous over 7 years ago

  • Has duplicate Bug #12191: Array-typed smart parameters changed to broken JSON added

#20 Updated by Julien Pivotto over 7 years ago

The problem is the same for hostgroups. You can not save hostgroup without breaking the smart variables.

#21 Updated by larry campbell over 7 years ago

Is this fix found in 1.10.0-RC1?

#22 Updated by Dominic Cleal over 7 years ago

larry campbell wrote:

Is this fix found in 1.10.0-RC1?

Yes.

#23 Updated by Dominic Cleal over 7 years ago

  • Related to Bug #12284: host group parameter overrides reformatting YAML values to strings added

#24 Updated by Anthony Lapenna over 7 years ago

Btw, once your parameters are 'corrupted' you'll not be able to edit your host or even access the YML dump.

See the attached file for more details about the stacktrace.

#25 Updated by Dominic Cleal over 7 years ago

  • Has duplicate Bug #12324: Hash and array smart parameters are getting corrupted after multiple saves added

#26 Updated by Anonymous over 7 years ago

Dominic Cleal wrote:

No promises, but I'll move it to make sure it appears on the right TODO list. Hopefully the backport's possible.

Will this make it to 1.9.3?

#27 Updated by Dominic Cleal over 7 years ago

Alyssa H wrote:

Will this make it to 1.9.3?

Yep, seems to be working fine! I hope to get .3 out early next week as we've got pretty much everything I hoped for.

#28 Updated by Vladimir Stackov over 7 years ago

Hallelujah! Thanks Dominic!

#29 Updated by Tim Verhoeven over 7 years ago

I've just upgrade my 1.9.2 install to 1.9.3. But it seems that the issue isn't fixed in 1.9.3. I'm still having the issue. Could somebody else test and verify?

#30 Updated by Anthony Lapenna over 7 years ago

We're still having this issue but with array type parameters only.

#31 Updated by Brad Cowie over 7 years ago

It seems like 1.9.3 fixed it for me when using arrays on the host/group override page.

I did initially have trouble updating overrides that had previously been escaped and stored in the db. Try fixing your overrides manually in the puppet classes page and then when the correct unescaped version of the override is stored in the db try again from the host/group override page.

#32 Updated by Vincent Miszczak about 7 years ago

I still have the issue in 1.10 for a host belonging to a nested group.
Group have original multi line yaml->NestedGroup inherits->Host inherits->broken yaml

#33 Updated by Ben Bettridge almost 7 years ago

This also impacts me on 1.11.1.

Also available in: Atom PDF