Project

General

Profile

Bug #23581

Upgrade to Foreman 1.17 converts YAML to JSON Hash

Added by Frederik Weber 7 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
DB migrations
Target version:
1.18.2
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
https://github.com/theforeman/foreman/pull/5559
Team Backlog:
Fixed in Releases:
Found in Releases:
1.17.0

Description

We have Puppet parameters with the type hash and a YAML hash as content (this is valid per Foreman documentation).
After the run of db:migrate the content gets migrated to a JSON hash.

If I extract an example value from the DB:

Before db:migrate

---             
frequency: daily

After db:migrate

 --- '{"frequency"=>"daily"}'

I see two solutions:
1. Change the db:migrate task that migrates valid YAML hashes to JSON hashes.
2. Update the documentation that YAML hashes are no longer valid.

Related issues

Related to Foreman - Bug #23382: Editing Smart Class Parameter of type YAML creates malformed value in release 1.17.0Closed2018-04-24
Related to Foreman - Bug #24640: 1.17 migration causes array/hash values for parameters to turn into strings with escaped quotesClosed

History

#1 Updated by Frederik Weber 7 months ago

Just verified that an upgrade from 1.16.0 to 1.16.1 doesn't convert hashes.
Therefore it is likely that the issue was introduced in the version 1.17.

#2 Updated by Frederik Weber 7 months ago

It is likely that this "bug" was introduced by PR #5403 (https://github.com/theforeman/foreman/pull/5403).
Should all YAML lookup_values become JSON hashes? Especially since it is possible to change the values back to YAML hashes (through the UI) and they get written like that to the DB.

#3 Updated by The Foreman Bot 7 months ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/5556 added

#4 Updated by Frederik Weber 7 months ago

  • Pull request deleted (https://github.com/theforeman/foreman/pull/5556)

#5 Updated by The Foreman Bot 7 months ago

  • Pull request https://github.com/theforeman/foreman/pull/5559 added

#6 Updated by Tomer Brisker 5 months ago

  • Triaged set to No
  • Legacy Backlogs Release (now unused) set to 360

#7 Updated by Tomer Brisker 5 months ago

  • Related to Bug #23382: Editing Smart Class Parameter of type YAML creates malformed value in release 1.17.0 added

#8 Updated by Tomer Brisker 4 months ago

  • Target version changed from 1.17.2 to 1.18.2

1.17.3 has been released, pushing out to 1.18.2

#9 Updated by Thomas BAERT 4 months ago

Hello,

https://github.com/theforeman/foreman/blob/1.17-stable/db/migrate/20170112175131_migrate_template_to_parameters_macros.rb#L34

parameter.value is a ruby Hash when Lookup_value type is Yaml/Hash in Foreman.
You convert it in String with to.s.
But you don't convert this String in Hash before update_attribute.
So YAML content is replaced by the String 'ruby Hash'.

Same problem with Array in Foreman (it's a ruby Array).
And same problem for LookupKey.default_value (https://github.com/theforeman/foreman/blob/1.17-stable/db/migrate/20170112175131_migrate_template_to_parameters_macros.rb#L25).

#10 Updated by Tomer Brisker 3 months ago

  • Status changed from Ready For Testing to Resolved

This issue has been resolved by #24640.

#11 Updated by Tomer Brisker 3 months ago

  • Related to Bug #24640: 1.17 migration causes array/hash values for parameters to turn into strings with escaped quotes added

Also available in: Atom PDF