Bug #9673
YAML type casting fails for lookup values in 1.7.3
Description
foreman-debug dump (rsync'd):
foreman-debug-4edVQ.tar.xz
Error logs (verbose enabled):
http://pastebin.com/raw.php?i=N2X0LuZx
- puppet agent --test
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: Error 400 on SERVER: Failed to find cloud.chrisaw.com via exec: Execution of '/etc/puppet/node.rb cloud.chrisaw.com' returned 1:
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed when searching for node cloud.chrisaw.com: Failed to find cloud.chrisaw.com via exec: Execution of '/etc/puppet/node.rb cloud.chrisaw.com' returned 1:
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
- /etc/puppet/node.rb cloud.chrisaw.com
Error retrieving node cloud.chrisaw.com: Net::HTTPPreconditionFailed
Check Foreman's /var/log/foreman/production.log for more information.
Related issues
Associated revisions
Fixes #9673 - treat YAML and JSON lookup keys as strings
(cherry picked from commit 25a7f806ed586bd32ee647f2acd8e5a0224cbbfb)
Fixes #9673 - treat YAML and JSON lookup keys as strings
(cherry picked from commit 25a7f806ed586bd32ee647f2acd8e5a0224cbbfb)
History
#1
Updated by Greg Sutcliffe almost 8 years ago
Looks like the fix to #8052 has added "logger" when it should be "Rails.logger" or similar, which is covering up a real error. Marek, this was your commit right?
#2
Updated by Greg Sutcliffe almost 8 years ago
- Subject changed from Puppet agent updates fail with Foreman 1.7.3 to YAML type casting fails for lookup values in 1.7.3
- Category set to External Nodes
Once the `logger` issue was resolved, we could get to the real issue. For some reason app/services/classification/base.rb#type_cast throws this error:
Unable to type cast {"ubuntu-main"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "ubuntu-security"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty-security", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "ubuntu-updates"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty-updates", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "puppetlabs"=>{"location"=>"http://apt.puppetlabs.com", "repos"=>"main", "key"=>"1054B7A24BD6EC30", "key_server"=>"pgp.mit.edu", "include_deb"=>"true", "include_src"=>"false"}, "foreman"=>{"location"=>"http://deb.theforeman.org/", "release"=>"trusty", "repos"=>"1.7", "key_source"=>"http://deb.theforeman.org/pubkey.gpg", "include_deb"=>"true", "include_src"=>"false"}, "foreman-plugins"=>{"location"=>"http://deb.theforeman.org/", "release"=>"plugins", "repos"=>"1.7", "key_source"=>"http://deb.theforeman.org/pubkey.gpg", "include_deb"=>"true", "include_src"=>"false"}, "docker"=>{"location"=>"https://get.docker.com/ubuntu", "release"=>"docker", "repos"=>"main", "key"=>"D8576A8BA88D21E9", "key_server"=>"keyserver.ubuntu.com", "include_deb"=>"true", "include_src"=>"false"}} to yaml
THis is when using a yaml hash like:
ubuntu-main:
location: http://archive.ubuntu.com/ubuntu/
release: trusty
repos: main restricted universe multiverse
include_src: true
include_deb: true
ubuntu-security:
location: http://archive.ubuntu.com/ubuntu/
release: trusty-security
repos: main restricted universe multiverse
include_src: true
include_deb: true
ubuntu-updates:
location: http://archive.ubuntu.com/ubuntu/
release: trusty-updates
repos: main restricted universe multiverse
include_src: true
include_deb: true
puppetlabs:
location: http://apt.puppetlabs.com
repos: main
key: 1054B7A24BD6EC30
key_server: pgp.mit.edu
include_deb: true
include_src: false
Note how the yaml has become a ruby hash in the log output. Because `type_cast` doesn't have a return in the rescue, it returns "true" leading to your ENC yaml being incorrect.
I gave Chris this workaround:
--- a/app/services/classification/base.rb +++ b/app/services/classification/base.rb @@ -147,6 +147,7 @@ module Classification key.cast_validate_value(value) rescue TypeError logger.warn "Unable to type cast #{value} to #{key.key_type}" + return value end
but there may be better options. Marek, thoughts?
#3
Updated by The Foreman Bot almost 8 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/foreman/pull/2228 added
- Pull request deleted (
)
#4
Updated by Dominic Cleal almost 8 years ago
- Legacy Backlogs Release (now unused) set to 36
#5
Updated by Marek Hulán almost 8 years ago
- Assignee set to Marek Hulán
- Pull request https://github.com/theforeman/foreman/pull/2232 added
- Pull request deleted (
https://github.com/theforeman/foreman/pull/2228)
Here's my version of the fix https://github.com/theforeman/foreman/pull/2232, JSON and YAML parameters were automatically parsed (thanks ActiveRecord) so they appeared as Hashes, then we tried to convert hash to YAML rather then string to YAML (or JSON...)
#6
Updated by Marek Hulán almost 8 years ago
- Status changed from Ready For Testing to Closed
- % Done changed from 0 to 100
Applied in changeset 25a7f806ed586bd32ee647f2acd8e5a0224cbbfb.
#7
Updated by Martin Hubert almost 8 years ago
- File stacktrace.txt stacktrace.txt added
I am also affected by this problem. The patch does not solve the problem for global Smart Class Parameters. I was able to log a stack trace.
My quick-hack is to add a "return value" inside the rescue block around line 148 in app/services/classification/base.rb
#8
Updated by Dominic Cleal almost 8 years ago
Martin Hubert wrote:
I am also affected by this problem. The patch does not solve the problem for global Smart Class Parameters. I was able to log a stack trace.
My quick-hack is to add a "return value" inside the rescue block around line 148 in app/services/classification/base.rb
Could I just check which patch you're using? The one from the linked pull request, or the one in comment 2?
#9
Updated by Martin Hubert almost 8 years ago
I use patch from the linked pull request (2232).
It works for host parameters but not for global smart class parameters. To get this working I also must apply the workaround like suggested by Greg in comment 2.
#10
Updated by Marek Hulán almost 8 years ago
Just to be sure, by "global smart class parameter" you actually mean Smart variable?
#11
Updated by Martin Hubert almost 8 years ago
- File SmartClassParameter.jpg SmartClassParameter.jpg added
I mean "Smart Class Parameters"
Accessible by :
Configure -> Puppet classes -> <class> -> Smart Class Parameters
(see screenshot)
#12
Updated by Marek Hulán almost 8 years ago
This works for me, but I was able to reproduce the behavior that returns "true" instead of that YAML in case there's no override and default value is used. Do you have some overrides?
#13
Updated by Marek Hulán almost 8 years ago
- Related to Bug #9787: Default values of smart class parameters can't be JSON or YAML added
#14
Updated by Dominic Cleal almost 8 years ago
- Related to Bug #8052: Smart class parameters with ERB snippets are broken for other that String types added
Fixes #9673 - treat YAML and JSON lookup keys as strings