Project

General

Profile

Bug #9673

YAML type casting fails for lookup values in 1.7.3

Added by Christopher Williamson over 4 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Urgent
Assignee:
Category:
External Nodes
Target version:
Difficulty:
Triaged:
Bugzilla link:
Team Backlog:
Fixed in Releases:
Found in Releases:

Description

foreman-debug dump (rsync'd):
foreman-debug-4edVQ.tar.xz

Error logs (verbose enabled):
http://pastebin.com/raw.php?i=N2X0LuZx

Command output:
  1. puppet agent --test
    Warning: Unable to fetch my node definition, but the agent run will continue:
    Warning: Error 400 on SERVER: Failed to find cloud.chrisaw.com via exec: Execution of '/etc/puppet/node.rb cloud.chrisaw.com' returned 1:
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Loading facts
    Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed when searching for node cloud.chrisaw.com: Failed to find cloud.chrisaw.com via exec: Execution of '/etc/puppet/node.rb cloud.chrisaw.com' returned 1:
    Warning: Not using cache on failed catalog
    Error: Could not retrieve catalog; skipping run
Deeper command output:
  1. /etc/puppet/node.rb cloud.chrisaw.com
    Error retrieving node cloud.chrisaw.com: Net::HTTPPreconditionFailed
    Check Foreman's /var/log/foreman/production.log for more information.
stacktrace.txt stacktrace.txt 12.2 KB Martin Hubert, 03/16/2015 08:40 AM
SmartClassParameter.jpg View SmartClassParameter.jpg 62 KB Screenshot Smart Class Parameter Martin Hubert, 03/16/2015 10:55 AM
Smartclassparameter

Related issues

Related to Foreman - Bug #9787: Default values of smart class parameters can't be JSON or YAMLClosed2015-03-16
Related to Foreman - Bug #8052: Smart class parameters with ERB snippets are broken for other that String typesClosed2014-10-23

Associated revisions

Revision 25a7f806 (diff)
Added by Marek Hulán over 4 years ago

Fixes #9673 - treat YAML and JSON lookup keys as strings

Revision a0c60c7d (diff)
Added by Marek Hulán over 4 years ago

Fixes #9673 - treat YAML and JSON lookup keys as strings

(cherry picked from commit 25a7f806ed586bd32ee647f2acd8e5a0224cbbfb)

Revision 7628450f (diff)
Added by Marek Hulán over 4 years ago

Fixes #9673 - treat YAML and JSON lookup keys as strings

(cherry picked from commit 25a7f806ed586bd32ee647f2acd8e5a0224cbbfb)

History

#1 Updated by Greg Sutcliffe over 4 years ago

Looks like the fix to #8052 has added "logger" when it should be "Rails.logger" or similar, which is covering up a real error. Marek, this was your commit right?

#2 Updated by Greg Sutcliffe over 4 years ago

  • Subject changed from Puppet agent updates fail with Foreman 1.7.3 to YAML type casting fails for lookup values in 1.7.3
  • Category set to External Nodes

Once the `logger` issue was resolved, we could get to the real issue. For some reason app/services/classification/base.rb#type_cast throws this error:

Unable to type cast {"ubuntu-main"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "ubuntu-security"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty-security", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "ubuntu-updates"=>{"location"=>"http://archive.ubuntu.com/ubuntu/", "release"=>"trusty-updates", "repos"=>"main restricted universe multiverse", "include_src"=>"true", "include_deb"=>"true"}, "puppetlabs"=>{"location"=>"http://apt.puppetlabs.com", "repos"=>"main", "key"=>"1054B7A24BD6EC30", "key_server"=>"pgp.mit.edu", "include_deb"=>"true", "include_src"=>"false"}, "foreman"=>{"location"=>"http://deb.theforeman.org/", "release"=>"trusty", "repos"=>"1.7", "key_source"=>"http://deb.theforeman.org/pubkey.gpg", "include_deb"=>"true", "include_src"=>"false"}, "foreman-plugins"=>{"location"=>"http://deb.theforeman.org/", "release"=>"plugins", "repos"=>"1.7", "key_source"=>"http://deb.theforeman.org/pubkey.gpg", "include_deb"=>"true", "include_src"=>"false"}, "docker"=>{"location"=>"https://get.docker.com/ubuntu", "release"=>"docker", "repos"=>"main", "key"=>"D8576A8BA88D21E9", "key_server"=>"keyserver.ubuntu.com", "include_deb"=>"true", "include_src"=>"false"}} to yaml

THis is when using a yaml hash like:

ubuntu-main:
location: http://archive.ubuntu.com/ubuntu/
release: trusty
repos: main restricted universe multiverse
include_src: true
include_deb: true
ubuntu-security:
location: http://archive.ubuntu.com/ubuntu/
release: trusty-security
repos: main restricted universe multiverse
include_src: true
include_deb: true
ubuntu-updates:
location: http://archive.ubuntu.com/ubuntu/
release: trusty-updates
repos: main restricted universe multiverse
include_src: true
include_deb: true
puppetlabs:
location: http://apt.puppetlabs.com
repos: main
key: 1054B7A24BD6EC30
key_server: pgp.mit.edu
include_deb: true
include_src: false

Note how the yaml has become a ruby hash in the log output. Because `type_cast` doesn't have a return in the rescue, it returns "true" leading to your ENC yaml being incorrect.

I gave Chris this workaround:

--- a/app/services/classification/base.rb
+++ b/app/services/classification/base.rb
@@ -147,6 +147,7 @@ module Classification
       key.cast_validate_value(value)
     rescue TypeError
       logger.warn "Unable to type cast #{value} to #{key.key_type}" 
+      return value
     end

but there may be better options. Marek, thoughts?

#3 Updated by The Foreman Bot over 4 years ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/2228 added
  • Pull request deleted ()

#4 Updated by Dominic Cleal over 4 years ago

  • Legacy Backlogs Release (now unused) set to 36

#5 Updated by Marek Hulán over 4 years ago

  • Assignee set to Marek Hulán
  • Pull request https://github.com/theforeman/foreman/pull/2232 added
  • Pull request deleted (https://github.com/theforeman/foreman/pull/2228)

Here's my version of the fix https://github.com/theforeman/foreman/pull/2232, JSON and YAML parameters were automatically parsed (thanks ActiveRecord) so they appeared as Hashes, then we tried to convert hash to YAML rather then string to YAML (or JSON...)

#6 Updated by Marek Hulán over 4 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#7 Updated by Martin Hubert over 4 years ago

I am also affected by this problem. The patch does not solve the problem for global Smart Class Parameters. I was able to log a stack trace.

My quick-hack is to add a "return value" inside the rescue block around line 148 in app/services/classification/base.rb

#8 Updated by Dominic Cleal over 4 years ago

Martin Hubert wrote:

I am also affected by this problem. The patch does not solve the problem for global Smart Class Parameters. I was able to log a stack trace.

My quick-hack is to add a "return value" inside the rescue block around line 148 in app/services/classification/base.rb

Could I just check which patch you're using? The one from the linked pull request, or the one in comment 2?

#9 Updated by Martin Hubert over 4 years ago

I use patch from the linked pull request (2232).

It works for host parameters but not for global smart class parameters. To get this working I also must apply the workaround like suggested by Greg in comment 2.

#10 Updated by Marek Hulán over 4 years ago

Just to be sure, by "global smart class parameter" you actually mean Smart variable?

#11 Updated by Martin Hubert over 4 years ago

I mean "Smart Class Parameters"

Accessible by :

Configure -> Puppet classes -> <class> -> Smart Class Parameters

(see screenshot)

#12 Updated by Marek Hulán over 4 years ago

This works for me, but I was able to reproduce the behavior that returns "true" instead of that YAML in case there's no override and default value is used. Do you have some overrides?

#13 Updated by Marek Hulán over 4 years ago

  • Related to Bug #9787: Default values of smart class parameters can't be JSON or YAML added

#14 Updated by Dominic Cleal over 4 years ago

  • Related to Bug #8052: Smart class parameters with ERB snippets are broken for other that String types added

Also available in: Atom PDF