Bug #2226
closedpuppet proxy does not detect certain classes
Description
I ran the foreman-proxy importer against a large codebase of about 50 Puppet modules containing around 200 classes. About 20 classes failed to be discovered, and did not up in the list of available classes.
I added some printf logging to puppet_class.rb and determined that it was able to parse the AST but did not find whatever it was looking for. As a workaround, I added some fallback logic that will make an educated guess as to what the classname should be.
--- puppet_class.rb.ORIG 2013-02-18 18:06:54.000000000 -0500 +++ puppet_class.rb.NEW 2013-02-19 17:47:52.000000000 -0500 @@ -39,6 +39,17 @@ klasses << new(klass.namespace, params) end end + # + # WORKAROUND: if we don't find any classes by examining the AST, + # assume that there is a single class that matches the filename. + # + # Example: + # + # file .../modules/foo/manifests/bar.pp becomes class foo::bar + # + if klasses.empty? + klasses << new(filename.clone.gsub(/.*\/modules\//, '').gsub('/manifests/', '::').gsub(/.pp$/, '').gsub('::init', ''), {}) + end klasses rescue => e puts "Error while parsing #{filename}: #{e}"
See below for an example of a class that the foreman-proxy was unable to discover:
class apache2 { # barebones minimum of apache, just the groups and devel package # some boxes require just this, cassandra for example. realize ( Group["bronto"], User["bronto"], Group["tomcat"], User["tomcat"], Group["apache"], User["apache"], ) package { "httpd-devel": ensure => "installed", alias => "httpd-devel", } }
Here are the versions I am running:
- foreman-proxy-1.1stable-1.el5
- puppet 2.7.11
- CentOS 5.4
- ruby 1.8.6 (2009-08-04 patchlevel 383) [x86_64-linux]
Updated by Dominic Cleal about 12 years ago
- Category set to Puppet
- Status changed from New to Feedback
I'm not able to reproduce this using Puppet 2.7.11 and the manifest provided:
$ pry -Ilib -rproxy
[1] pry(main)> Puppet.version
=> "2.7.11"
[2] pry(main)> Proxy::Puppet::PuppetClass.scan_manifest File.read("/tmp/apache2.pp"), "apache2.pp"
=> [#<Proxy::Puppet::PuppetClass:0x7fce41ae3230 @klass="apache2", @params={}>]
(showing the apache2 class was found)
Any chance you can add some more print statements and provide the output?
puts ast.instantiate('').inspect
puts hostclasses.inspect
puts already_seen
Updated by Mark Heily almost 12 years ago
The failed import was actually caused by bug 2067. There were statements in site.pp to the effect of "import 'apache2'". When I removed the import() calls, the classes were imported correctly.
See below for the diff of the printf() logging I added.
[root@util-prod-004 puppet]# diff -u puppet_class.rb.ORIG puppet_class.rb --- puppet_class.rb.ORIG 2013-03-13 22:03:26.618426862 -0400 +++ puppet_class.rb 2013-03-13 22:24:39.820410303 -0400 @@ -4,6 +4,10 @@ class PuppetClass + require 'logger' + + $pclogger = Logger.new('/tmp/puppet-class-import.log') + class << self # scans a given directory and its sub directory for puppet classes # returns an array of PuppetClass objects. @@ -11,7 +15,9 @@ # Get a Puppet Parser to parse the manifest source Initializer.load parser = Puppet::Parser::Parser.new Puppet::Node::Environment.new + $pclogger.info "scanning all files under directory = #{directory}" Dir.glob("#{directory}/*/manifests/**/*.pp").map do |filename| + $pclogger.info "scanning #{filename}" scan_manifest File.read(filename), filename, parser end.compact.flatten end @@ -39,6 +45,13 @@ klasses << new(klass.namespace, params) end end + if klasses.empty? + $pclogger.warn "unable to parse #{filename}" + $pclogger.warn manifest + $pclogger.warn ast.instantiate('').inspect + $pclogger.warn hostclasses.inspect + $pclogger.warn already_seen + end klasses rescue => e puts "Error while parsing #{filename}: #{e}"
Updated by Dominic Cleal almost 12 years ago
- Status changed from Feedback to Closed
Thanks for finding that Mark! It's a one line fix if you want to test it locally:
https://github.com/theforeman/smart-proxy/pull/70
(closing as dupe of #2067)