Bug #2226
closedpuppet proxy does not detect certain classes
Description
I ran the foreman-proxy importer against a large codebase of about 50 Puppet modules containing around 200 classes. About 20 classes failed to be discovered, and did not up in the list of available classes.
I added some printf logging to puppet_class.rb and determined that it was able to parse the AST but did not find whatever it was looking for. As a workaround, I added some fallback logic that will make an educated guess as to what the classname should be.
--- puppet_class.rb.ORIG 2013-02-18 18:06:54.000000000 -0500
+++ puppet_class.rb.NEW 2013-02-19 17:47:52.000000000 -0500
@@ -39,6 +39,17 @@
klasses << new(klass.namespace, params)
end
end
+ #
+ # WORKAROUND: if we don't find any classes by examining the AST,
+ # assume that there is a single class that matches the filename.
+ #
+ # Example:
+ #
+ # file .../modules/foo/manifests/bar.pp becomes class foo::bar
+ #
+ if klasses.empty?
+ klasses << new(filename.clone.gsub(/.*\/modules\//, '').gsub('/manifests/', '::').gsub(/.pp$/, '').gsub('::init', ''), {})
+ end
klasses
rescue => e
puts "Error while parsing #{filename}: #{e}"
See below for an example of a class that the foreman-proxy was unable to discover:
class apache2 {
# barebones minimum of apache, just the groups and devel package
# some boxes require just this, cassandra for example.
realize (
Group["bronto"],
User["bronto"],
Group["tomcat"],
User["tomcat"],
Group["apache"],
User["apache"],
)
package { "httpd-devel":
ensure => "installed",
alias => "httpd-devel",
}
}
Here are the versions I am running:
- foreman-proxy-1.1stable-1.el5
- puppet 2.7.11
- CentOS 5.4
- ruby 1.8.6 (2009-08-04 patchlevel 383) [x86_64-linux]
Updated by Dominic Cleal over 13 years ago
- Category set to Puppet
- Status changed from New to Feedback
I'm not able to reproduce this using Puppet 2.7.11 and the manifest provided:
$ pry -Ilib -rproxy
[1] pry(main)> Puppet.version
=> "2.7.11"
[2] pry(main)> Proxy::Puppet::PuppetClass.scan_manifest File.read("/tmp/apache2.pp"), "apache2.pp"
=> [#<Proxy::Puppet::PuppetClass:0x7fce41ae3230 @klass="apache2", @params={}>]
(showing the apache2 class was found)
Any chance you can add some more print statements and provide the output?
puts ast.instantiate('').inspect
puts hostclasses.inspect
puts already_seen
Updated by Mark Heily over 13 years ago
The failed import was actually caused by bug 2067. There were statements in site.pp to the effect of "import 'apache2'". When I removed the import() calls, the classes were imported correctly.
See below for the diff of the printf() logging I added.
[root@util-prod-004 puppet]# diff -u puppet_class.rb.ORIG puppet_class.rb
--- puppet_class.rb.ORIG 2013-03-13 22:03:26.618426862 -0400
+++ puppet_class.rb 2013-03-13 22:24:39.820410303 -0400
@@ -4,6 +4,10 @@
class PuppetClass
+ require 'logger'
+
+ $pclogger = Logger.new('/tmp/puppet-class-import.log')
+
class << self
# scans a given directory and its sub directory for puppet classes
# returns an array of PuppetClass objects.
@@ -11,7 +15,9 @@
# Get a Puppet Parser to parse the manifest source
Initializer.load
parser = Puppet::Parser::Parser.new Puppet::Node::Environment.new
+ $pclogger.info "scanning all files under directory = #{directory}"
Dir.glob("#{directory}/*/manifests/**/*.pp").map do |filename|
+ $pclogger.info "scanning #{filename}"
scan_manifest File.read(filename), filename, parser
end.compact.flatten
end
@@ -39,6 +45,13 @@
klasses << new(klass.namespace, params)
end
end
+ if klasses.empty?
+ $pclogger.warn "unable to parse #{filename}"
+ $pclogger.warn manifest
+ $pclogger.warn ast.instantiate('').inspect
+ $pclogger.warn hostclasses.inspect
+ $pclogger.warn already_seen
+ end
klasses
rescue => e
puts "Error while parsing #{filename}: #{e}"
Updated by Dominic Cleal over 13 years ago
- Status changed from Feedback to Closed
Thanks for finding that Mark! It's a one line fix if you want to test it locally:
https://github.com/theforeman/smart-proxy/pull/70
(closing as dupe of #2067)