Project

General

Profile

Actions

Bug #2226

closed

puppet proxy does not detect certain classes

Added by Mark Heily about 11 years ago. Updated about 11 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Puppet
Target version:
-
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

I ran the foreman-proxy importer against a large codebase of about 50 Puppet modules containing around 200 classes. About 20 classes failed to be discovered, and did not up in the list of available classes.

I added some printf logging to puppet_class.rb and determined that it was able to parse the AST but did not find whatever it was looking for. As a workaround, I added some fallback logic that will make an educated guess as to what the classname should be.

--- puppet_class.rb.ORIG    2013-02-18 18:06:54.000000000 -0500
+++ puppet_class.rb.NEW    2013-02-19 17:47:52.000000000 -0500
@@ -39,6 +39,17 @@
             klasses << new(klass.namespace, params)
           end
         end
+    #
+    # WORKAROUND: if we don't find any classes by examining the AST,
+    # assume that there is a single class that matches the filename.
+     #
+    # Example: 
+    #   
+    #   file .../modules/foo/manifests/bar.pp becomes class foo::bar
+    #
+        if klasses.empty?
+          klasses << new(filename.clone.gsub(/.*\/modules\//, '').gsub('/manifests/', '::').gsub(/.pp$/, '').gsub('::init', ''), {})
+        end
         klasses
       rescue => e
         puts "Error while parsing #{filename}: #{e}" 

See below for an example of a class that the foreman-proxy was unable to discover:

class apache2 {
  # barebones minimum of apache, just the groups and devel package
  # some boxes require just this, cassandra for example.
  realize (
    Group["bronto"],
    User["bronto"],
    Group["tomcat"],
    User["tomcat"],
    Group["apache"],
    User["apache"],
  )
  package { "httpd-devel":
    ensure  => "installed",
    alias   => "httpd-devel",
  }
}

Here are the versions I am running:

  • foreman-proxy-1.1stable-1.el5
  • puppet 2.7.11
  • CentOS 5.4
  • ruby 1.8.6 (2009-08-04 patchlevel 383) [x86_64-linux]

Related issues 1 (0 open1 closed)

Is duplicate of Smart Proxy - Bug #2067: puppet:import:puppet_classes silently ignores classes when the .pp contains an "import" statementClosedDominic Cleal12/20/2012Actions
Actions #1

Updated by Dominic Cleal about 11 years ago

  • Category set to Puppet
  • Status changed from New to Feedback

I'm not able to reproduce this using Puppet 2.7.11 and the manifest provided:

$ pry -Ilib -rproxy
[1] pry(main)> Puppet.version
=> "2.7.11"
[2] pry(main)> Proxy::Puppet::PuppetClass.scan_manifest File.read("/tmp/apache2.pp"), "apache2.pp"
=> [#&lt;Proxy::Puppet::PuppetClass:0x7fce41ae3230 @klass=&quot;apache2&quot;, @params={}&gt;]

(showing the apache2 class was found)

Any chance you can add some more print statements and provide the output?

puts ast.instantiate('').inspect
puts hostclasses.inspect
puts already_seen
Actions #2

Updated by Mark Heily about 11 years ago

The failed import was actually caused by bug 2067. There were statements in site.pp to the effect of "import 'apache2'". When I removed the import() calls, the classes were imported correctly.

See below for the diff of the printf() logging I added.

[root@util-prod-004 puppet]# diff -u puppet_class.rb.ORIG puppet_class.rb
--- puppet_class.rb.ORIG    2013-03-13 22:03:26.618426862 -0400
+++ puppet_class.rb    2013-03-13 22:24:39.820410303 -0400
@@ -4,6 +4,10 @@

   class PuppetClass

+   require 'logger'
+
+   $pclogger = Logger.new('/tmp/puppet-class-import.log')
+
     class << self
       # scans a given directory and its sub directory for puppet classes
       # returns an array of PuppetClass objects.
@@ -11,7 +15,9 @@
         # Get a Puppet Parser to parse the manifest source
         Initializer.load
         parser = Puppet::Parser::Parser.new Puppet::Node::Environment.new
+        $pclogger.info "scanning all files under directory = #{directory}" 
         Dir.glob("#{directory}/*/manifests/**/*.pp").map do |filename|
+          $pclogger.info "scanning #{filename}" 
           scan_manifest File.read(filename), filename, parser
         end.compact.flatten
       end
@@ -39,6 +45,13 @@
             klasses << new(klass.namespace, params)
           end
         end
+        if klasses.empty?
+          $pclogger.warn "unable to parse #{filename}" 
+          $pclogger.warn manifest
+          $pclogger.warn ast.instantiate('').inspect
+      $pclogger.warn hostclasses.inspect
+      $pclogger.warn already_seen
+        end
         klasses
       rescue => e
         puts "Error while parsing #{filename}: #{e}" 

Actions #3

Updated by Dominic Cleal about 11 years ago

  • Status changed from Feedback to Closed

Thanks for finding that Mark! It's a one line fix if you want to test it locally:

https://github.com/theforeman/smart-proxy/pull/70

(closing as dupe of #2067)

Actions

Also available in: Atom PDF