Project

General

Profile

Feature #14447

Add serialised cache of parsed manifests

Added by Dominic Cleal over 5 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Difficulty:
Triaged:
No
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

This tickets covers the idea of a third JSON-based parser mentioned at https://groups.google.com/d/msg/foreman-dev/Ijvodu1hJkE/Ssl83t35IAAJ and in https://github.com/theforeman/kafo_parsers/pull/13. However, instead of a JSON-based parser in kafo_parsers, adding a cache from a serialised file around the parser in Kafo itself would have a few benefits:

  • Kafo already had kafo-export-params, which I thought would be the perfect tool to extend to generate the YAML files. It already handles loading the configuration, output formats etc, so adding a new output format for the cache is trivial.
  • If extending kafo-export-params to write the cache, it made sense to keep the cache reader and writer in the same project so it's easy to keep in sync.
  • The cache location can be set in the Kafo configuration, which is hard to access from kafo_parsers right now.

This helps with support for Puppet AIO packages as when the cache is in use, there's no need to load the Puppet dependency in kafo_parsers.

This does delay the requirement for a puppet-strings parser, as we can generate the parser cache on the current Puppet 3 installation at build time and use it with a Puppet 4 AIO installation. It also means users won't have to install puppet-strings. We will still want a puppet-strings parser so we're not limited to building on Puppet 3.


Related issues

Related to Kafo - Bug #14683: Parser cache is configured by default, causing warning when it's missingClosed2016-04-18
Related to Kafo - Feature #14946: Add command line option to force use of parser cacheClosed2016-05-06
Related to Kafo - Feature #16230: Read multiple parser caches for a single configurationClosed2016-08-22
Has duplicate Kafo - Bug #7530: Speed up katello-installer --helpResolved2014-09-18
Blocks Foreman - Tracker #8447: Puppet 4 supportClosed2014-11-19

Blocks Installer - Feature #14453: Generate Kafo parser cache during buildClosed2016-04-04

Associated revisions

Revision 01858dac (diff)
Added by Dominic Cleal over 5 years ago

fixes #14447 - add cache around parser using YAML file

A cache of parsed modules can be configured with `:parser_cache_path` to
skip calls out to kafo_parsers when loading Puppet modules. This helps
when no parser is available for the current Puppet installation (e.g.
under Puppet 4 with AIO at the time of writing), and may provide a small
peformance benefit.

The cache can be generated with `kafo-export-params -f parsercache
--no-parser-cache`.

The cache is skipped if the mtime of the manifest is greater than the
mtime when the cache was generated, causing Kafo to use kafo_parsers as
normal.

Revision 2bbc79ea
Added by Marek Hulán over 5 years ago

Merge pull request #135 from domcleal/parser-cache

fixes #14447 - add cache around parser using serialised YAML file

History

#1 Updated by Dominic Cleal over 5 years ago

#2 Updated by Dominic Cleal over 5 years ago

  • Blocks Feature #14453: Generate Kafo parser cache during build added

#3 Updated by Dominic Cleal over 5 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#4 Updated by Dominic Cleal over 5 years ago

  • Related to Bug #14683: Parser cache is configured by default, causing warning when it's missing added

#5 Updated by Dominic Cleal over 5 years ago

  • Related to Feature #14946: Add command line option to force use of parser cache added

#6 Updated by Dominic Cleal over 5 years ago

  • Has duplicate Bug #7530: Speed up katello-installer --help added

#7 Updated by Adam Price about 5 years ago

  • Related to Feature #16230: Read multiple parser caches for a single configuration added

Also available in: Atom PDF