Project

General

Profile

Feature #4528

Support Facter 2 structured facts

Added by Greg Sutcliffe over 5 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Facts
Target version:
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Facter 2 is at RC1, and while it has no structered facts by default, it will have in the future, and people will start writing their own probably even sooner.

I quickly looked into this using the foreman_chef plugin. I came up with this:

1. Add 'plainify' and 'get_key' to node.rb from chef_foreman_handler, and change 'build_body' to call:

{'facts' => plainify(puppet_facts['values'].merge({:_type => 'foreman_chef'})).flatten.inject(&:merge), 'name' => hostname, 'certname' => certname}

2. Add foreman_chef plugin to Foreman
3. Fix https://github.com/theforeman/foreman/blob/develop/app/services/fact_importer.rb#L79 to not drop Integers
4. (optional) Handle changes of fact type. Currently if you upload "foo: bar" and then upload "foo: { "bar" => "quux" }" it won't display properly due to the old string-type fact in the db.

We'll probably want to move the hash-handling code out of foreman_chef and into core. I'd also consider investigating the way we compress and uncompress the facts hash with '::' vs just sending/parsing the whole hash directly.


Related issues

Related to Foreman - Feature #2293: Support structured factsDuplicate2013-03-08
Related to Foreman - Feature #3339: Support for nested facts and specialised pluginsClosed2013-10-17
Related to Foreman - Feature #7525: Report volume sizes via facterClosed2014-09-18
Related to Discovery - Feature #3860: Implement fully automated provisioningClosed2013-12-12
Related to Foreman - Tracker #8447: Puppet 4 supportClosed2014-11-19

Related to Foreman - Feature #15522: Support for array/multi-valued factsNew2016-06-27
Related to Salt - Refactor #15591: Use new structured facts parserNew2016-07-06
Related to Foreman - Bug #16576: Structured facts importer is extremely slowClosed2016-09-15
Has duplicate Foreman - Bug #6414: structured facts not displayed in foremanDuplicate2014-06-27
Has duplicate Foreman - Feature #12703: Layout aggregated fact value display formatDuplicate2015-12-04

Associated revisions

Revision c696f740 (diff)
Added by Dominic Cleal about 3 years ago

fixes #4528 - import structured Puppet facts

A new structured fact importer converts nested hashes into the
FactName::SEPARATOR layout used for internal structured fact storage and
extends the regular fact importer to create the hierarchy of FactNames.
This can be used by other plugins requiring structured storage. It does
not support storage of arrays beyond current conversion to strings.

History

#1 Updated by Dominic Cleal over 5 years ago

  • Subject changed from Support structured facts to Support Facter 2 structured facts

#2 Updated by Dominic Cleal over 5 years ago

#3 Updated by Dominic Cleal over 5 years ago

  • Related to Feature #3339: Support for nested facts and specialised plugins added

#4 Updated by Greg Sutcliffe over 5 years ago

  • Description updated (diff)

#5 Updated by Dominic Cleal about 5 years ago

  • Has duplicate Bug #6414: structured facts not displayed in foreman added

#6 Updated by Stephen Benjamin about 5 years ago

ForemanChef's code "just works" for Salt's structured facts as well (after plainifying). It would be awesome to have a nested fact importer in Core that could just take the hash.

#7 Updated by Marek Hulán about 5 years ago

Let's merge it into core :-) Note that facts should still be scoped by name so we should add ForemanSalt::FactImporter inheriting from ForemanChef::FactImporter or ideally extract the generic part to some new NestedFactImporter class and let inherit both from it. Important methods that would remain importer specific: fact_name_class and support_background. Importer from ForemanChef is also lacking tests at the moment so it should be covered first.

#8 Updated by Stephen Benjamin about 5 years ago

  • Bugzilla link set to 1132402

#9 Updated by Lukas Zapletal almost 5 years ago

#10 Updated by Lukas Zapletal almost 5 years ago

Guys I need to get some integers in (discovery, hard disk volume sizes), can I post a patch to simply convert the number to string? See the referenced issue.

#11 Updated by Lukas Zapletal almost 5 years ago

For the record, I filed a PR to convert non-string values to strings so it prevents from failure. We might want to do something smarter than this in this ticket, but since Staypuft is blocked by this I propose this solution: https://github.com/theforeman/foreman_discovery/pull/90

#12 Updated by Lukas Zapletal almost 5 years ago

  • Related to Feature #3860: Implement fully automated provisioning added

#13 Updated by Lukas Zapletal almost 5 years ago

Bumping up the priority a bit, if this is implemented, our Discovery rule engine is much more powerful.

#14 Updated by Joshua Hoblitt almost 5 years ago

I just smacked into this in 1.6.3. Not even string structured facts are queryable via the gui.

#15 Updated by Jeff Sparrow over 4 years ago

This has been open for sometime and puppet has really pushed to using structured data facts more often, within their future parser it makes things much easier. Hope this is included very soon as hashes and arrays are fairly common and should be supported

#16 Updated by Sven Milinski about 4 years ago

Hello,

can anyone tell me when Foreman is able to manage facter 2 array/hash values ?
Is it on the roadmap ?

Would be nice if someone can update this issue

Thanks

#17 Updated by Jeff Sparrow about 4 years ago

Any update? Not having structured facts in foreman is getting to be a big hassle to get around. Would be nice to get an update, its been almost a year. Thanks.

#18 Updated by Dominic Cleal almost 4 years ago

  • Has duplicate Feature #12703: Layout aggregated fact value display format added

#19 Updated by Michael Moll over 3 years ago

#20 Updated by Maxim Burgerhout over 3 years ago

Doesn't the foreman_ansible implementation by @dLobatog have nice structured facts already? It might be possible to use that same code?

#21 Updated by Daniel Lobato Garcia over 3 years ago

Yup - it's also partly taken from Salt so hopefully we can provide this from core to all plugins.

#22 Updated by Dominic Cleal about 3 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal

#23 Updated by Dominic Cleal about 3 years ago

  • Related to Feature #15522: Support for array/multi-valued facts added

#24 Updated by Dominic Cleal about 3 years ago

Tickets relating to handling of non-structured, non-string facts (ints, bools) which are mentioned above:

  • #7525 changed the importer to handle non-string facts by converting their values to strings, this helps notably with integer facts.
  • #11150 changed fact searching to support integer comparisons, since Foreman stores integers as strings internally.
  • I don't propose changing the storage of non-string facts in Foreman, it's sufficient to store them as strings and convert them when used (e.g. when searching).

Tickets relating to handling of array values:

  • #15522 filed to enable storage of multiple fact values for arrays, instead of storing them as a string (e.g. ["a", "b"])

Tickets relating to handling of structured/nested/hash facts:

  • #3339 added support for structured facts - that is, facts whose value is a hash of more facts
  • This ticket will be used only to update the Puppet fact importer to store structured facts/hashes of facts.

#25 Updated by The Foreman Bot about 3 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/3613 added

#26 Updated by Stephen Benjamin about 3 years ago

#27 Updated by Dominic Cleal about 3 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#28 Updated by Dominic Cleal about 3 years ago

  • Legacy Backlogs Release (now unused) set to 160

#29 Updated by Dominic Cleal about 3 years ago

  • Related to Bug #16576: Structured facts importer is extremely slow added

Also available in: Atom PDF