Project

General

Profile

Feature #4528

Support Facter 2 structured facts

Added by Greg Sutcliffe about 9 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Dominic Cleal
Category:
Facts
Target version:
1.13.0
Difficulty:
Triaged:
Bugzilla link:
1132402
Pull request:
https://github.com/theforeman/foreman/pull/3613
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Facter 2 is at RC1, and while it has no structered facts by default, it will have in the future, and people will start writing their own probably even sooner.

I quickly looked into this using the foreman_chef plugin. I came up with this:

1. Add 'plainify' and 'get_key' to node.rb from chef_foreman_handler, and change 'build_body' to call:

{'facts' => plainify(puppet_facts['values'].merge({:_type => 'foreman_chef'})).flatten.inject(&:merge), 'name' => hostname, 'certname' => certname}

2. Add foreman_chef plugin to Foreman
3. Fix https://github.com/theforeman/foreman/blob/develop/app/services/fact_importer.rb#L79 to not drop Integers
4. (optional) Handle changes of fact type. Currently if you upload "foo: bar" and then upload "foo: { "bar" => "quux" }" it won't display properly due to the old string-type fact in the db.

We'll probably want to move the hash-handling code out of foreman_chef and into core. I'd also consider investigating the way we compress and uncompress the facts hash with '::' vs just sending/parsing the whole hash directly.

Related issues

Related to Foreman - Feature #2293: Support structured factsDuplicate2013-03-08
Related to Foreman - Feature #3339: Support for nested facts and specialised pluginsClosed2013-10-17
Related to Foreman - Feature #7525: Report volume sizes via facterClosed2014-09-18
Related to Discovery - Feature #3860: Implement fully automated provisioningClosed2013-12-12
Related to Foreman - Tracker #8447: Puppet 4 supportClosed2014-11-19

Related to Foreman - Feature #15522: Support for array/multi-valued factsNew2016-06-27
Related to Salt - Refactor #15591: Use new structured facts parserNew2016-07-06
Related to Foreman - Bug #16576: Structured facts importer is extremely slowClosed2016-09-15
Has duplicate Foreman - Bug #6414: structured facts not displayed in foremanDuplicate2014-06-27
Has duplicate Foreman - Feature #12703: Layout aggregated fact value display formatDuplicate2015-12-04

Associated revisions

Revision c696f740 (diff)
Added by Dominic Cleal over 6 years ago

fixes #4528 - import structured Puppet facts

A new structured fact importer converts nested hashes into the
FactName::SEPARATOR layout used for internal structured fact storage and
extends the regular fact importer to create the hierarchy of FactNames.
This can be used by other plugins requiring structured storage. It does
not support storage of arrays beyond current conversion to strings.

History

#1 Updated by Dominic Cleal about 9 years ago

  • Subject changed from Support structured facts to Support Facter 2 structured facts

#2 Updated by Dominic Cleal about 9 years ago

#3 Updated by Dominic Cleal about 9 years ago

  • Related to Feature #3339: Support for nested facts and specialised plugins added

#4 Updated by Greg Sutcliffe about 9 years ago

  • Description updated (diff)

#5 Updated by Dominic Cleal almost 9 years ago

  • Has duplicate Bug #6414: structured facts not displayed in foreman added

#6 Updated by Stephen Benjamin over 8 years ago

ForemanChef's code "just works" for Salt's structured facts as well (after plainifying). It would be awesome to have a nested fact importer in Core that could just take the hash.

#7 Updated by Marek Hulán over 8 years ago

Let's merge it into core :-) Note that facts should still be scoped by name so we should add ForemanSalt::FactImporter inheriting from ForemanChef::FactImporter or ideally extract the generic part to some new NestedFactImporter class and let inherit both from it. Important methods that would remain importer specific: fact_name_class and support_background. Importer from ForemanChef is also lacking tests at the moment so it should be covered first.

#8 Updated by Stephen Benjamin over 8 years ago

  • Bugzilla link set to 1132402

#9 Updated by Lukas Zapletal over 8 years ago

#10 Updated by Lukas Zapletal over 8 years ago

Guys I need to get some integers in (discovery, hard disk volume sizes), can I post a patch to simply convert the number to string? See the referenced issue.

#11 Updated by Lukas Zapletal over 8 years ago

For the record, I filed a PR to convert non-string values to strings so it prevents from failure. We might want to do something smarter than this in this ticket, but since Staypuft is blocked by this I propose this solution: https://github.com/theforeman/foreman_discovery/pull/90

#12 Updated by Lukas Zapletal over 8 years ago

  • Related to Feature #3860: Implement fully automated provisioning added

#13 Updated by Lukas Zapletal over 8 years ago

Bumping up the priority a bit, if this is implemented, our Discovery rule engine is much more powerful.

#14 Updated by Joshua Hoblitt over 8 years ago

I just smacked into this in 1.6.3. Not even string structured facts are queryable via the gui.

#15 Updated by Jeff Sparrow almost 8 years ago

This has been open for sometime and puppet has really pushed to using structured data facts more often, within their future parser it makes things much easier. Hope this is included very soon as hashes and arrays are fairly common and should be supported

#16 Updated by Sven Milinski over 7 years ago

Hello,

can anyone tell me when Foreman is able to manage facter 2 array/hash values ?
Is it on the roadmap ?

Would be nice if someone can update this issue

Thanks

#17 Updated by Jeff Sparrow over 7 years ago

Any update? Not having structured facts in foreman is getting to be a big hassle to get around. Would be nice to get an update, its been almost a year. Thanks.

#18 Updated by Dominic Cleal over 7 years ago

  • Has duplicate Feature #12703: Layout aggregated fact value display format added

#19 Updated by Anonymous about 7 years ago

#20 Updated by Maxim Burgerhout almost 7 years ago

Doesn't the foreman_ansible implementation by @dLobatog have nice structured facts already? It might be possible to use that same code?

#21 Updated by Daniel Lobato Garcia almost 7 years ago

Yup - it's also partly taken from Salt so hopefully we can provide this from core to all plugins.

#22 Updated by Dominic Cleal almost 7 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal

#23 Updated by Dominic Cleal almost 7 years ago

  • Related to Feature #15522: Support for array/multi-valued facts added

#24 Updated by Dominic Cleal almost 7 years ago

Tickets relating to handling of non-structured, non-string facts (ints, bools) which are mentioned above:

  • #7525 changed the importer to handle non-string facts by converting their values to strings, this helps notably with integer facts.
  • #11150 changed fact searching to support integer comparisons, since Foreman stores integers as strings internally.
  • I don't propose changing the storage of non-string facts in Foreman, it's sufficient to store them as strings and convert them when used (e.g. when searching).

Tickets relating to handling of array values:

  • #15522 filed to enable storage of multiple fact values for arrays, instead of storing them as a string (e.g. ["a", "b"])

Tickets relating to handling of structured/nested/hash facts:

  • #3339 added support for structured facts - that is, facts whose value is a hash of more facts
  • This ticket will be used only to update the Puppet fact importer to store structured facts/hashes of facts.

#25 Updated by The Foreman Bot almost 7 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/3613 added

#26 Updated by Stephen Benjamin over 6 years ago

#27 Updated by Dominic Cleal over 6 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#28 Updated by Dominic Cleal over 6 years ago

  • Legacy Backlogs Release (now unused) set to 160

#29 Updated by Dominic Cleal over 6 years ago

  • Related to Bug #16576: Structured facts importer is extremely slow added

Also available in: Atom PDF