Feature #335

Host facts not displayed in the /hosts/ view

Added by Brian Gallew about 4 years ago. Updated over 1 year ago.

Status:ClosedStart date:07/01/2010
Priority:NormalDue date:
Assigned To:Ohad Levy% Done:

100%

Category:Web Interface
Target version:0.2
Difficulty: Bugzilla link:
Found in release: Pull request:
Story points-
Velocity based estimate-

Description

When using Foreman strictly as a reporting tool, the individual host view doesn't display the values for Domain, Puppet Environment, Host Architecture, Host Operating System, or Host Group, even though facts are available for all of them. I'll see if I can put together a patch that will only pull the facts in if the system isn't otherwise configured.

hosts-view.png - Snapshot of display (9.22 KB) Brian Gallew, 07/01/2010 06:40 PM

facts_as_necessary.diff Magnifier (1.93 KB) Brian Gallew, 07/01/2010 09:34 PM

foreman_info.patch Magnifier - Patch adding new behavior. (1.88 KB) Brian Gallew, 10/25/2010 11:14 PM

foreman_info.patch Magnifier - The *right* version of the patch (1.89 KB) Brian Gallew, 10/26/2010 12:30 AM

Associated revisions

Revision 38dbbddf
Added by Ohad Levy over 3 years ago

fixes #335 - this patch will auto import the facts (making the rake task or import_facts script redundent).

this patch introduce a new settings options :using_storeconfigs:
while we try to automaticily detect if we run storeconfigs, its not always possible
when foreman is not running on the same host as the puppetmaster.

History

#1 Updated by Ohad Levy about 4 years ago

  • Status changed from New to Feedback
  • Target version deleted (0.1-6)

this is not really a bug, but rather that foreman has not been configured to fetch the facts from puppet.

see the rake fact importer tasks (or populate task if you are using storeconfigs) or the import_facts scripts.

#2 Updated by Brian Gallew about 4 years ago

Attached is a diff that will use the values from host.fv() if the appropriate values are not set for the host (e.g. host.domain). This patch is against 0.1-5.

#3 Updated by Brian Gallew about 4 years ago

Ohad Levy wrote:

this is not really a bug, but rather that foreman has not been configured to fetch the facts from puppet.

see the rake fact importer tasks (or populate task if you are using storeconfigs) or the import_facts scripts.

I'm using storeconfigs and I've run the importer task. I've never seen any documentation pointing to a populate task. Reading the rakefile, that doesn't touch any of the fields that Foreman generally manages (domain, arch, os, model, and environment). And again, if you aren't using Foreman to manage that data, then nothing is displayed in either the _overview or _minilist. The diff I've attached will only affect the display when the associated host.whatever field is empty, and doesn't touch the DB.

#4 Updated by Ohad Levy almost 4 years ago

I'm willing to accept the patch, but it seems like a lot of repetition between the fact importer code...

Do you think it might be better to update the docs so other people wont have the same problem as you do? (IMHO it would also be faster not to query the facts directly).

#5 Updated by Brian Gallew almost 4 years ago

I have to admit that 9 lines of code (replacing 5 existing lines) doesn't seem like a lot of repetition to me. OTOH, if the fact importer code is already supposed to be importing data from the hosts, maybe it makes sense for me to look there and patch that to populate the Foreman tables when there is no existing data? If that's what it would take to make an acceptable patch, I can work on it.

#6 Updated by Ohad Levy almost 4 years ago

Brian Gallew wrote:

I have to admit that 9 lines of code (replacing 5 existing lines) doesn't seem like a lot of repetition to me. OTOH, if the fact importer code is already supposed to be importing data from the hosts, maybe it makes sense for me to look there and patch that to populate the Foreman tables when there is no existing data? If that's what it would take to make an acceptable patch, I can work on it.

Sorry for the late response.
Yes, thats what I meant, it makes far more sense to make sure we import the data correctly.
(its also important for other usage cases - so imho - it would be better).

thanks

#7 Updated by Brian Gallew almost 4 years ago

OK, I'll get this refactored and submit a new patch. It'll take me a few days, probably.

#8 Updated by Ohad Levy almost 4 years ago

I think this issue is resolved after the fix for #336, can you confirm?

#9 Updated by Brian Gallew almost 4 years ago

Assuming this fix is in the latest snapshots, then no. However, it occurs
to me that I may not have been specific enough when I created this ticket.
I don't run a manual yaml import. Rather, I expect that the clients which
are reporting to Foreman via the "report=foreman" line in puppet.conf will
have correct information if for these fields if they are not explicitly
pre-populated.

#10 Updated by Ohad Levy almost 4 years ago

Brian Gallew wrote:

Assuming this fix is in the latest snapshots, then no. However, it occurs
to me that I may not have been specific enough when I created this ticket.
I don't run a manual yaml import. Rather, I expect that the clients which
are reporting to Foreman via the "report=foreman" line in puppet.conf will
have correct information if for these fields if they are not explicitly
pre-populated.

The report information does not contain any facts...
Foreman can collect facts from the filesystem (via ~puppet/yaml/facts) or via storeconfigs (if sharing the same db).

then based on the facts, additional information is extracted (like os) to populate additional tables in the db...

So I'm not really sure I understand what you want to solve...

#11 Updated by Ohad Levy almost 4 years ago

  • Target version deleted (0.1-6)

#12 Updated by Brian Gallew over 3 years ago

I've got a brand-new patch for you. The behavioral changes are all controlled by a new settings.yaml value ":reportonly: true". If :reportonly is set to any other value (or missing) then Foreman behaves as before. If set to "true", then every time a client reports, Foreman will update the environment, OS, and model with values derived from Puppet facts.

You should discard the previous patch.

#13 Updated by Brian Gallew over 3 years ago

Having multiple copies of a patch file laying around occasionally causes problems. Sorry.

#14 Updated by Ohad Levy over 3 years ago

  • Assigned To changed from Ohad Levy to Brian Gallew

Brian Gallew wrote:

Having multiple copies of a patch file laying around occasionally causes problems. Sorry.

I'm not sure I understand why you do it in the report model, the data that you process are facts, which are imported (if using storeconfigs), using the Host.importFacts method.

that means,that if a report came before its facts, then there would be no data to import from.

if I understand correctly, what you are asking, is basically item 1 in http://theforeman.org/issues/284#note-4

#15 Updated by Brian Gallew over 3 years ago

Except that, without my patch, Foreman is not pulling in the host facts when using storeconfig unless you have a separate rake task. If I delete hosts from my Foreman instance and change the config setting, then the hosts, when displayed in the "Hosts" view as well as the individual host pages, have no information on Environment, Model, or OS.

#16 Updated by Ohad Levy over 3 years ago

Brian Gallew wrote:

Except that, without my patch, Foreman is not pulling in the host facts when using storeconfig unless you have a separate rake task. If I delete hosts from my Foreman instance and change the config setting, then the hosts, when displayed in the "Hosts" view as well as the individual host pages, have no information on Environment, Model, or OS.

Why dont you run the cronjob as described in Puppet_Facts?

there are also examples of it in the foreman puppet module

#17 Updated by Brian Gallew over 3 years ago

If you advocate running the cron job, then why waste time supporting a web reporting interface? The whole point of web reporting is, "update the database/GUI with current information". While Model and OS won't change very often, my environments frequently do. Why shouldn't the web reporting interface update all the same information as the cron job? Does the value of the information change depending on the source?

#18 Updated by Ohad Levy over 3 years ago

Brian Gallew wrote:

If you advocate running the cron job, then why waste time supporting a web reporting interface? The whole point of web reporting is, "update the database/GUI with current information". While Model and OS won't change very often, my environments frequently do. Why shouldn't the web reporting interface update all the same information as the cron job? Does the value of the information change depending on the source?

the main difference is:
Puppet reporting interface is plugable, i.e when a report is sent to puppet, we can simply forward it to foreman and process it,
Puppet facts however don't have such an interface, and we don't know when a new fact is being collected.

at the moment, we support two different approches, based on weather or not you use storeconfigs.

using storeconfigs

when using storeconfigs, puppet itself will update the database, making foreman completly unaware of it.
for this problem, we've created the populate rake task, which basicilly reimport the facts into the fields foreman is interested in, now that I think of it a bit more, we could have a setting option letting us know if you use storeconfigs or not, and upon new report, also import the fields from the db.

not using storeconfigs

here we have a simple script which runs in cron, checking the time stamp of the fact yaml files, and send new ones every run

#19 Updated by Ohad Levy over 3 years ago

  • Status changed from Feedback to Assigned
  • Assigned To changed from Brian Gallew to Ohad Levy

Does the following sounds like an acceptable solution:

if using Storeconfigs, then auto import the host facts once a report comes in?

#20 Updated by Ohad Levy over 3 years ago

  • Status changed from Assigned to Ready For Testing
  • % Done changed from 0 to 100

#21 Updated by Brian Gallew over 3 years ago

Thanks, that'll be very helpful.

#22 Updated by Ohad Levy over 3 years ago

Brian Gallew wrote:

Thanks, that'll be very helpful.

Can you confirm that its behaving as you expect? - thanks

#23 Updated by Ohad Levy over 3 years ago

  • Status changed from Ready For Testing to Closed

Also available in: Atom PDF