Project

General

Profile

Feature #1906

Add Trends Page to track changes in Fact host counts

Added by Robert Birnie almost 10 years ago. Updated almost 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Inventory
Target version:
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

We currently have a page to track historical trends in our infrastructure but it's datasource is going the way of the Dodo, so I worked to recreate this information in Foreman.

Goal: Be able to track historical host count for each Environment, Model, and Operatingsystem. Able to add any puppet Fact to the Trends page and start collecting stats. For example this will let a user view the rate that a now unsupported OS is being removed from the infrastructure and how quickly a new OS is being adopted.

My current code progress is here:

[[https://github.com/rbirnie/foreman/commit/e4b4b4fa0d8a934ac578821e9c2a1fc1a96474c7]]

Outstanding issues:
  • Would be better if Trends ignored hosts which are out of sync or have never reported. Host counts are coming through FactValue though so not sure how to pair that with Host.recent data.
  • I can't seem to add the Trends button to the top. Added a line in app/views/home/_topbar.rhtml which I thought would do it but no bueno.
  • Pie charts for Environment, Model, and Operatingsystem don't work - they are passing their ID code as Facts and will show the Pie Chart for that fact_name_id. Rather than their respective models.
Setup:
  • There is a new rake task that needs a crontab entry, this guy will add TrendCounter items for each item tracked. I think hourly is best, up to user though /foreman/rake trends:counter

Related issues

Related to Foreman - Feature #924: Tread reporting for facts and historical dataDuplicate2011-05-20

Associated revisions

Revision 10834815 (diff)
Added by Ohad Levy almost 10 years ago

refs #1906 - checked in wrong migration file

History

#1 Updated by Robert Birnie almost 10 years ago

Also as a note - I'm more of a systems guy than a rails programmer. So if I'm doing something wrong let me know. =)

#2 Updated by Corey Osman almost 10 years ago

This is very similar to a ticket I opened. http://theforeman.org/issues/924

We should combine both of these tickets to a trends feature.

I am not sure about the crontab rake task though. Is it possible to just build the task inside the code and have it executed when the query is performed?

#3 Updated by Robert Birnie almost 10 years ago

I thought about that, issue is that then the granularity is completely random. If you don't load the page in a year then you'd have no idea what happened in that year. It'd be totally dependent on how obsessed you were at looking at stats. I could maybe tie it to Reports or host updates (maybe Audit area?).

#4 Updated by Ohad Levy almost 10 years ago

Robert Birnie wrote:

I thought about that, issue is that then the granularity is completely random. If you don't load the page in a year then you'd have no idea what happened in that year. It'd be totally dependent on how obsessed you were at looking at stats. I could maybe tie it to Reports or host updates (maybe Audit area?).

we could run it after every major object changes, (just like audit) or alternativly feed from the audit log instead of each attribute directly (afair, audit has a way to rebuild the old object as it was before or something similar).

#5 Updated by Ohad Levy almost 10 years ago

on second thought, we don't really use the audit on facts, because of the way they are imported (various optimizations upon save/update).

#6 Updated by Ohad Levy almost 10 years ago

  • Target version set to 1.1

#7 Updated by Robert Birnie almost 10 years ago

I made a few changes to it and --amended the commit. I think its ready for a pull request now.

  • As recommended by Ohad, I added after_save calls on OS, Model, and Env so they get created automagically. In hand with this I removed my changes from /model/fact_importer.rb.
  • Added a "Check for Missing" button that will do the initial creation of all existing Facts/OS/Model/Env so that after upgrading to this version you can do an initial create (and so it doesn't have to check these each run).
  • Move all the creation checks into after_save in the model. Cleaning up the Controller.

#8 Updated by Robert Birnie almost 10 years ago

Could maybe add it as a after_save to run each time a report comes in. Could do a check if there's been a count run in last hour, and if not then run it. Only issue is that a busy server may have a hundred or more reports come in each minute, so checking after each report might add a lot of overhead.

#9 Updated by Ohad Levy almost 10 years ago

  • Category set to Inventory
  • Status changed from New to Closed
  • Assignee set to Robert Birnie

merged at fe4629de

Also available in: Atom PDF