Feature #12163

Add ability to expire facts based on fact name.

Added by Ryan Sabatini about 2 years ago. Updated 7 months ago.

Status:Closed
Priority:Normal
Assigned To:Marek Hulán
Category:Facts
Target version:-
Difficulty: Bugzilla link:1422458
Found in release: Pull request:https://github.com/theforeman/foreman/pull/4275
Story points-
Velocity based estimate-
Release1.15.0Release relationshipAuto

Description

Currently there isn't a way to expire old facts that are no longer being collected from the database. This feature adds a rake task similar to reports:expire to delete facts based on the provided fact name.


Related issues

Related to Foreman - Bug #16576: Structured facts importer is extremely slow Closed 09/15/2016
Duplicated by Foreman - Feature #18479: Add a rake task to cleanup facts that are not used anymore Duplicate 02/13/2017

Associated revisions

Revision 89d84f63
Added by Marek Hulán 7 months ago

Fixes #12163 - add a rake task to clean up orphaned facts

History

#1 Updated by Ohad Levy about 2 years ago

  • Status changed from New to Need more information

once new facts comes in, old facts (e.g. facts that dont exists in the new facts set) gets deleted automatically.

#2 Updated by Ryan Sabatini about 2 years ago

Ohad Levy wrote:

once new facts comes in, old facts (e.g. facts that dont exists in the new facts set) gets deleted automatically.

If I stop collecting a custom fact or rename the custom fact, does that fact automatically get deleted? The main reason I came up with the solution is that we were collecting some poorly built custom facts that resulted in ~174k fact_name entries and ~12 million fact values in the database. The sheer volume of the facts broke the javascript on the drop down box for "Trendable" when creating a new trend counter.

We have since stopped collecting the facts that were adding the extra entries that we weren't really using. I have come up with a coded solution similar to reports:expire that allows me to delete the fact name and values for the fact names that we are not longer collecting which allowed me to get the total values down far enough to use the trendable drop down box to create a new trend.

#3 Updated by Dominic Cleal almost 2 years ago

  • Status changed from Need more information to New

Ryan Sabatini wrote:

Ohad Levy wrote:

once new facts comes in, old facts (e.g. facts that dont exists in the new facts set) gets deleted automatically.

If I stop collecting a custom fact or rename the custom fact, does that fact automatically get deleted?

It should delete the fact values, but not the fact names. If there were many custom fact names added, as you suggest, then these don't get cleaned up during an import of facts for a host (assuming no other host has a value for it).

I'm changing the ticket status as I think what you're describing is valid.

#4 Updated by Dominic Cleal about 1 year ago

  • Related to Bug #16576: Structured facts importer is extremely slow added

#5 Updated by Marek Hulán 8 months ago

  • Duplicated by Feature #18479: Add a rake task to cleanup facts that are not used anymore added

#6 Updated by The Foreman Bot 8 months ago

  • Status changed from New to Ready For Testing
  • Assigned To set to Marek Hulán
  • Pull request https://github.com/theforeman/foreman/pull/4275 added

#7 Updated by Marek Hulán 8 months ago

  • Bugzilla link set to 1422458

#8 Updated by Marek Hulán 7 months ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#9 Updated by Dominic Cleal 7 months ago

  • Release set to 1.15.0

Also available in: Atom PDF