Taken from thread in Foreman users mailing list: https://groups.google.com/forum/?hl=en#!topic/foreman-users/ABA18RmYets

Upload of facts from host to Foreman is working « only the first time », when I validate a new host certificate on Foreman web UI. After that, there is no more facts updates.

Exception : All is working fine for the local server facts (= for the server hosting foreman).

The error is : ActiveRecord::RecordInvalid: Validation failed: Fact name has already been taken

When I inserted logging into the affected file (/usr/share/foreman/app/services/fact_importer.rb), it showed this result:

2016-04-20 20:19:49 [app] [I] facts_to_create called for fact '_timestamp'
2016-04-20 20:19:49 [app] [W] Action failed | ActiveRecord::RecordInvalid: Validation failed: Fact name has already been taken

When I looked through the YAML file on the host, it had this fact in it:
"_timestamp": 2016-04-20 20:19:48.685814 +00:00

A colleague was searching out the "_timestamp" issue, and discovered it's a "hidden" fact that puppet agent sends to the puppet master.

It's in the 3.x versions of Puppet, but deprecated and removed from Puppet 4.

https://tickets.puppetlabs.com/browse/PDB-337
https://tickets.puppetlabs.com/browse/PUP-3129
https://tickets.puppetlabs.com/browse/PUP-3130

The timestamp is not displayed as part of the Foreman stored facts (either in the GUI or via hammer fact list --search host={hostname}. However, it seems that Foreman does use a timestamp in some fashion, and this is where the conflict comes from, which prevents the foreman report processor from adding new facts to the host facts.

1. pwd && grep -r "_timestamp" * | grep -v "log"
   /usr/share/foreman
   app/models/host_status/status.rb: before_validation :update_timestamp, :if => ->(status) { status.reported_at.blank? }
   app/models/host_status/status.rb: update_timestamp
   app/models/host_status/status.rb: def update_timestamp
   app/models/host_status/configuration_status.rb: def update_timestamp
   app/models/fact_value.rb: scope :no_timestamp_facts, lambda {
   app/models/fact_value.rb: includes(:fact_name).where("fact_names.name <> ?",:_timestamp)
   app/models/fact_value.rb: joins(:fact_name).where("fact_names.name = ?",:_timestamp)
   app/models/fact_name.rb: scope :no_timestamp_fact, -> { where("fact_names.name <> ?",:_timestamp) }
   app/models/fact_name.rb: scope :timestamp_facts, -> { where(:name => :_timestamp) }
   app/models/host/base.rb: time = facts[:_timestamp]
   app/controllers/api/v2/fact_values_controller.rb: no_timestamp_facts.
   app/controllers/api/v1/fact_values_controller.rb: no_timestamp_facts.
   app/controllers/fact_values_controller.rb: base = resource_base.no_timestamp_facts
   app/controllers/fact_values_controller.rb: @fact_values = values.no_timestamp_facts.required_fields.paginate :page => params[:page]
   app/controllers/facts_controller.rb: render :json => FactName.no_timestamp_fact
   app/helpers/trends_helper.rb: next_timestamp = counter.try(:interval_end) || Time.now
   app/helpers/trends_helper.rb: data << [next_timestamp.to_i*1000 - 1, counter.count]
   app/views/users/_user_fact.html.erb: <%= f.collection_select :fact_name_id, FactName.no_timestamp_fact, :id, :name, {}, :class => "form-control" >
   app/views/trends/_form.html.erb: <= select_f(f, :trendable_id, FactName.no_timestamp_fact, :id, :name, { :include_blank => true }, :disabled => true) %>
   lib/timed_cached_store.rb: delete_if_expired($1) if key =~ /(\w+)_timestamp/
   lib/timed_cached_store.rb: "#{name}_timestamp"
   lib/tasks/convert.rake: DevelopmentModelClass.record_timestamps = false
   and

pwd && grep -r "_timestamp" *

/usr/share/katello-installer
modules/foreman/spec/static_fixtures/fake.host.fqdn.com.yaml: !ruby/sym "_timestamp": 2013-08-02 18:40:14.147766 +01:00

Our puppet version is 3.8.6, and this prevents any updates on host facts beyond the initial checkin.