The "report" factory used in our test suite uses a sequence for the "reported_at" field, incrementally making new reports that are further and further back in time:

  0 FactoryGirl.define do
  1   factory :report do
  2     host
  3     sequence(:reported_at) { |n| n.minutes.ago }
[..]

In a full test run, this can eventually start creating reports that are out of sync compared to the current time.

This causes test failures on 1.13-stable when running rake test:

HostTest.test_0038_host #refresh_statuses saves all relevant statuses and refreshes global status
Expected: 0
  Actual: 1 (Minitest::Assertion)
/var/lib/workspace/workspace/test_1_13_stable/database/sqlite3/ruby/2.3/slave/fast/test/unit/host_test.rb:509 

Debugging the above test shows a host was created with a new report at 11:08, but the reported_at time is 08:57, therefore the host is out of sync:

[20] pry(#<HostTest>)> host.host_statuses.first.last_report
=> #<ConfigReport:0x00000018608048
 id: 5,
 host_id: 1,
 reported_at: Fri, 25 Nov 2016 08:57:35 UTC +00:00,
 created_at: Fri, 25 Nov 2016 11:08:35 UTC +00:00,
 updated_at: Fri, 25 Nov 2016 11:08:35 UTC +00:00,
[..]
[21] pry(#<HostTest>)> host.host_statuses.first.out_of_sync?
=> true

This causes the global status error and assertion to fail.

Running the test in isolation works as reported_at is closer to the current time:

[3] pry(#<HostTest>)> host.host_statuses.first.last_report
=> #<ConfigReport:0x0000000f3109e8
 id: 5,
 host_id: 1,
 reported_at: Fri, 25 Nov 2016 11:22:17 UTC +00:00,
 created_at: Fri, 25 Nov 2016 11:27:17 UTC +00:00,
 updated_at: Fri, 25 Nov 2016 11:27:17 UTC +00:00,
[..]
[4] pry(#<HostTest>)> host.host_statuses.first.out_of_sync?
=> false

And the test passes.