Project

General

Profile

Actions

Bug #14155

closed

Tests are order dependent, should be runnable with random order

Added by Dominic Cleal almost 9 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Tests
Target version:
Fixed in Releases:
Found in Releases:

Description

When enabling activesupport/minitest's random test ordering on Rails 4.2, some failures are seen:

Some when executing Api::V2 OS controller tests in this order:

10:10:13 Api::V2::OperatingsystemsControllerTest#test_0005_should not create os without version = 0.11 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0014_should show os if id is description = 0.14 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0013_should show os if id is fullname = 0.10 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0009_should update associated architectures by name with UNWRAPPED node = 0.16 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0004_should create os with os parameters = 0.01 s = E
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0011_should add association of architectures by name with WRAPPED node = 0.17 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0006_should update os = 0.14 s = .
10:10:13 Api::V2::OperatingsystemsControllerTest#test_0001_should get index = 0.06 s = .
10:10:14 Api::V2::OperatingsystemsControllerTest#test_0010_should add association of architectures by ids with WRAPPED node = 0.51 s = .
10:10:14 Api::V2::OperatingsystemsControllerTest#test_0003_should create os = 0.06 s = F
10:10:14 Api::V2::OperatingsystemsControllerTest#test_0008_should update associated architectures by ids with UNWRAPPED node = 0.16 s = .
10:10:14 Api::V2::OperatingsystemsControllerTest#test_0002_should show os = 0.13 s = .
10:10:14 Api::V2::OperatingsystemsControllerTest#test_0007_should destroy os = 0.08 s = .
10:10:15 Api::V2::OperatingsystemsControllerTest#test_0012_should remove association of architectures with WRAPPED node = 0.16 s = .

This functional test has a class-level os variable which is being reassigned with OS resources.

And also seed tests in this order:

SeedsTest::populating an initial admin user#test_0002_with environment overrides = 6.05 s = .
SeedsTest::populating an initial admin user#test_0001_with defaults = 5.61 s = F

This unit test uses the with_env helper, which doesn't unset previously unassigned environment variables on completion.


Files

consoleText1.txt consoleText1.txt 789 KB index test failures Dominic Cleal, 03/11/2016 09:45 AM

Related issues 1 (0 open1 closed)

Related to Foreman - Feature #13244: Upgrade Ruby on Rails to 4.2ClosedDominic Cleal01/15/2016Actions
Actions

Also available in: Atom PDF