Bug #20049

Retry various integration tests before failing

Added by Daniel Lobato Garcia 4 months ago. Updated 3 months ago.

Status:Closed
Priority:Normal
Assigned To:Daniel Lobato Garcia
Category:Tests
Target version:-
Difficulty: Bugzilla link:
Found in release: Pull request:https://github.com/theforeman/foreman/pull/4605, https://github.com/theforeman/foreman/pull/4615
Story points-
Velocity based estimate-
Release1.15.2Release relationshipAuto

Description

Thanks to #15894 we can retry some tests. Some of the failures I'm seeing I'm having a hard time to reproduce, like "Capybara::Poltergeist::StatusFailError: Request to 'http://127.0.0.1:41550/compute_profiles/980190962-1-Small' failed to reach server, check DNS and/or server status - Timed out with no open resource requests...". I think it's not a bad idea to let these test run 3 times and only count them as failed if they fail 3 times.

The tests in question are:

ComputeProfileJSTest.test_0001_edit compute attribute page - Capybara::Poltergeist::StatusFailError: Request to 'http://127.0.0.1:41550/compute_profiles/980190962-1-Small' failed to reach server, check DNS and/or server status - Timed out with no open resource requests...

ComputeResourceJSIntegrationTest.test_0002_add new compute attributes two pane - Capybara::Poltergeist::StatusFailError: Request to 'http://127.0.0.1:33103/compute_resources/928692541-amazon123' failed to reach server, check DNS and/or server status - Timed out with no open resource requests...

UserIntegrationTest.test_0002_edit page - Capybara::ElementNotFound: Unable to find field "user_login"...

DashboardIntegrationTest.test_0005_dashboard link hosts that had pending changes - Capybara::ExpectationNotMet: expected "/dashboard" to equal "/hosts"...

AboutIntegrationTest.test_0002_about page proxies should have version

Associated revisions

Revision 0132933a
Added by Daniel Lobato Garcia 4 months ago

Fixes #20049 - Retry various integration tests before failing

Thanks to #15894 we can retry some tests. Some of the failures I'm
seeing I'm having a hard time to reproduce, like
"Capybara::Poltergeist::StatusFailError: Request to
'http://127.0.0.1:41550/compute_profiles/980190962-1-Small' failed to
reach server, check DNS and/or server status - Timed out with no open
resource requests...". I think it's not a bad idea to let these test run
3 times and only count them as failed if they fail 3 times.

Revision 680338b6
Added by Daniel Lobato Garcia 4 months ago

Refs #20049 - Retry various integration tests

New breakages found that could be fixed by retrying (timeouts, DNS
errors...)

Revision 928c0b17
Added by Daniel Lobato Garcia 4 months ago

Fixes #20049 - Retry various integration tests before failing

Thanks to #15894 we can retry some tests. Some of the failures I'm
seeing I'm having a hard time to reproduce, like
"Capybara::Poltergeist::StatusFailError: Request to
'http://127.0.0.1:41550/compute_profiles/980190962-1-Small' failed to
reach server, check DNS and/or server status - Timed out with no open
resource requests...". I think it's not a bad idea to let these test run
3 times and only count them as failed if they fail 3 times.

(cherry picked from commit 0132933af131f85d1f4c1574fc42c446ac3b686b)

Revision b66c7167
Added by Daniel Lobato Garcia 4 months ago

Refs #20049 - Retry various integration tests

New breakages found that could be fixed by retrying (timeouts, DNS
errors...)

(cherry picked from commit 680338b623e48e941d8890e573b6801fe7a81593)

History

#1 Updated by The Foreman Bot 4 months ago

  • Status changed from New to Ready For Testing
  • Assigned To set to Daniel Lobato Garcia
  • Pull request https://github.com/theforeman/foreman/pull/4605 added

#2 Updated by Daniel Lobato Garcia 4 months ago

  • Release set to 1.15.2

Marking as 1.15.2 as I'd want to avoid failures on the test_1.15 branch.

#3 Updated by Anonymous 4 months ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#4 Updated by Daniel Lobato Garcia 4 months ago

Other breakages found:

SubnetJSIntegrationTest.test_0001_create new page

Capybara::Poltergeist::TimeoutError: Timed out waiting for response to {"id":"733683ce-32ee-4278-a08c-5cc922942636","name":"visit","args":["http://127.0.0.1:36190/subnets/new",60]}. It's possible that this happened because something took a very long time (for example a page load was slow).

Also:

AuditJSTest.test_0001_show audit with diff
HostJSTest::create new host page.test_0003_saves correct values for inherited fields without hostgroup
HostsControllerTest.test_0076_select multiple action with not exists host_ids should redirect to hosts page
ComputeResourceJSIntegrationTest.test_0002_add new compute attributes two pane
StatisticIntegrationTest.test_0001_statistics page

#5 Updated by The Foreman Bot 4 months ago

  • Pull request https://github.com/theforeman/foreman/pull/4615 added

#6 Updated by Daniel Lobato Garcia 3 months ago

  • Category set to Tests

Also available in: Atom PDF