Project

General

Profile

Bug #12331

Transient permission test failures

Added by Justin Sherrill over 6 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Tests
Target version:
Difficulty:
hard
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Currently we are seeing random test failures that happens roughly 2-3x per week across everywhere the katello tests run:

11 tests always fail:

Katello::OrganizationAuthorizationNoPermsTest.test_promotable_promotion_paths_one    0.50152355    2
Katello::OrganizationAuthorizationNoPermsTest.test_read_promotion_paths_one 0.7418944 2
Katello::ProductAuthorizationNoPermsTest.test_readable_repositories_with_search 0.79982895 2
Katello::RepositoryAuthorizationNonAuthUserTest.test_readable_with_content_view 0.7553367 2
Katello::RepositoryAuthorizationNonAuthUserTest.test_readable_with_product 0.6109226 2
Katello::RepositoryAuthorizationNonAuthUserTest.test_readable_with_versions 0.8154077 2
Katello::SyncManagementControllerTest.test_destroy_protected 0.7546445 2
Katello::SyncManagementControllerTest.test_index_protected 0.7302906 2
Katello::SyncManagementControllerTest.test_sync_protected 0.46630755 2
Katello::SyncManagementControllerTest.test_sync_status_protected 0.4743731 2
Katello::SystemAuthorizationWithPermsTest.test_all_editable?

Some debug was added in http://projects.theforeman.org/issues/11939 which shows that the katello permissions seem to randomly disappear. In fact other plugin permissions (foreman-docker) also disappear.

This has only been seen on jenkins and not locally.


Related issues

Related to Katello - Bug #11939: add debugging of transiet fixture test failureClosed2015-09-23

Associated revisions

Revision f6841582 (diff)
Added by John Mitsch over 6 years ago

Refs #12331 - Fixes authorization tests

This is not a long term fix, but the authorization tests errors that are only showing up in the full test suite can be avoided by rearranging the tests to run in a different order.

Revision 9d867270
Added by John Mitsch over 6 years ago

Merge pull request #5574 from johnpmitsch/permissions_fix

Refs #12331 - Fixes authorization tests

History

#1 Updated by Justin Sherrill over 6 years ago

  • Related to Bug #11939: add debugging of transiet fixture test failure added

#2 Updated by Justin Sherrill over 6 years ago

The output from the debug added in #11939:

00:25:21.607 test_all_editable?(Katello::SystemAuthorizationWithPermsTest):
00:25:21.607 RuntimeError: Permissions not found: edit_content_hosts, some permissions were not found: ["view_architectures", "create_architectures", "edit_architectures", "destroy_architectures", "view_authenticators", "create_authenticators", "edit_authenticators", "destroy_authenticators", "view_bookmarks", "create_bookmarks", "edit_bookmarks", "destroy_bookmarks", "view_compute_resources", "create_compute_resources", "edit_compute_resources", "destroy_compute_resources", "view_templates", "create_templates", "edit_templates", "destroy_templates", "deploy_templates", "access_dashboard", "view_domains", "create_domains", "edit_domains", "destroy_domains", "view_realms", "create_realms", "edit_realms", "destroy_realms", "view_environments", "create_environments", "edit_environments", "destroy_environments", "import_environments", "view_external_variables", "create_external_variables", "edit_external_variables", "destroy_external_variables", "view_globals", "create_globals", "edit_globals", "destroy_globals", "create_params", "edit_params", "destroy_params", "view_hostgroups", "create_hostgroups", "edit_hostgroups", "destroy_hostgroups", "view_hosts", "create_hosts", "edit_hosts", "destroy_hosts", "build_hosts", "power_hosts", "console_hosts", "ipmi_boot", "puppetrun_hosts", "view_images", "create_images", "edit_images", "destroy_images", "view_locations", "create_locations", "edit_locations", "destroy_locations", "assign_locations", "view_media", "create_media", "edit_media", "destroy_media", "view_models", "create_models", "edit_models", "destroy_models", "view_operatingsystems", "create_operatingsystems", "edit_operatingsystems", "destroy_operatingsystems", "view_organizations", "create_organizations", "edit_organizations", "destroy_organizations", "assign_organizations", "view_ptables", "create_ptables", "edit_ptables", "destroy_ptables", "view_puppetclasses", "create_puppetclasses", "edit_puppetclasses", "destroy_puppetclasses", "import_puppetclasses", "view_reports", "destroy_reports", "upload_reports", "access_settings", "view_smart_proxies", "create_smart_proxies", "edit_smart_proxies", "destroy_smart_proxies", "view_smart_proxies_autosign", "create_smart_proxies_autosign", "destroy_smart_proxies_autosign", "view_smart_proxies_puppetca", "edit_smart_proxies_puppetca", "destroy_smart_proxies_puppetca", "view_statistics", "view_subnets", "create_subnets", "edit_subnets", "destroy_subnets", "import_subnets", "view_tasks", "view_trends", "create_trends", "edit_trends", "destroy_trends", "update_trends", "view_usergroups", "create_usergroups", "edit_usergroups", "destroy_usergroups", "view_users", "create_users", "edit_users", "destroy_users", "edit_classes", "view_compute_resources_vms", "create_compute_resources_vms", "edit_compute_resources_vms", "destroy_compute_resources_vms", "power_compute_resources_vms", "console_compute_resources_vms", "view_facts", "upload_facts", "view_audit_logs", "view_plugins", "view_provisioning_templates", "view_roles"]

#3 Updated by Eric Helms over 6 years ago

  • Legacy Backlogs Release (now unused) set to 86
  • Triaged changed from No to Yes

#4 Updated by Dominic Cleal over 6 years ago

Justin Sherrill wrote:

Some debug was added in http://projects.theforeman.org/issues/11939 which shows that the katello permissions seem to randomly disappear. In fact other plugin permissions (foreman-docker) also disappear.

I don't know about Katello's tests, but in Foreman we wouldn't ever expect permissions defined through the plugin registration API to be present in the test environment. Tests are usually only run with a clean database and populated through fixtures (see #12143). Plugins typically add fixtures or factories when testing their own permissions if required, so I wouldn't expect to see foreman-docker's defined permissions.

The frequency of this failing a test run has increased since adding an additional Ruby version to the test_katello_core matrix.

#5 Updated by Justin Sherrill over 6 years ago

Dominic, good point about my comment about foreman-docker, those should not appear. This started happening in the rails 4 branch consistently (reproducible every time). I didn't look into it myself and the rails 4 guys were not able to nail to a 'real' solution. A workaround was used (changing the order the set of tests are run in). We could see about porting that to master or I could spend some time on the rails 4 branch digging in myself.

https://github.com/Katello/katello/commit/f6841582bbfd35bb3c5239dc313455f980e01071 & https://github.com/Katello/katello/commit/72e8da23b97b42d9da2696acfc4f1b7f51d81db6

were the two changes made as a workaround.

#6 Updated by Eric Helms about 6 years ago

  • Priority changed from High to Normal
  • Legacy Backlogs Release (now unused) changed from 86 to 114

#7 Updated by Justin Sherrill about 5 years ago

  • Status changed from New to Resolved
  • Legacy Backlogs Release (now unused) changed from 114 to 166

This is no longer an issue.

Also available in: Atom PDF