Bug #30413


Adding more than 600+ roles to non-admin user causes 500 error and Ruby stacktrace

Added by Will Foster over 3 years ago. Updated 7 months ago.

Target version:
Fixed in Releases:
Found in Releases:


Hello Foreman team,

We've discovered what looks like a RBAC scalability bug where if you associate more than 600+ roles to a non-admin user it breaks their UI and hammer CLI and returns 500 errors.

We've been able to reproduce this on both 1.19.1 and 1.23.0.

This is what our process looks like (we manage large-scale R&D scale/performance environments in engineering via QUADS -

1) associate one role per bare-metal system (named either via FQDN or short hostname)

hammer role create --name

2) associate one filter for that role with permissions

hammer filter create --role --search "name =" --permissions view_hosts,edit_hosts,build_hosts,power_hosts,console_hosts --role-id $(hammer role info --name | egrep ^Id: | awk '{ print $NF }')

3) add a non-admin user

hammer user create --login cloud01 --password password --mail --auth-source-id 1

4) add that role to the new user.

5) Do this 635+ times for 635+ hosts or more.

  • Up until 500+ you can still get the return of hammer host list -u cloud01 -p mypassword
  • After 500-600 roles you only get 500's for that user only.

We have since moved to using system ownership in lieu of 1 x role per bare-metal system, however we think this is a scalability bug and it can be easily recreated by creating a non-admin user, creating 600+ roles with filters, then associating those roles to that user and trying to login to the UI or run any hammer host commands.


foreman_rbac_roles_bug.txt foreman_rbac_roles_bug.txt 908 KB mod_passenger stack trace and debug info Will Foster, 07/15/2020 11:34 AM
Actions #1

Updated by Will Foster over 3 years ago

On a related note, tbrisker on #theforeman / Freenode suggested that we switch to system-ownership based views so we've made appropriate adjustments to our project codebase to use this instead:

However, we still feel that we're not "holding it wrong" (Apple iphone 4 antenna bug gaffe) and there can still be valid use cases that folks might need more than 600+ roles associated with a non-admin user so hence filing this.


Also available in: Atom PDF