Non-admin users can only be assigned the 'view_authenticators' (or edit, etc...) permission.
However, the API and UI controllers do not take that into account, and use 'view_auth_source_ldaps' (and the rest).

The fix is simple, override `controller_permission` in the controllers to make sure users are checked against the right kind of permission.