Project

General

Profile

Bug #13608

LDAP Authentication failure: stack level too deep

Added by Sameer Syed about 4 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Authentication
Target version:
Difficulty:
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

I've couple of Katello/foreman servers and I want to use SLDAP to authenticate users. While the LDAP auth works on one server (Katello 2.3), it fails on the other after I upgraded to Katello 2.4 version. I get a "stack level too deep" error. Full trace:

SystemStackError
stack level too deep
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications/instrumenter.rb:23

I've the exact same SLDAP settings on both the servers. I had build another server with Katello 2.3 and it fails on this 3rd server as well with the same errors as above.

production.log production.log 93.3 KB production.log with debug Sameer Syed, 02/10/2016 02:52 PM
ldap_traceback.txt ldap_traceback.txt 206 KB Josh Baird, 06/08/2016 12:16 PM

Related issues

Has duplicate Foreman - Bug #14413: External AD authentication with Satellite 6.1 errors with stack level too deepDuplicate2016-03-31
Has duplicate Foreman - Bug #14407: foreman-rake ldap:refresh_usergroups ends in endless loop with external user group in Active DirectoryDuplicate2016-03-31

History

#1 Updated by Dominic Cleal about 4 years ago

  • Subject changed from LDAP Authentication failure to LDAP Authentication failure: stack level too deep
  • Category set to Authentication
  • Priority changed from Urgent to Normal

Is there no more information than that in production.log?

#2 Updated by Sameer Syed about 4 years ago

Dominic Cleal wrote:

Is there no more information than that in production.log?

Hi Dominic,

Here are the logs from production.log

2016-02-05 00:47:11 [app] [I] | | Started GET "/" for 10.3.28.159 at 2016-02-05 00:47:11 +0000
2016-02-05 00:47:11 [app] [I] Processing by DashboardController#index as HTML
2016-02-05 00:47:11 [app] [I] Redirected to https://katello.example.com/users/login
2016-02-05 00:47:11 [app] [I] Filter chain halted as :require_login rendered or redirected
2016-02-05 00:47:11 [app] [I] Completed 302 Found in 1ms (ActiveRecord: 0.0ms)
2016-02-05 00:47:11 [app] [I] | | Started GET "/users/login" for 10.3.28.159 at 2016-02-05 00:47:11 +0000
2016-02-05 00:47:11 [app] [I] Processing by UsersController#login as HTML
2016-02-05 00:47:11 [app] [I] Rendered users/login.html.erb within layouts/login (1.8ms)
2016-02-05 00:47:11 [app] [I] Rendered layouts/base.html.erb (1.0ms)
2016-02-05 00:47:11 [app] [I] Completed 200 OK in 8ms (Views: 3.7ms | ActiveRecord: 0.3ms)
2016-02-05 00:47:18 [app] [I] | | Started POST "/users/login" for 10.3.28.159 at 2016-02-05 00:47:18 +0000
2016-02-05 00:47:18 [app] [I] Processing by UsersController#login as HTML
2016-02-05 00:47:18 [app] [I] Parameters: {"utf8"=>"�", "authenticity_token"=>"ZcCTOHoRTMFAUV0xtVyiZZ0EEFHOL2DDAOR97/G0iK0=", "login"=>{"login"=>"sameer", "password"=>"[FILT}
2016-02-05 00:47:18 [sql] [I] Successfully decrypted field for AuthSourceLdap LDAP-Auth
2016-02-05 00:47:20 [app] [I] Expire fragment views/tabs_and_title_records-4 (0.1ms)
2016-02-05 00:47:20 [sql] [I] Successfully decrypted field for AuthSourceLdap LDAP-Auth
2016-02-05 00:48:53 [foreman-tasks/dynflow] [I] start terminating client dispatcher...
2016-02-05 00:48:53 [foreman-tasks/dynflow] [I] stop listening for new events...
2016-02-05 00:48:53 [foreman-tasks/dynflow] [I] start terminating clock...
2016-02-05 00:49:53 [app] [W] Action failed | SystemStackError: stack level too deep | /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications/instrumenter.rb:23
2016-02-05 00:49:53 [app] [I] Rendered common/500.html.erb within layouts/application (1.5ms)
2016-02-05 00:49:53 [app] [I] Rendered layouts/base.html.erb (1.1ms)
2016-02-05 00:49:53 [app] [I] Completed 500 Internal Server Error in 154696ms (Views: 4.0ms | ActiveRecord: 1.7ms)

#3 Updated by Dominic Cleal about 4 years ago

Thanks, are you able to do that with debug logs enabled? http://theforeman.org/manuals/1.10/index.html#7.2Debugging

#4 Updated by Sameer Syed about 4 years ago

Dominic Cleal wrote:

Thanks, are you able to do that with debug logs enabled? http://theforeman.org/manuals/1.10/index.html#7.2Debugging

I've enabled debug, attaching the production.log (sanitized).

#5 Updated by Dominic Cleal about 4 years ago

What's the output of rpm -q tfm-rubygem-ldap_fluff? Check it's up to date, should be 0.4.0 on Foreman 1.10.

#6 Updated by Sameer Syed about 4 years ago

Dominic Cleal wrote:

What's the output of rpm -q tfm-rubygem-ldap_fluff? Check it's up to date, should be 0.4.0 on Foreman 1.10.

It is already at 0.4.0:

rpm -q tfm-rubygem-ldap_fluff
tfm-rubygem-ldap_fluff-0.4.0-1.el6.noarch

#7 Updated by Dominic Cleal about 4 years ago

Are you able to see anything in the layout of the LDAP groups assigned to the user that might cause infinite recursion? You'll see in production.log that it's cycling between two groups - CN=CoreSprintReview, CN=OPsSprintReview - are they members of each other or anything you can see?

#8 Updated by Karli Sjöberg about 4 years ago

Dominic Cleal wrote:

Are you able to see anything in the layout of the LDAP groups assigned to the user that might cause infinite recursion? You'll see in production.log that it's cycling between two groups - CN=CoreSprintReview, CN=OPsSprintReview - are they members of each other or anything you can see?

Oh oh! Me me me! I have just installed Foreman 1.10.1 and ldap-fluff 0.4.0, and had this exact issue! I looked through the two groups it was looping and saw that, while there was no infinite recursion, one of the nested groups was empty. As soon as I removed it, the task completed without issue.

And when I re-membered that group, it started looping again! So that´s a new angle to this problem perhaps? that empty groups makes it loop. Don´t think I´ve seen that reported before... And I would consider that a bug. Just because a group happens to be emtpy doesn´t mean it should prevent people from logging in, wouldn´t you agree?

/K

#9 Updated by Dominic Cleal about 4 years ago

  • Legacy Backlogs Release (now unused) set to 123

Very useful info, thanks Karli.

#10 Updated by Sameer Syed almost 4 years ago

Dominic Cleal wrote:

Very useful info, thanks Karli.

Thanks Karli, indeed very useful information, unfortunately the AD sever is managed by me. I will have to get the AD administrator involved into this. I will update if I make any progress on this and I agree this bug needs to be filed.

#11 Updated by Dominic Cleal almost 4 years ago

  • Legacy Backlogs Release (now unused) changed from 123 to 145

#12 Updated by Dominic Cleal almost 4 years ago

  • Status changed from New to Assigned
  • Assignee set to Dominic Cleal

#13 Updated by Dominic Cleal almost 4 years ago

  • Status changed from Assigned to Ready For Testing
  • Legacy Backlogs Release (now unused) deleted (145)

https://github.com/theforeman/ldap_fluff/pull/49 fixes this. I don't think it's a 1.10 regression, but as a workaround either apply the patch locally or disable group syncing in the LDAP auth settings.

#14 Updated by Sameer Syed almost 4 years ago

Dominic Cleal wrote:

https://github.com/theforeman/ldap_fluff/pull/49 fixes this. I don't think it's a 1.10 regression, but as a workaround either apply the patch locally or disable group syncing in the LDAP auth settings.

Thanks Dominic, I will try out your suggestions and will update this thread.

#15 Updated by Sameer Syed almost 4 years ago

Sameer Syed wrote:

Dominic Cleal wrote:

https://github.com/theforeman/ldap_fluff/pull/49 fixes this. I don't think it's a 1.10 regression, but as a workaround either apply the patch locally or disable group syncing in the LDAP auth settings.

Thanks Dominic, I will try out your suggestions and will update this thread.

Update: Disabling the group syncing works, I don't need to sync groups. I will leave at that. Thanks it works now. Will this bug be fixed in 1.10.3?

#16 Updated by Dominic Cleal almost 4 years ago

It'll probably only be fixed in 1.11 as it needs an update to ldap_fluff which will be tricky to backport to 1.10.

#17 Updated by Dominic Cleal almost 4 years ago

  • Status changed from Ready For Testing to Closed
  • Legacy Backlogs Release (now unused) set to 71

Fixed in ldap_fluff 0.4.1.

#18 Updated by Dominic Cleal almost 4 years ago

  • Has duplicate Bug #14413: External AD authentication with Satellite 6.1 errors with stack level too deep added

#19 Updated by Josh Baird over 3 years ago

  • Status changed from Closed to Assigned

It looks like I may be hitting this in 1.11.1. I can consistently produce this in both my QA and PROD environments.

One user from the LDAP directory (AD) is able to login to both QA/PROD, and several users are not in both environments. I have attached the production.log snippet.

#21 Updated by Dominic Cleal over 3 years ago

  • Legacy Backlogs Release (now unused) deleted (71)

https://github.com/theforeman/ldap_fluff/pull/53 might do a better job of preventing this.

#22 Updated by Josh Baird over 3 years ago

I can confirm that the patch in PR #53 referenced above fixes my issue. Thanks!

#23 Updated by Dominic Cleal over 3 years ago

  • Status changed from Assigned to Closed
  • Legacy Backlogs Release (now unused) set to 136

The patch in ldap_fluff was merged and will be in 1.12.0-RC2.

#24 Updated by Dennis Graht about 3 years ago

  • Legacy Backlogs Release (now unused) changed from 136 to 210

Hi this issue looks like still existing in 1.14. i use fee ipa as ldap

2017-01-29 11:47:13 c48e8a11 [app] [I] Processing by UsergroupsController#edit as /*
2017-01-29 11:47:13 c48e8a11 [app] [I] Parameters: {"id"=>"1"}
2017-01-29 11:47:13 c48e8a11 [app] [I] Rendered usergroups/_external.html.erb (3.7ms)
2017-01-29 11:47:13 c48e8a11 [app] [I] Rendered usergroups/_external.html.erb (1.9ms)
2017-01-29 11:47:13 c48e8a11 [app] [I] Rendered usergroups/_form.html.erb (49.7ms)
2017-01-29 11:47:13 c48e8a11 [app] [I] Rendered usergroups/edit.html.erb (53.8ms)
2017-01-29 11:47:13 c48e8a11 [app] [I] Completed 200 OK in 67ms (Views: 54.0ms | ActiveRecord: 5.3ms)
2017-01-29 11:47:18 c48e8a11 [app] [I] Started PATCH "/usergroups/1-ldap-sat-admin" for 192.168.181.204 at 2017-01-29 11:47:18 +0100
2017-01-29 11:47:18 c48e8a11 [app] [I] Processing by UsergroupsController#update as */

2017-01-29 11:47:18 c48e8a11 [app] [I] Parameters: {"utf8"=>"✓", "authenticity_token"=>"0dk7ZG7/ELLXBAKOgzL258cxqzHbvkq7UOSo8AwIKmhjmepwVeyUYrnudqmODRVOvisIp1ko23X8mmbFpjQ1LA==", "usergroup"=>{"name"=>"ldap-sat-admin", "usergroup_ids"=>["", "1"], "user_ids"=>[""], "admin"=>"1", "role_ids"=>[""], "external_usergroups_attributes"=>{"0"=>{"_destroy"=>"false", "name"=>"sat-admin", "auth_source_id"=>"4", "id"=>"1"}}}, "id"=>"1-ldap-sat-admin"}
2017-01-29 11:47:19 c48e8a11 [app] [W] Action failed | SystemStackError: stack level too deep | /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/postgresql/utils.rb:67:in `scan' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/postgresql/utils.rb:67:in `extract_schema_qualified_name' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/postgresql/quoting.rb:31:in `quote_table_name' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:776:in `quote_table_name' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:711:in `visit_Arel_Attributes_Attribute' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/reduce.rb:13:in `visit' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:675:in `visit_Arel_Nodes_Equality' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/reduce.rb:13:in `visit' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:796:in `block in inject_join' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:792:in `each' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:792:in `each_with_index' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:792:in `each' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:792:in `inject' | /opt/rh/sclo-ror42/root/usr/share/gems/gems/arel-6.0.3/lib/arel/visitors/to_sql.rb:792:in `inject_join'

#25 Updated by Dominic Cleal about 3 years ago

  • Legacy Backlogs Release (now unused) changed from 210 to 136

Dennis, please do not modify the shipped release on closed bugs. If you're encountering a bug, open a new issue.

#26 Updated by Marek Hulán almost 3 years ago

  • Has duplicate Bug #14407: foreman-rake ldap:refresh_usergroups ends in endless loop with external user group in Active Directory added

Also available in: Atom PDF