Project

General

Profile

Actions

Bug #18179

closed

smart proxy refresh throws stackerror

Added by Klaas D over 7 years ago. Updated over 7 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

Hi,
I recently upgraded my main foreman/katello server to 3.2.3. Afterwards I reinstalled my secondary puppet smartproxy. Now everytime I refresh the smart proxy there is an error "SystemStackError (stack level too deep)". I have other smart proxies that work fine. Even though it throws an error the refresh seems to work because I can turn off/on features.

CLI:

# hammer proxy refresh-features --name smartproxypuppet.tld
Refresh of smart proxy features failed:
  500 Internal Server Error

Production Log:

2017-01-20 16:13:21 7344a677 [app] [I] Started GET "/api/smart_proxies?search=name+%3D+%22smartproxypuppet.tld%22" for 1.1.1.20 at 2017-01-20 16:13:21 +0100
2017-01-20 16:13:21 7344a677 [app] [I] Processing by Api::V2::SmartProxiesController#index as JSON
2017-01-20 16:13:21 7344a677 [app] [I]   Parameters: {"search"=>"name = \"smartproxypuppet.tld\"", "apiv"=>"v2", "smart_proxy"=>{}}
2017-01-20 16:13:21 7344a677 [app] [I] Authorized user admin(Admin User)
2017-01-20 16:13:21 7344a677 [app] [I]   Rendered api/v2/smart_proxies/index.json.rabl within api/v2/layouts/index_layout (44.5ms)
2017-01-20 16:13:21 7344a677 [app] [I] Completed 200 OK in 113ms (Views: 60.4ms | ActiveRecord: 12.3ms)
2017-01-20 16:13:22 713e4c44 [app] [I] Started PUT "/api/smart_proxies/2/refresh" for 1.1.1.20 at 2017-01-20 16:13:22 +0100
2017-01-20 16:13:22 713e4c44 [app] [I] Processing by Api::V2::SmartProxiesController#refresh as JSON
2017-01-20 16:13:22 713e4c44 [app] [I]   Parameters: {"apiv"=>"v2", "id"=>"2", "smart_proxy"=>{}}
2017-01-20 16:13:22 713e4c44 [app] [I] Authorized user admin(Admin User)
2017-01-20 16:14:00 713e4c44 [app] [I] Completed 500 Internal Server Error in 38278ms (ActiveRecord: 1116.1ms)
2017-01-20 16:14:00 713e4c44 [app] [F] 
 | SystemStackError (stack level too deep):
 |   app/models/concerns/audit_extensions.rb:56:in `ensure_auditable_and_associated_name'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 [...many many many more...]
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path'
 |   katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path'
 |   katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:66:in `refresh_with_puppet_path'
 |   foreman_remote_execution (1.2.2) app/models/concerns/foreman_remote_execution/smart_proxy_extensions.rb:21:in `refresh_with_remote_execution'
 |   app/controllers/api/v2/smart_proxies_controller.rb:60:in `refresh'
 |   app/controllers/api/v2/base_controller.rb:152:in `disable_json_root'
 |   app/controllers/concerns/application_shared.rb:14:in `set_timezone'
 |   app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 |   lib/middleware/catch_json_parse_errors.rb:9:in `call'
 |   lib/middleware/tagged_logging.rb:18:in `call'

Foreman Installer for Capsule:

foreman-installer --scenario capsule\
                    --capsule-parent-fqdn                         "foreman.tld"\
                    --foreman-proxy-register-in-foreman           "true"\
                    --foreman-proxy-foreman-base-url              "https://foreman.tld"\
                    --foreman-proxy-trusted-hosts                 "foreman.tld"\
                    --foreman-proxy-trusted-hosts                 "smartproxypuppet.tld"\
                    --foreman-proxy-oauth-consumer-key            "KEY"\
                    --foreman-proxy-oauth-consumer-secret         "KEY"\
                    --capsule-pulp-oauth-secret                   "KEY"\
                    --capsule-certs-tar                           "~/smartproxypuppet.tld-certs.tar"\
                    --capsule-puppet-server-implementation puppetserver\
                    --capsule-puppet-ca-proxy "https://foreman.tld"\
                    --foreman-proxy-puppetca false\
                    --foreman-proxy-templates false\
                    --foreman-proxy-plugin-remote-execution-ssh-enabled false\
                    --no-enable-foreman-proxy-plugin-pulp\
                    --foreman-proxy-plugin-pulp-enabled false

I need to disable pulp in /etc/foreman-proxy/settings.d/pulp.yml afterwards and install httpd before running installer but for that I'll open a separate bug :)

Greetings
Klaas


Related issues 1 (0 open1 closed)

Related to Katello - Bug #18185: smart proxy refresh throws stackerrorClosed01/20/2017Actions
Actions #1

Updated by Anonymous over 7 years ago

Could you post smart-proxy log for when this failure happens? It's not clear from Foreman's log where the failure is occurring.

Actions #2

Updated by Klaas D over 7 years ago

Hi,
on the puppet smart proxy there are logs like this:

I, [2017-01-20T17:15:19.681161 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:15:19.766234 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0092

I, [2017-01-20T17:15:19.851935 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:15:19.932268 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:15:20.014565 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:20 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.056454 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.126090 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.220527 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.309505 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.400620 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.485953 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0003

I, [2017-01-20T17:16:45.575194 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.667581 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.750023 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.835060 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.907984 #50530]  INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

(lots of them) no errors

the proxy log on the foreman server is not showing anything while this request is happening

the request itsself seems to work:

# curl -vks --cacert /etc/foreman-proxy/foreman_ssl_ca.pem --cert /etc/foreman-proxy/foreman_ssl_cert.pem --key /etc/foreman-proxy/foreman_ssl_key.pem "https://smartproxypuppet.tld:9090/features" 
* About to connect() to smartproxypuppet.tld port 9090 (#0)
*   Trying 10.165.1.22...
* Connected to smartproxypuppet.tld (10.165.1.22) port 9090 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate from file
*       subject: CN=foreman.tld,OU=FOREMAN_PROXY,O=FOREMAN,ST=State,C=TLD
*       start date: Jun 15 15:58:33 2016 GMT
*       expire date: Jun 17 15:58:33 2036 GMT
*       common name: foreman.tld
*       issuer: CN=foreman.tld,OU=Name,O=Name,L=Munich,ST=State,C=TLD
* SSL connection using TLS_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=smartproxypuppet.tld,OU=SMART_PROXY,O=FOREMAN,ST=North Carolina,C=US
*       start date: Jun 24 10:23:07 2016 GMT
*       expire date: Jun 26 10:23:07 2036 GMT
*       common name: smartproxypuppet.tld
*       issuer: CN=foreman.tld,OU=Name,O=Name,L=Munich,ST=State,C=TLD
> GET /features HTTP/1.1
> User-Agent: curl/7.29.0
> Host: smartproxypuppet.tld:9090
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json;charset=utf-8
< Content-Length: 10
< X-Content-Type-Options: nosniff
< Server: WEBrick/1.3.1 (Ruby/2.0.0/2015-12-16) OpenSSL/1.0.1e
< Date: Fri, 20 Jan 2017 16:19:15 GMT
< Connection: Keep-Alive
<
* Connection #0 to host smartproxypuppet.tld left intact
["puppet"]

Actions #3

Updated by Anonymous over 7 years ago

  • Status changed from New to Rejected

Looks like a bug in Katello here: https://github.com/Katello/katello/blob/KATELLO-3.2/app/models/katello/concerns/smart_proxy_extensions.rb#L51. If none of pulp-related features aren't available on smart-proxy puppet_path will end up recursively calling itself. I don't see a way to move the bug to Katello directly, I'll close it here and open a new one in Katello bug tracker.

Actions #4

Updated by Anonymous over 7 years ago

  • Related to Bug #18185: smart proxy refresh throws stackerror added
Actions #5

Updated by Klaas D over 7 years ago

Thanks

Actions #6

Updated by Anonymous over 7 years ago

FYI: The issue filed under Katello project is in "Related Issues" above.

Actions

Also available in: Atom PDF