Project

General

Profile

Bug #18185

smart proxy refresh throws stackerror

Added by Dmitri Dolguikh about 2 years ago. Updated 8 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Foreman Proxy Content
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Team Backlog:
Fixed in Releases:
Found in Releases:

Description

Hi,
I recently upgraded my main foreman/katello server to 3.2.3. Afterwards I reinstalled my secondary puppet smartproxy. Now everytime I refresh the smart proxy there is an error "SystemStackError (stack level too deep)". I have other smart proxies that work fine. Even though it throws an error the refresh seems to work because I can turn off/on features.

CLI:

  1. hammer proxy refresh-features --name smartproxypuppet.tld
    Refresh of smart proxy features failed:
    500 Internal Server Error
    Production Log:

2017-01-20 16:13:21 7344a677 [app] [I] Started GET "/api/smart_proxies?search=name+%3D+%22smartproxypuppet.tld%22" for 1.1.1.20 at 2017-01-20 16:13:21 +0100
2017-01-20 16:13:21 7344a677 [app] [I] Processing by Api::V2::SmartProxiesController#index as JSON
2017-01-20 16:13:21 7344a677 [app] [I] Parameters: {"search"=>"name = \"smartproxypuppet.tld\"", "apiv"=>"v2", "smart_proxy"=>{}}
2017-01-20 16:13:21 7344a677 [app] [I] Authorized user admin(Admin User)
2017-01-20 16:13:21 7344a677 [app] [I] Rendered api/v2/smart_proxies/index.json.rabl within api/v2/layouts/index_layout (44.5ms)
2017-01-20 16:13:21 7344a677 [app] [I] Completed 200 OK in 113ms (Views: 60.4ms | ActiveRecord: 12.3ms)
2017-01-20 16:13:22 713e4c44 [app] [I] Started PUT "/api/smart_proxies/2/refresh" for 1.1.1.20 at 2017-01-20 16:13:22 +0100
2017-01-20 16:13:22 713e4c44 [app] [I] Processing by Api::V2::SmartProxiesController#refresh as JSON
2017-01-20 16:13:22 713e4c44 [app] [I] Parameters: {"apiv"=>"v2", "id"=>"2", "smart_proxy"=>{}}
2017-01-20 16:13:22 713e4c44 [app] [I] Authorized user admin(Admin User)
2017-01-20 16:14:00 713e4c44 [app] [I] Completed 500 Internal Server Error in 38278ms (ActiveRecord: 1116.1ms)
2017-01-20 16:14:00 713e4c44 [app] [F] | SystemStackError (stack level too deep): | app/models/concerns/audit_extensions.rb:56:in `ensure_auditable_and_associated_name' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save'
[...many many many more...] | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:51:in `puppet_path' | katello (3.2.1.1) app/lib/katello/lazy_accessor.rb:80:in `save' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:60:in `update_puppet_path' | katello (3.2.1.1) app/models/katello/concerns/smart_proxy_extensions.rb:66:in `refresh_with_puppet_path' | foreman_remote_execution (1.2.2) app/models/concerns/foreman_remote_execution/smart_proxy_extensions.rb:21:in `refresh_with_remote_execution' | app/controllers/api/v2/smart_proxies_controller.rb:60:in `refresh' | app/controllers/api/v2/base_controller.rb:152:in `disable_json_root' | app/controllers/concerns/application_shared.rb:14:in `set_timezone' | app/models/concerns/foreman/thread_session.rb:32:in `clear_thread' | lib/middleware/catch_json_parse_errors.rb:9:in `call' | lib/middleware/tagged_logging.rb:18:in `call'
Foreman Installer for Capsule:

foreman-installer --scenario capsule\
--capsule-parent-fqdn "foreman.tld"\
--foreman-proxy-register-in-foreman "true"\
--foreman-proxy-foreman-base-url "https://foreman.tld"\
--foreman-proxy-trusted-hosts "foreman.tld"\
--foreman-proxy-trusted-hosts "smartproxypuppet.tld"\
--foreman-proxy-oauth-consumer-key "KEY"\
--foreman-proxy-oauth-consumer-secret "KEY"\
--capsule-pulp-oauth-secret "KEY"\
--capsule-certs-tar "~/smartproxypuppet.tld-certs.tar"\
--capsule-puppet-server-implementation puppetserver\
--capsule-puppet-ca-proxy "https://foreman.tld"\
--foreman-proxy-puppetca false\
--foreman-proxy-templates false\
--foreman-proxy-plugin-remote-execution-ssh-enabled false\
--no-enable-foreman-proxy-plugin-pulp\
--foreman-proxy-plugin-pulp-enabled false

I need to disable pulp in /etc/foreman-proxy/settings.d/pulp.yml afterwards and install httpd before running installer but for that I'll open a separate bug :)
Greetings
Klaas


Related issues

Related to Smart Proxy - Bug #18179: smart proxy refresh throws stackerrorRejected2017-01-20
Related to Katello - Feature #16456: Update Katello to synchronize puppet content to capsule based on capsules puppet path.Closed2016-09-06

Associated revisions

Revision 1e8efa1e (diff)
Added by Klaas Demter about 2 years ago

Fixes #18185 - smart proxy refresh throws stackerror

puppet_path is nil when capsule doesn't have pulp

History

#1 Updated by Dmitri Dolguikh about 2 years ago

Hi,
on the puppet smart proxy there are logs like this:

I, [2017-01-20T17:15:19.681161 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:15:19.766234 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0092

I, [2017-01-20T17:15:19.851935 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:15:19.932268 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:19 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:15:20.014565 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:15:20 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.056454 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.126090 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.220527 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.309505 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.400620 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.485953 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0003

I, [2017-01-20T17:16:45.575194 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.667581 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

I, [2017-01-20T17:16:45.750023 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.835060 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0004

I, [2017-01-20T17:16:45.907984 #50530] INFO -- : 1.1.1.20 - - [20/Jan/2017:17:16:45 +0100] "GET /features HTTP/1.1" 200 10 0.0005

(lots of them) no errors
the proxy log on the foreman server is not showing anything while this request is happening

the request itsself seems to work:

  1. curl -vks --cacert /etc/foreman-proxy/foreman_ssl_ca.pem --cert /etc/foreman-proxy/foreman_ssl_cert.pem --key /etc/foreman-proxy/foreman_ssl_key.pem "https://smartproxypuppet.tld:9090/features"
    • About to connect() to smartproxypuppet.tld port 9090 (#0)
    • Trying 10.165.1.22...
    • Connected to smartproxypuppet.tld (10.165.1.22) port 9090 (#0)
    • Initializing NSS with certpath: sql:/etc/pki/nssdb
    • skipping SSL peer certificate verification
    • NSS: client certificate from file
    • subject: CN=foreman.tld,OU=FOREMAN_PROXY,O=FOREMAN,ST=State,C=TLD
    • start date: Jun 15 15:58:33 2016 GMT
    • expire date: Jun 17 15:58:33 2036 GMT
    • common name: foreman.tld
    • issuer: CN=foreman.tld,OU=Name,O=Name,L=Munich,ST=State,C=TLD
    • SSL connection using TLS_RSA_WITH_AES_128_GCM_SHA256
    • Server certificate:
    • subject: CN=smartproxypuppet.tld,OU=SMART_PROXY,O=FOREMAN,ST=North Carolina,C=US
    • start date: Jun 24 10:23:07 2016 GMT
    • expire date: Jun 26 10:23:07 2036 GMT
    • common name: smartproxypuppet.tld
    • issuer: CN=foreman.tld,OU=Name,O=Name,L=Munich,ST=State,C=TLD

GET /features HTTP/1.1
User-Agent: curl/7.29.0
Host: smartproxypuppet.tld:9090
Accept: */*

< HTTP/1.1 200 OK
< Content-Type: application/json;charset=utf-8
< Content-Length: 10
< X-Content-Type-Options: nosniff
< Server: WEBrick/1.3.1 (Ruby/2.0.0/2015-12-16) OpenSSL/1.0.1e
< Date: Fri, 20 Jan 2017 16:19:15 GMT
< Connection: Keep-Alive
<
  • Connection #0 to host smartproxypuppet.tld left intact
    ["puppet"]

#2 Updated by Dmitri Dolguikh about 2 years ago

This looks like a bug in here: https://github.com/Katello/katello/blob/KATELLO-3.2/app/models/katello/concerns/smart_proxy_extensions.rb#L51. If none of pulp-related features aren't available on smart-proxy puppet_path will end up recursively calling itself.

#3 Updated by Dmitri Dolguikh about 2 years ago

  • Related to Bug #18179: smart proxy refresh throws stackerror added

#4 Updated by Klaas D about 2 years ago

I tried to look into the issue a little, do all smart proxies now have a "puppet_path"? So you can't have a dedicated puppet/dhcp/dns smart proxy anymore?

That puppet_path is only used for pulp to sync modules if you use Katello(pulp) for puppet modules, right?

#5 Updated by Klaas D about 2 years ago

  • Related to Feature #16456: Update Katello to synchronize puppet content to capsule based on capsules puppet path. added

#6 Updated by The Foreman Bot about 2 years ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/6565 added

#7 Updated by Klaas D about 2 years ago

I think that is a quick fix for the problem, at least if my assumptions are correct :D

#8 Updated by Anonymous about 2 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#9 Updated by Justin Sherrill about 2 years ago

  • Category set to Foreman Proxy Content
  • Legacy Backlogs Release (now unused) set to 188

#10 Updated by Klaas D almost 2 years ago

  • Legacy Backlogs Release (now unused) changed from 188 to 226

This one was not in 3.3.0 or 3.3.1 :)

#11 Updated by Klaas D almost 2 years ago

Scratch that, maybe I was in the wrong repository, it was included in 3.3.1 sorry for the noise ;)
https://github.com/Katello/katello/blob/KATELLO-3.3/app/models/katello/concerns/smart_proxy_extensions.rb

Also available in: Atom PDF