Project

General

Profile

Actions
Copy link

Bug #23985

open

"406 Not Acceptable" when attempting to import salt states

Added by Ben Howell over 6 years ago. Updated over 5 years ago.

Status:
Assigned
Priority:
Normal
Assignee:
Greg Sutcliffe
Category:
-
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

ERF12-4701 [ProxyAPI::ProxyException]: Unable to fetch Salt states list ([ProxyAPI::ProxyException]: ERF12-7301 [ProxyAPI::ProxyException]: Unable to fetch Salt environments list ([RestClient::NotAcceptable]: 406 Not Acceptable) for proxy https://foreman-rutabaga.thecitybase.net:8443/salt/) for proxy https://foreman-rutabaga.thecitybase.net:8443/salt/

salt master:

autosign_file: /etc/salt/autosign.conf

external_auth:
  pam:
    saltuser:
      - '@runner'

rest_cherrypy:
  port: 9191
  host: 0.0.0.0
  ssl_crt: /etc/puppetlabs/puppet/ssl/certs/foreman-rutabaga.thecitybase.net.pem
  ssl_key: /etc/puppetlabs/puppet/ssl/private_keys/foreman-rutabaga.thecitybase.net.pem

master_tops:
  ext_nodes: /usr/bin/foreman-node

state_verbose: False

file_roots:
  base:
    - /srv/salt

pillar_roots:
  base:
    - /srv/pillar/base

ext_pillar:
  - puppet: /usr/bin/foreman-node

pillar_safe_render_error: True

/etc/salt/foreman.yml:

---
:proto: https
:host: foreman-rutabaga.thecitybase.net
:port: 443
# if using http with ssl certificates
:ssl_ca: "/etc/puppetlabs/puppet/ssl/ca/ca_crt.pem" 
:ssl_cert: "/etc/puppetlabs/puppet/ssl/certs/foreman-rutabaga.thecitybase.net.pem" 
:ssl_key: "/etc/puppetlabs/puppet/ssl/private_keys/foreman-rutabaga.thecitybase.net.pem" 
# if using http with username and password instead of https with certicates
#:username: saltuser
#:password: saltpassword
:timeout:  10
:salt:  /usr/bin/salt
:upload_grains:  true

/etc/foreman-proxy/settings.d/salt.yml:

---
:enabled: https
:autosign_file: /etc/salt/autosign.conf
:salt_command_user: root
# Some features require using the Salt API - such as listing environments and retrieving state info
:use_api: true
:api_url: https://foreman-rutabaga.thecitybase.net:9191
:api_auth: pam
:api_username: saltuser
:api_password: ****

/var/log/foreman/production.log:

2018-06-18T13:22:24 912aa695 [app] [I] Completed 200 OK in 22ms (Views: 0.3ms | ActiveRecord: 2.5ms)
2018-06-18T13:22:26 70ad08c5 [app] [W] ProxyAPI operation FAILED
 | ProxyAPI::ProxyException: ERF12-4701 [ProxyAPI::ProxyException]: Unable to fetch Salt states list ([ProxyAPI::ProxyException]: ERF12-7301 [ProxyAPI::ProxyException]: Unable to fetch Salt environments list ([RestClient::NotAcceptable]: 406 Not Acceptable) for proxy https://foreman-rutabaga.thecitybase.net:8443/salt/) for proxy https://foreman-rutabaga.thecitybase.net:8443/salt/
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_salt-10.0.0/app/lib/proxy_api/salt.rb:43:in `rescue in states_list'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_salt-10.0.0/app/lib/proxy_api/salt.rb:35:in `states_list'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_salt-10.0.0/app/controllers/foreman_salt/state_importer.rb:16:in `fetch_states_from_proxy'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_salt-10.0.0/app/controllers/foreman_salt/salt_modules_controller.rb:61:in `import'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:186:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:108:in `block in run_callbacks'
 | /usr/share/foreman/app/controllers/concerns/application_shared.rb:15:in `set_timezone'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.5.0/lib/audited/sweeper.rb:14:in `around'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.5.0/lib/audited/sweeper.rb:14:in `around'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:135:in `run_callbacks'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/rescue.rb:20:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `block in instrument'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `instrument'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.1.4/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:124:in `process'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionview-5.1.4/lib/action_view/rendering.rb:30:in `process'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:189:in `dispatch'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:253:in `dispatch'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:31:in `serve'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.8/lib/apipie/static_dispatcher.rb:65:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.8/lib/apipie/extractor/recorder.rb:136:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.8/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/etag.rb:25:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/conditional_get.rb:25:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/head.rb:12:in `call'
 | /usr/share/foreman/lib/middleware/session_safe_logging.rb:17:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:232:in `context'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:226:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/cookies.rb:613:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:97:in `run_callbacks'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:36:in `call_app'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:26:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 | /usr/share/foreman/lib/middleware/tagged_logging.rb:18:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/request_id.rb:25:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/method_override.rb:22:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/runtime.rb:22:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.1.4/lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/executor.rb:12:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/static.rb:125:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/sendfile.rb:111:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/secure_headers-3.7.3/lib/secure_headers/middleware.rb:12:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/urlmap.rb:68:in `block in call'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `each'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.5/lib/rack/urlmap.rb:53:in `call'
 | /usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'
 | /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:152:in `accept_and_process_next_request'
 | /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:113:in `main_loop'
 | /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:416:in `block (3 levels) in start_threads'
 | /usr/lib/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Salt verion:

Salt Version:
           Salt: 2018.3.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-116-generic
         system: Linux
        version: Ubuntu 16.04 xenial

Foreman version 1.17.1

Documentation followed: https://theforeman.org/plugins/foreman_salt/7.0/index.html

Actions
Copy link
 #1

Updated by Anonymous over 6 years ago

  • Project changed from Foreman to Salt
Actions
Copy link
 #2

Updated by Ben Howell over 6 years ago

Potential fix:

apt-get autoremove --purge python-cherrypy3
apt-get install python-pip python-routes python-repoze.lru python-webob
pip uninstall cherrypy #this might fail because it was installed with apt
pip install cherrypy==3.2.3

/etc/salt/master:

external_auth:
  pam:
    saltuser:
      - '@runner'
      - .*

/etc/foreman-proxy/settings.d/salt.yml:

:salt_command_user: saltuser

Summary: downgrading cherrypy to 3.2.3 (default install is 3.5.0), including '.*' in the saltuser permissions, and changing the salt_command_user to match the API username. I'm now able to import states and environments via the Foreman interface with no errors. If 'pip install cherrypy==3.2.3' tries to install another version of cherrypy, there might be a pip cache directory somewhere (e.g. /root/.cache/pip) that needs to be removed.

Actions
Copy link
 #3

Updated by Ben Howell over 6 years ago

Apparently it actually does need

:salt_command_user: root

It appears that the issue was with cherrypy itself, and the configuration of external_auth, which requires both '@runner' and '.*' for the saltuser. The foreman-salt plugin documentation should be updated to reflect this. I don't know the status of cherrypy 3.5.0 in Ubuntu 16.04 though, that's another issue that causes TLS/SSL errors. I think it's being addressed by the cherrypy maintainers.

Actions
Copy link
 #4

Updated by Greg Sutcliffe over 6 years ago

  • Status changed from New to Feedback
  • Assignee set to Greg Sutcliffe
  • Triaged set to No

Is it fair to say we can close this then Ben? Or do we need a docs update to mention this upstream bug?

Actions
Copy link
 #5

Updated by Ben Howell over 6 years ago

It may be worth noting in documentation, as the current 7.0 documentation at "@runner" is the minimum level of permissions, while ".*" is needed to run highstates and generate reports, from my findings.

Actions
Copy link
 #6

Updated by Greg Sutcliffe over 6 years ago

  • Status changed from Feedback to Assigned

Fair enough, do you want to take a crack at it? You did the debugging work, you should get the credit in the git repo :)

If not, I'll take a look at it later. By the way, can you let me know your GitHub handle (if you have one!), I'd like your thoughts on a couple of other patches :D

Actions
Copy link
 #7

Updated by Jason C. Hammons over 5 years ago

Thank you for the tip!

Actions
Copy link

Also available in: Atom PDF