Feature #8923
closedAbility to use Negotiate/Kerberos authentication to API and hammer
Description
The support for external authentication tracked via http://projects.theforeman.org/issues/5031 was mainly focused on WebUI authentication so far. It's desirable to bring the external authentication (meaning Negotiate, SAML, certificate-based) to API as well. At least for Negotiate (Kerberos), the primary blocker is that while it might be possible to configure the authentication in Apache, it would mean renegotiation upon every request, making the API operation slow.
The feature requests http://projects.theforeman.org/issues/8852 and http://projects.theforeman.org/issues/8016 talk about authentication using bearer tokens which by itself is mostly an internal capability. This RFE looks at using the external functionality which becomes possible when bearer tokens become available -- using Kerberos authentication once at the start of the session and then using the session token.
Updated by Jan Pazdziora almost 10 years ago
- Blocked by Feature #8016: Ability to use tokenized authentication to hammer in lieu of username/password in configuration file. added
Updated by Jan Pazdziora almost 10 years ago
- Related to Feature #8852: add token-based authentication feature for API added
Updated by Jan Pazdziora almost 10 years ago
- Blocks Tracker #5031: External authentication support added
Updated by Dominic Cleal over 9 years ago
- Related to Feature #11317: Hammer + external authentication via FreeIPA added
Updated by Ears Down over 9 years ago
Not sure if this is related, but I'm also struggling a bit with something that smells a lot like this topic. Setup is Freeipa 4.1 + Foreman 1.8.3 on Centos7 (fully patched).
- Kerberos setup
[root@puppet01 ~]# kdestroy [root@puppet01 ~]# kinit earsdown Password for earsdown@EXAMPLE.COM: [root@puppet01 ~]# klist Ticket cache: KEYRING:persistent:0:0 Default principal: earsdown@EXAMPLE.COM Valid starting Expires Service principal 09/07/2015 09:25:30 09/08/2015 09:25:27 krbtgt/EXAMPLE.COM@EXAMPLE.COM [root@puppet01 ~]#
- First attempt without session_id cookie or auth - which should (and does) fail
[root@puppet01 ~]# foreman_url=https://puppet01.example.com [root@puppet01 ~]# curl -vv -b /root/cookies.txt -c /root/cookies.txt -s $foreman_url/api/hosts | jq . * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM > GET /api/hosts HTTP/1.1 > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > < HTTP/1.1 401 Unauthorized < Date: Mon, 07 Sep 2015 09:29:41 GMT < Server: Apache/2.4.6 (CentOS) < Apipie-Checksum: 4dc90ee161b2f69d513ec4065c3fe739 < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: no-cache < X-Request-Id: 200e84a99465d4cb0454c48b93cf2be0 < X-Runtime: 0.022239 < X-Rack-Cache: miss < X-Powered-By: Phusion Passenger 4.0.18 * Added cookie request_method="" for domain puppet01.example.com, path /, expire 1 < Set-Cookie: request_method=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT < Status: 401 Unauthorized < Connection: close < Transfer-Encoding: chunked < Content-Type: application/json; charset=utf-8 < { [data not shown] * Closing connection 0 { "message": "Unable to authenticate user " }
- This time we're going to hit /users/extlogin with "--negotiate -u :", which will log us in and give us a session_id cookie.
[root@puppet01 ~]# curl -vv --negotiate -u : -b /root/cookies.txt -c /root/cookies.txt -s $foreman_url/users/extlogin * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM > GET /users/extlogin HTTP/1.1 > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > < HTTP/1.1 401 Unauthorized < Date: Mon, 07 Sep 2015 09:29:59 GMT < Server: Apache/2.4.6 (CentOS) < WWW-Authenticate: Negotiate < Content-Length: 120 < Connection: close < Content-Type: text/html; charset=iso-8859-1 < * Closing connection 0 * Issue another request to this URL: 'https://puppet01.example.com/users/extlogin' * About to connect() to puppet01.example.com port 443 (#1) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#1) * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM * Server auth using GSS-Negotiate with user '' > GET /users/extlogin HTTP/1.1 > Authorization: Negotiate <FILTERED> > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > < HTTP/1.1 302 Found < Date: Mon, 07 Sep 2015 09:29:59 GMT < Server: Apache/2.4.6 (CentOS) < WWW-Authenticate: Negotiate <FILTERED> < Strict-Transport-Security: max-age=631152000; includeSubdomains < X-Frame-Options: SAMEORIGIN < Content-Security-Policy: default-src 'self'; connect-src 'self' ws: wss:; font-src 'self'; frame-src 'self'; img-src 'self' *.gravatar.com data:; media-src 'self'; object-src 'self'; script-src 'unsafe-eval' 'unsafe-inline' 'self'; style-src 'unsafe-inline' 'self'; < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < X-Download-Options: noopen < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: no-cache, private < X-Request-Id: d0c49e289a4f945717f8445a1e1741bd < X-Runtime: 0.742713 < X-Rack-Cache: miss < X-Powered-By: Phusion Passenger 4.0.18 * Added cookie _session_id="09f9771105c20aebedad6a5fbbd8af13" for domain puppet01.example.com, path /, expire 0 < Set-Cookie: _session_id=09f9771105c20aebedad6a5fbbd8af13; path=/; secure; HttpOnly * Replaced cookie request_method="" for domain puppet01.example.com, path /, expire 1 < Set-Cookie: request_method=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT < Location: https://puppet01.example.com/hosts < Status: 302 Found < Connection: close < Transfer-Encoding: chunked < Content-Type: text/html; charset=utf-8 < * Closing connection 1 <html><body>You are being <a href="https://puppet01.example.com/hosts">redirected</a>.</body></html>
- Now, armed with our session_id cookie, we can hit /api/hosts and it works without the need for auth.
[root@puppet01 ~]# curl -vv -b /root/cookies.txt -c /root/cookies.txt -s "$foreman_url/api/hosts?per_page=999&search=proxyvpngw" | jq . * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM > GET /api/hosts?per_page=999&search=proxyvpngw HTTP/1.1 > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > Cookie: _session_id=09f9771105c20aebedad6a5fbbd8af13 > < HTTP/1.1 200 OK < Date: Mon, 07 Sep 2015 09:44:11 GMT < Server: Apache/2.4.6 (CentOS) < Foreman_version: 1.8.3 < Foreman_api_version: 1 < Apipie-Checksum: 4dc90ee161b2f69d513ec4065c3fe739 < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: must-revalidate, private, max-age=0 < X-Request-Id: d38e7d2816708bb51f08963c868c735a < X-Runtime: 0.184627 < X-Rack-Cache: miss < X-Powered-By: Phusion Passenger 4.0.18 * Replaced cookie request_method="" for domain puppet01.example.com, path /, expire 1 < Set-Cookie: request_method=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT < ETag: "239b2e5e3ac4a9827bba8c05e735214c" < Status: 200 OK < Connection: close < Transfer-Encoding: chunked < Content-Type: application/json; charset=utf-8 < { [data not shown] * Closing connection 0 [ { "host": { "operatingsystem_id": 1, "hostgroup_id": 6, "id": 17, "name": "proxyvpngw-i-1f7c8fc0.example.com" } }, { "host": { "operatingsystem_id": 1, "hostgroup_id": 6, "id": 14, "name": "proxyvpngw-i-790cfca6.example.com" } }, { "host": { "operatingsystem_id": 1, "hostgroup_id": 6, "id": 15, "name": "proxyvpngw-i-cb50a014.example.com" } } ]
- But when I try to delete a host (via DELETE /api/hosts/<hostid>), foreman freaks out and invalidates our credentials:
[root@puppet01 ~]# curl -vv -b /root/cookies.txt -c /root/cookies.txt -s -X DELETE $foreman_url/api/hosts/15 | jq . * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM > DELETE /api/hosts/15 HTTP/1.1 > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > Cookie: _session_id=09f9771105c20aebedad6a5fbbd8af13 > < HTTP/1.1 401 Unauthorized < Date: Mon, 07 Sep 2015 09:48:36 GMT < Server: Apache/2.4.6 (CentOS) < Apipie-Checksum: 4dc90ee161b2f69d513ec4065c3fe739 < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: no-cache < X-Request-Id: b81193ba08de9cbd609957dfc9a2b918 < X-Runtime: 0.024028 < X-Rack-Cache: invalidate, pass < X-Powered-By: Phusion Passenger 4.0.18 * Replaced cookie _session_id="d57d0690d37af112761afda6695ac4a1" for domain puppet01.example.com, path /, expire 0 < Set-Cookie: _session_id=d57d0690d37af112761afda6695ac4a1; path=/; secure; HttpOnly * Replaced cookie request_method="DELETE" for domain puppet01.example.com, path /, expire 0 < Set-Cookie: request_method=DELETE; path=/ < Status: 401 Unauthorized < Connection: close < Transfer-Encoding: chunked < Content-Type: application/json; charset=utf-8 < { [data not shown] * Closing connection 0 { "message": "Unable to authenticate user " }
Interestingly, it throws a 401 but doesn't offer a WWW-Authenticate header.
- ...and here's what appears in the production.log:
Started DELETE "/api/hosts/15" for 10.239.12.20 at 2015-09-07 09:48:36 +0000 2015-09-07 09:48:36 [I] Processing by Api::V1::HostsController#destroy as JSON 2015-09-07 09:48:36 [I] Parameters: {"apiv"=>"v1", "id"=>"15"} 2015-09-07 09:48:36 [W] WARNING: Can't verify CSRF token authenticity 2015-09-07 09:48:36 [I] Rendered api/v1/errors/unauthorized.json.rabl (0.6ms) 2015-09-07 09:48:36 [I] Filter chain halted as :authorize rendered or redirected 2015-09-07 09:48:36 [I] Completed 401 Unauthorized in 13ms (Views: 1.8ms | ActiveRecord: 3.0ms)
Foreman does actually invalidate the previously-working credentials - i.e., if I now try to HTTP GET /api/hosts I'll get the same "unable to authenticate user " error - I would have to re-auth against /users/extlogin and get a new session_id.
Calling HTTP DELETE on /api/hosts DOES work if I pass "-u earsdown:<password>", ie basic auth with the credentials of a FreeIPA user...BUT it takes ages because I can see foreman is re-authenticating me against LDAP even though I'm supplying a valid session_id cookie. When passing "-u earsdown:<password>" (basic auth) for other HTTP GET APIs, it's quick, and it doesn't re-auth me against LDAP - which means it must be using the session_id cookie.
- Here's the output of what happens when calling HTTP DELETE on /api/hosts using basic auth creds (and a previously working session_id - which gets ignored) of a FreeIPA user:
[root@puppet01 ~]# curl -vv -u earsdown:<password> -b /root/cookies.txt -c /root/cookies.txt -s -X DELETE $foreman_url/api/v2/hosts/15 | jq . * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM * Server auth using Basic with user 'earsdown' > DELETE /api/v2/hosts/15 HTTP/1.1 > Authorization: Basic <FILTERED> > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > Cookie: _session_id=7f0d4f8d07225ef1e0843f6419e12990 > < HTTP/1.1 200 OK < Date: Mon, 07 Sep 2015 11:00:57 GMT < Server: Apache/2.4.6 (CentOS) < Foreman_version: 1.8.3 < Foreman_api_version: 2 < Apipie-Checksum: 4dc90ee161b2f69d513ec4065c3fe739 < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: max-age=0, private, must-revalidate < X-Request-Id: bddf2c807912140b391694e55e74dd01 < X-Runtime: 5.440107 < X-Rack-Cache: invalidate, pass < X-Powered-By: Phusion Passenger 4.0.18 * Replaced cookie request_method="DELETE" for domain puppet01.example.com, path /, expire 0 < Set-Cookie: request_method=DELETE; path=/ < ETag: "e95300684778ae24bef975e168f8756b" < Status: 200 OK < Connection: close < Transfer-Encoding: chunked < Content-Type: application/json; charset=utf-8 < { [data not shown] * Closing connection 0 { "uuid": null, "use_image": null, "updated_at": "2015-09-04T10:41:02Z", "source_file_id": null, "serial": null, "root_pass": "", "realm_id": null, "puppet_status": 6, "puppet_proxy_id": 1, "installed_at": null, "image_id": null, "image_file": "", "id": 15, "hostgroup_id": 6, "grub_pass": "", "environment_id": 1, "enabled": true, "architecture_id": 1, "build": false, "certname": "proxyvpngw-i-cb50a014.example.com", "comment": null, "compute_profile_id": null, "compute_resource_id": null, "created_at": "2015-09-04T10:01:58Z", "disk": null, "last_compile": "2015-09-04T10:39:44Z", "last_freshcheck": null, "last_report": "2015-09-04T10:39:32Z", "location_id": 2, "managed": false, "medium_id": null, "model_id": 1, "name": "proxyvpngw-i-cb50a014.example.com", "operatingsystem_id": 1, "organization_id": 4, "otp": null, "owner_id": 5, "owner_type": "User", "provision_method": "build", "ptable_id": null, "puppet_ca_proxy_id": null } [root@puppet01 ~]# curl -vv -u earsdown:<password> -b /root/cookies.txt -c /root/cookies.txt -s -X DELETE $foreman_url/api/v2/hosts/15 | jq . * About to connect() to puppet01.example.com port 443 (#0) * Trying 10.239.12.20... * Connected to puppet01.example.com (10.239.12.20) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=puppet01.example.com,O=EXAMPLE.COM * start date: Aug 19 12:19:46 2015 GMT * expire date: Aug 19 12:19:46 2017 GMT * common name: puppet01.example.com * issuer: CN=Certificate Authority,O=EXAMPLE.COM * Server auth using Basic with user 'earsdown' > DELETE /api/v2/hosts/15 HTTP/1.1 > Authorization: Basic <FILTERED> > User-Agent: curl/7.29.0 > Host: puppet01.example.com > Accept: */* > Cookie: request_method=DELETE; _session_id=7f0d4f8d07225ef1e0843f6419e12990 > < HTTP/1.1 404 Not Found < Date: Mon, 07 Sep 2015 11:01:21 GMT < Server: Apache/2.4.6 (CentOS) < Foreman_version: 1.8.3 < Foreman_api_version: 2 < Apipie-Checksum: 4dc90ee161b2f69d513ec4065c3fe739 < X-UA-Compatible: IE=Edge,chrome=1 < Cache-Control: no-cache < X-Request-Id: 91f18369a51c901f045ef64e2f981170 < X-Runtime: 4.021407 < X-Rack-Cache: invalidate, pass < X-Powered-By: Phusion Passenger 4.0.18 * Replaced cookie request_method="DELETE" for domain puppet01.example.com, path /, expire 0 < Set-Cookie: request_method=DELETE; path=/ < Status: 404 Not Found < Connection: close < Transfer-Encoding: chunked < Content-Type: application/json; charset=utf-8 < { [data not shown] * Closing connection 0 { "error": { "message": "Resource host not found by id '15'" } }
Btw, this behavior is reproducible on /api/v2 URLs as well.
My use-case is a host-retire script which cleans up terminated hosts in foreman (and freeipa, and puppetdb, and nagios, ...you get the idea) based on what's running in $cloud. The script will run via cron by unprivileged user account, and the only credential I want to give it is keytab linked to a FreeIPA user principal... thus... I'm keen to get this working :-)
Ideas?
Updated by Dominic Cleal over 9 years ago
Ears Down wrote:
Not sure if this is related, but I'm also struggling a bit with something that smells a lot like this topic. Setup is Freeipa 4.1 + Foreman 1.8.3 on Centos7 (fully patched).
It would be better to take this to a new ticket or the mailing list, as this is about adding support to Hammer.
- This time we're going to hit /users/extlogin with "--negotiate -u :", which will log us in and give us a session_id cookie.
I'd have thought you could use this against the API, but I haven't tested. (#11317 is about this, but I can't tell if it's PEBKAC or not working.)
- But when I try to delete a host (via DELETE /api/hosts/<hostid>), foreman freaks out and invalidates our credentials:
[...]Interestingly, it throws a 401 but doesn't offer a WWW-Authenticate header.
- ...and here's what appears in the production.log:
[...]Foreman does actually invalidate the previously-working credentials - i.e., if I now try to HTTP GET /api/hosts I'll get the same "unable to authenticate user " error - I would have to re-auth against /users/extlogin and get a new session_id.
Calling HTTP DELETE on /api/hosts DOES work if I pass "-u earsdown:<password>", ie basic auth with the credentials of a FreeIPA user...BUT it takes ages because I can see foreman is re-authenticating me against LDAP even though I'm supplying a valid session_id cookie. When passing "-u earsdown:<password>" (basic auth) for other HTTP GET APIs, it's quick, and it doesn't re-auth me against LDAP - which means it must be using the session_id cookie.
This behaviour is intended, implemented in #4895. A session ID is only meant to be used by a web browser with web-based forms and not really for scripted API use. The reason support for session IDs are there is to support web frameworks written against Foreman's API.
As such, the CSRF support is kicking in to prevent you making any changes to Foreman without knowing that it came from a trusted form, or authentication was explicitly passed - else a browser with an active session could be tricked into making changes in Foreman.
The solution is probably just to ensure GSSAPI etc works against the API without use of session IDs etc.
Updated by Dominic Cleal over 8 years ago
- Has duplicate Feature #14633: Support for Kerberos Authentication added
Updated by Tomáš Strachota about 8 years ago
- Category set to Hammer core
- Target version set to 1.10.1
Updated by Tomáš Strachota about 8 years ago
- Target version changed from 1.10.1 to 115
Updated by Shira Maximov about 5 years ago
- Triaged set to No
- In Kanboard set to yes
Updated by Shira Maximov almost 5 years ago
- Triaged changed from No to Yes
- Team Backlog Hammer added
- Team Backlog deleted (
Marek)
Updated by Ondřej Ezr almost 4 years ago
- Bugzilla link changed from 1264161 to 1266407
Updated by Ondřej Ezr almost 4 years ago
- Bugzilla link changed from 1266407 to 1264161
Updated by Ondřej Ezr almost 4 years ago
- Pull request https://github.com/theforeman/hammer-cli-foreman/pull/555 added
Updated by Tomer Brisker almost 4 years ago
- Status changed from New to Ready For Testing
Updated by The Foreman Bot over 2 years ago
- Fixed in Releases hammer-cli-foreman-3.3.0 added
Updated by Ondřej Ezr over 2 years ago
- Status changed from Ready For Testing to Closed
Applied in changeset hammer-cli-foreman|da21f130852f69e310babab19f5fda1c4d39d216.