Project

General

Profile

Bug #10581

UTF8 encoding error on realm update with freeipa

Added by Stephen Benjamin over 5 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Realm
Target version:
-
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

E, [2015-05-21T14:11:32.388938 #15389] ERROR -- : "\x82" from ASCII-8BIT to UTF-8

TommyTheKid ran into it when updating a host's hostgroup, not sure what the underlying problem is. net-ldap had a similar issue before, so maybe we need to check the conversion for the data we send to FreeIPA's XMLRPC API.


Related issues

Related to Foreman Remote Execution - Bug #12456: Handilng non-utf characters ssh exceptionsClosed2015-11-12

Associated revisions

Revision e7c224d9 (diff)
Added by Ivan Necas over 4 years ago

Fixes #10581 - ensure string from IPA XML-RPM is marked as UTF8

Otherwise, Ruby considers them being ASCII-8BIT, which leads to ugly
errors when trying to convert to UTF8.

History

#1 Updated by Dominic Cleal over 5 years ago

  • Project changed from Foreman to Smart Proxy
  • Category changed from Realm to Realm

Tentatively moving, think that's a smart proxy log file.

#2 Updated by Stephen Benjamin over 5 years ago

Oops, it is, you're right.

#3 Updated by Tommy McNeely over 5 years ago

Correct. The error log pasted above is from the smart proxy. The foreman log just indicates the 400 error from the smart proxy. Here it is in "Debug" mode (probably a lot more useful)

104.236.26.171 - - [22/May/2015 11:06:05] "POST /realm/IPA.LARK-IT.COM/ HTTP/1.1" 200 481 2.2598
D, [2015-05-22T11:19:31.671434 #16599] DEBUG -- : verifying remote client 104.236.26.171 against trusted_hosts ["lark-foreman-01.lark-it.com"]
I, [2015-05-22T11:19:31.673762 #16599]  INFO -- : freeipa: realm keytab is '/etc/foreman-proxy/freeipa.keytab' and using principal 'realm-proxy@IPA.LARK-IT.COM'
I, [2015-05-22T11:19:31.674824 #16599]  INFO -- : freeipa: realm IPA.LARK-IT.COM
I, [2015-05-22T11:19:31.675127 #16599]  INFO -- : freeipa: server is https://lark-ipa-3.lark-it.com/ipa/xml
I, [2015-05-22T11:19:31.676674 #16599]  INFO -- : Requesting credentials for Kerberos principal realm-proxy@IPA.LARK-IT.COM using keytab /etc/foreman-proxy/freeipa.keytab
D, [2015-05-22T11:19:31.711766 #16599] DEBUG -- : Kerberos credential cache initialised with principal: realm-proxy@IPA.LARK-IT.COM
E, [2015-05-22T11:19:33.032725 #16599] ERROR -- : "\x82" from ASCII-8BIT to UTF-8
D, [2015-05-22T11:19:33.033390 #16599] DEBUG -- : /usr/share/gems/gems/json-1.7.7/lib/json/common.rb:286:in `encode'
/usr/share/gems/gems/json-1.7.7/lib/json/common.rb:286:in `generate'
/usr/share/gems/gems/json-1.7.7/lib/json/common.rb:286:in `pretty_generate'
/usr/share/foreman-proxy/modules/realm/freeipa.rb:110:in `create'
/usr/share/foreman-proxy/modules/realm/realm_api.rb:28:in `block in <class:Api>'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:1293:in `call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:1293:in `block in compile!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:860:in `[]'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:860:in `block (3 levels) in route!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:876:in `route_eval'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:860:in `block (2 levels) in route!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:897:in `block in process_route'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:895:in `catch'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:895:in `process_route'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:859:in `block in route!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:858:in `each'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:858:in `route!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:963:in `block in dispatch!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `block in invoke'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `catch'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `invoke'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:960:in `dispatch!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:794:in `block in call!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `block in invoke'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `catch'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:946:in `invoke'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:794:in `call!'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:780:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/commonlogger.rb:33:in `call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:161:in `call'
/usr/share/foreman-proxy/lib/proxy/log.rb:35:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/xss_header.rb:27:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/path_traversal.rb:16:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/json_csrf.rb:17:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/base.rb:48:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/base.rb:48:in `call'
/usr/share/gems/gems/rack-protection-1.3.2/lib/rack/protection/xss_header.rb:27:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/nulllogger.rb:9:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/showexceptions.rb:21:in `call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:124:in `call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:1417:in `block in call'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:1499:in `synchronize'
/usr/share/gems/gems/sinatra-1.3.5/lib/sinatra/base.rb:1417:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in `block in call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `each'
/usr/share/gems/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
/usr/share/gems/gems/rack-1.5.2/lib/rack/handler/webrick.rb:60:in `service'
/usr/share/ruby/webrick/httpserver.rb:138:in `service'
/usr/share/ruby/webrick/httpserver.rb:94:in `run'
/usr/share/ruby/webrick/server.rb:295:in `block in start_thread'
104.236.26.171 - - [22/May/2015 11:19:33] "POST /realm/IPA.LARK-IT.COM/ HTTP/1.1" 400 31 1.3654

#4 Updated by Adam Ruzicka over 5 years ago

I've encountered the same error several times with puppet, the cause was my system's locale wasn't set properly and defaulted to something non-UTF8. Maybe changing the system-wide or smart-proxy user's locale to its UTF8 variant might help.

#5 Updated by Dmitri Dolguikh over 5 years ago

Probably should document the locale/character encoding we expect.

#6 Updated by Bryan Kearney about 5 years ago

  • Bugzilla link set to 1281687

#7 Updated by Brad Buckingham over 4 years ago

#8 Updated by Brad Buckingham over 4 years ago

#9 Updated by Brad Buckingham over 4 years ago

#10 Updated by Brad Buckingham over 4 years ago

#11 Updated by Ivan Necas over 4 years ago

  • Related to Bug #12456: Handilng non-utf characters ssh exceptions added

#12 Updated by Ivan Necas over 4 years ago

We have seen similar issue in remote execution, the problem was we were not indicating properly that the
external string was UTF8 (found this https://www.ruby-forum.com/topic/463233 related to the XML-RPC).

The solution we could go with (and was what both and people in the tread did) was marking the
string as UTF8 by `force_encoding('ASCII-8BIT')`

The easiest way to simulate this is `JSON.pretty_generate("žluťoučký".force_encoding('ASCII-8BIT'))`

#13 Updated by Ivan Necas over 4 years ago

Also, I expect this issue is with ruby 1.9+, right?

#14 Updated by Ivan Necas over 4 years ago

  • Status changed from New to Assigned
  • Assignee set to Ivan Necas
  • Target version set to 1.7.0

#15 Updated by The Foreman Bot over 4 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/smart-proxy/pull/452 added

#16 Updated by Ivan Necas over 4 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#17 Updated by Dominic Cleal over 4 years ago

  • Legacy Backlogs Release (now unused) set to 175

#18 Updated by Greg Sutcliffe over 2 years ago

  • Target version deleted (1.12.2)

Also available in: Atom PDF