Bug #20592
opendifficult to dectect goferd plugin errors from bad ca cert
Description
OS: RHEL7
katello-agent-2.7.0-1
I ran into this issue when my AMQ certs were reset. goferd (or more accurately katelloplugin.py) sat in a loop being unable to connect, but since the service was "running" I had no idea this occurred and it was difficult to locate.
If the service died on fatal errors (bad certs, network timeouts, etc) like this it'd have been easier to find.
Additionally, it'd be great if goferd would have some mechanism to report it's working state. Right now the best I can do is grep logs to see if it's been recently connected. But that's beside the point.
I believe the issue is the try/except Exception hides too many possible exception cases that really should be fatal.
Excerpt from /usr/lib/gofer/plugins/katelloplugin.py
:
@initializer def init_plugin(): """ Initialize the plugin. Called (once) immediately after the plugin is loaded. - setup path monitoring. - validate registration. If registered: - setup plugin configuration. - send an initial repository enabled report. """ path = ConsumerIdentity.certpath() path_monitor.add(path, certificate_changed) path_monitor.add(REPOSITORY_PATH, send_enabled_report) path_monitor.start() while True: try: validate_registration() if registered: update_settings() send_enabled_report() # DONE break except Exception, e: log.warn(str(e)) sleep(60)
Log file excerpt:
Jul 18 07:59:57 test-foo-01 goferd[7785]: [INFO][MainThread] rhsm.connection:830 - Connection built: host=tst-inf-003 port=8443 handler=/rhsm auth=identity_cert ca_dir=/etc/rhsm/ca/ verify=False Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - tlsv1 alert unknown ca Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - Traceback (most recent call last): Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib/gofer/plugins/katelloplugin.py", line 201, in validate_registration Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - consumer = uep.getConsumer(consumer_id) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 1064, in getConsumer Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - return self.conn.request_get(method) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 694, in request_get Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - return self._request("GET", method) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 584, in _request Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - conn.request(request_type, handler, body=body, headers=headers) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/httplib.py", line 1017, in request Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - self._send_request(method, url, body, headers) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - self.endheaders(body) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - self._send_output(message_body) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - self.send(msg) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/httplib.py", line 826, in send Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - self.connect() Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 250, in connect Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - sock.connect((self.host, self.port)) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 185, in connect Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - ret = self.connect_ssl() Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - File "/usr/lib64/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 178, in connect_ssl Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - return m2.ssl_connect(self.ssl, self._timeout) Jul 18 07:59:57 test-foo-01 goferd[7785]: [ERROR][MainThread] katelloplugin:208 - SSLError: tlsv1 alert unknown ca Jul 18 07:59:57 test-foo-01 goferd[7785]: [WARNING][MainThread] katelloplugin:87 - tlsv1 alert unknown ca