Project

General

Profile

Bug #20993

[Katello 3.4] regular qdrouterd segfaults

Added by Klaas D over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

Hey,
after the update from katello 3.3 to 3.4.5 in conjunction with moving from rhel 7.3 to rhel 7.4 I've noticed regular segfaults of qdrouterd on my systems.

[Sep11 11:42] qdrouterd[68786]: segfault at 88 ip 00007f2bb65f4770 sp 00007f2ba913a298 error 6 in libqpid-proton.so.10.0.0[7f2bb65d6000+4b000]
[Sep11 19:20] qdrouterd[87470]: segfault at 88 ip 00007fdadadc4770 sp 00007fdace10b298 error 6 in libqpid-proton.so.10.0.0[7fdadada6000+4b000]
[Sep12 09:56] qdrouterd[48816]: segfault at 88 ip 00007f4c12e7a770 sp 00007ffe3d538198 error 6 in libqpid-proton.so.10.0.0[7f4c12e5c000+4b000]
[Sep13 02:34] qdrouterd[111436]: segfault at 88 ip 00007f3b75827770 sp 00007f3b63ffe298 error 6 in libqpid-proton.so.10.0.0[7f3b75809000+4b000]
[Sep13 17:11] qdrouterd[69032]: segfault at 88 ip 00007f80c801e770 sp 00007f80ba363298 error 6 in libqpid-proton.so.10.0.0[7f80c8000000+4b000]
[Sep14 07:47] qdrouterd[85459]: segfault at 88 ip 00007fd7eb37b770 sp 00007ffe377a37c8 error 6 in libqpid-proton.so.10.0.0[7fd7eb35d000+4b000]
[Sep15 01:30] qdrouterd[4718]: segfault at 88 ip 00007f0067168770 sp 00007f005a4af298 error 6 in libqpid-proton.so.10.0.0[7f006714a000+4b000]
[Sep15 19:46] qdrouterd[84734]: segfault at 88 ip 00007fc539c36770 sp 00007fff8fe383e8 error 6 in libqpid-proton.so.10.0.0[7fc539c18000+4b000]
[Sep16 14:03] qdrouterd[80819]: segfault at 88 ip 00007fdf7b5ed770 sp 00007fff1b886908 error 6 in libqpid-proton.so.10.0.0[7fdf7b5cf000+4b000]
[Sep17 08:21] qdrouterd[23812]: segfault at 88 ip 00007fc32eedb770 sp 00007fc322222298 error 6 in libqpid-proton.so.10.0.0[7fc32eebd000+4b000]
[Sep18 02:47] qdrouterd[97833]: segfault at 88 ip 00007ff8c6be7770 sp 00007ff8b972d298 error 6 in libqpid-proton.so.10.0.0[7ff8c6bc9000+4b000]
[Sep18 21:09] qdrouterd[36139]: segfault at 88 ip 00007f0304d3e770 sp 00007ffda49253c8 error 6 in libqpid-proton.so.10.0.0[7f0304d20000+4b000]

One thing that strikes me as weird is that my content proxies don't crash nearly as often as the main server does although 95% of clients connect to those. 95% of the clients still run RHEL 7.3 thought :)

# rpm -qa|egrep "qpid|proton"|sort
[the cert rpms]
python-gofer-proton-2.7.6-1.el7.noarch
python-gofer-qpid-2.7.6-1.el7.noarch
python-qpid-1.36.0-1.el7.noarch
python-qpid-proton-0.17.0-4.el7.x86_64
python-qpid-qmf-1.36.0-1.el7.x86_64
qpid-cpp-client-1.36.0-1.el7.x86_64
qpid-cpp-client-devel-1.36.0-1.el7.x86_64
qpid-cpp-server-1.36.0-1.el7.x86_64
qpid-cpp-server-linearstore-1.36.0-1.el7.x86_64
qpid-dispatch-router-0.8.0-1.el7.x86_64
qpid-proton-c-0.17.0-4.el7.x86_64
qpid-qmf-1.36.0-1.el7.x86_64
qpid-tools-1.36.0-1.el7.noarch
tfm-rubygem-qpid_messaging-0.34.1-1.el7.x86_64

Foreman packages on my main foreman/katello server:

candlepin-2.0.40-1.el7.noarch
candlepin-selinux-2.0.40-1.el7.noarch
foremanmain.tld-foreman-client-1.0-1.noarch
foremanmain.tld-foreman-proxy-1.0-1.noarch
foremanmain.tld-foreman-proxy-client-1.0-1.noarch
foremanmain.tld-qpid-broker-1.0-2.noarch
foremanmain.tld-qpid-client-cert-1.0-1.noarch
foremanmain.tld-qpid-router-client-1.0-1.noarch
foremanmain.tld-qpid-router-server-1.0-1.noarch
foreman-1.15.3-1.el7.noarch
foreman-cli-1.15.3-1.el7.noarch
foreman-compute-1.15.3-1.el7.noarch
foreman-debug-1.15.3-1.el7.noarch
foreman-installer-1.15.3-1.el7.noarch
foreman-installer-katello-3.4.5-1.el7.noarch
foreman-postgresql-1.15.3-1.el7.noarch
foreman-proxy-1.15.3-1.el7.noarch
foreman-release-1.15.3-1.el7.noarch
foreman-release-scl-3-1.el7.noarch
foreman-selinux-1.15.3-1.el7.noarch
foreman-vmware-1.15.3-1.el7.noarch
katello-3.4.5-1.el7.noarch
katello-agent-3.0.2-1.el7.noarch
katello-ca-consumer-foremanmain.tld-1.0-1.noarch
katello-certs-tools-2.4.0-1.el7.noarch
katello-client-bootstrap-1.4.0-2.el7.noarch
katello-common-3.4.5-1.el7.noarch
katello-debug-3.4.5-1.el7.noarch
katello-default-ca-1.0-1.noarch
katello-host-tools-3.0.2-1.el7.noarch
katello-host-tools-fact-plugin-3.0.2-1.el7.noarch
katello-installer-base-3.4.5-1.el7.noarch
katello-repos-3.4.0-3.el7.noarch
katello-selinux-3.0.2-1.el7.noarch
katello-server-ca-1.0-1.noarch
katello-service-3.4.5-1.el7.noarch
pulp-admin-client-2.13.2-1.el7.noarch
pulp-client-1.0-1.noarch
pulp-docker-plugins-2.4.0-2.el7.noarch
pulp-katello-1.0.2-1.el7.noarch
pulp-puppet-plugins-2.13.2-1.el7.noarch
pulp-puppet-tools-2.13.2-1.el7.noarch
pulp-rpm-admin-extensions-2.13.2-1.el7.noarch
pulp-rpm-handlers-2.13.2-1.el7.noarch
pulp-rpm-plugins-2.13.2-1.el7.noarch
pulp-selinux-2.13.2-1.el7.noarch
pulp-server-2.13.2-1.el7.noarch
python-gofer-qpid-2.7.6-1.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
python-pulp-agent-lib-2.13.2-1.el7.noarch
python-pulp-bindings-2.13.2-1.el7.noarch
python-pulp-client-lib-2.13.2-1.el7.noarch
python-pulp-common-2.13.2-1.el7.noarch
python-pulp-docker-common-2.4.0-2.el7.noarch
python-pulp-oid_validation-2.13.2-1.el7.noarch
python-pulp-puppet-common-2.13.2-1.el7.noarch
python-pulp-repoauth-2.13.2-1.el7.noarch
python-pulp-rpm-common-2.13.2-1.el7.noarch
python-pulp-streamer-2.13.2-1.el7.noarch
python-qpid-1.36.0-1.el7.noarch
python-qpid-proton-0.17.0-4.el7.x86_64
python-qpid-qmf-1.36.0-1.el7.x86_64
qpid-cpp-client-1.36.0-1.el7.x86_64
qpid-cpp-client-devel-1.36.0-1.el7.x86_64
qpid-cpp-server-1.36.0-1.el7.x86_64
qpid-cpp-server-linearstore-1.36.0-1.el7.x86_64
qpid-dispatch-router-0.8.0-1.el7.x86_64
qpid-proton-c-0.17.0-4.el7.x86_64
qpid-qmf-1.36.0-1.el7.x86_64
qpid-tools-1.36.0-1.el7.noarch
rubygem-smart_proxy_pulp-1.3.0-1.el7.noarch
tfm-rubygem-activerecord-session_store-0.1.2-2.el7.noarch
tfm-rubygem-addressable-2.3.6-4.el7.noarch
tfm-rubygem-algebrick-0.7.3-4.el7.noarch
tfm-rubygem-ancestry-2.2.1-1.el7.noarch
tfm-rubygem-anemone-0.7.2-14.el7.noarch
tfm-rubygem-angular-rails-templates-1.0.2-1.el7.noarch
tfm-rubygem-apipie-bindings-0.2.0-1.el7.noarch
tfm-rubygem-apipie-params-0.0.5-1.el7.noarch
tfm-rubygem-apipie-rails-0.4.0-2.el7.noarch
tfm-rubygem-audited-4.4.1-1.el7.noarch
tfm-rubygem-awesome_print-1.7.0-1.el7.noarch
tfm-rubygem-bastion-5.0.10-1.fm1_15.el7.noarch
tfm-rubygem-bundler_ext-0.4.1-1.el7.noarch
tfm-rubygem-clamp-1.0.0-5.el7.noarch
tfm-rubygem-concurrent-ruby-1.0.3-1.fm1_15.el7.noarch
tfm-rubygem-concurrent-ruby-edge-0.2.3-1.fm1_15.el7.noarch
tfm-rubygem-css_parser-1.4.7-1.el7.noarch
tfm-rubygem-daemons-1.2.3-3.el7.noarch
tfm-rubygem-deacon-1.0.0-1.el7.noarch
tfm-rubygem-deep_cloneable-2.2.2-1.el7.noarch
tfm-rubygem-deface-1.2.0-1.el7.noarch
tfm-rubygem-diffy-3.0.1-3.el7.noarch
tfm-rubygem-docker-api-1.28.0-2.el7.noarch
tfm-rubygem-domain_name-0.5.20160310-1.el7.noarch
tfm-rubygem-dynflow-0.8.24-1.fm1_15.el7.noarch
tfm-rubygem-excon-0.51.0-1.el7.noarch
tfm-rubygem-facter-2.4.0-3.el7.x86_64
tfm-rubygem-fast_gettext-1.1.0-1.el7.noarch
tfm-rubygem-fog-1.38.0-1.el7.noarch
tfm-rubygem-fog-aws-1.2.0-1.el7.noarch
tfm-rubygem-fog-core-1.42.0-1.el7.noarch
tfm-rubygem-fog-google-0.1.0-2.el7.noarch
tfm-rubygem-fog-json-1.0.2-4.el7.noarch
tfm-rubygem-fog-openstack-0.1.18-1.el7.noarch
tfm-rubygem-fog-rackspace-0.1.4-1.el7.noarch
tfm-rubygem-fog-vsphere-1.7.0-1.el7.noarch
tfm-rubygem-fog-xenserver-0.2.3-1.el7.noarch
tfm-rubygem-fog-xml-0.1.2-4.el7.noarch
tfm-rubygem-foreman-tasks-0.9.4-1.fm1_15.el7.noarch
tfm-rubygem-foreman-tasks-core-0.1.4-1.fm1_15.el7.noarch
tfm-rubygem-foreman_docker-3.2.1-1.fm1_15.el7.noarch
tfm-rubygem-foreman_hooks-0.3.14-1.fm1_15.el7.noarch
tfm-rubygem-foreman_remote_execution-1.3.3-1.fm1_15.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.0.5-1.fm1_15.el7.noarch
tfm-rubygem-foreman_scc_manager-1.0.1-1.fm1_15.el7.noarch
tfm-rubygem-foreman_snapshot_management-1.0.0-1.fm1_15.el7.noarch
tfm-rubygem-foreman_templates-5.0.1-1.fm1_15.el7.noarch
tfm-rubygem-formatador-0.2.1-9.el7.noarch
tfm-rubygem-friendly_id-5.1.0-3.el7.noarch
tfm-rubygem-get_process_mem-0.2.1-1.el7.noarch
tfm-rubygem-gettext_i18n_rails-1.2.1-3.el7.noarch
tfm-rubygem-git-1.2.5-7.el7.noarch
tfm-rubygem-hammer_cli-0.10.2-1.el7.noarch
tfm-rubygem-hammer_cli_foreman-0.10.2-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.1.3-5.el7.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.4-2.el7.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.10-2.fm1_12.el7.noarch
tfm-rubygem-hammer_cli_import-0.11.3-1.el7.noarch
tfm-rubygem-hammer_cli_katello-0.10.0-1.el7.noarch
tfm-rubygem-hashie-2.0.5-4.el7.noarch
tfm-rubygem-highline-1.7.8-2.el7.noarch
tfm-rubygem-http-cookie-1.0.2-1.el7.noarch
tfm-rubygem-ipaddress-0.8.0-9.el7.noarch
tfm-rubygem-jquery-ui-rails-4.1.2-5.el7.noarch
tfm-rubygem-katello-3.4.5-1.el7.noarch
tfm-rubygem-ldap_fluff-0.4.6-1.el7.noarch
tfm-rubygem-little-plugger-1.1.3-21.el7.noarch
tfm-rubygem-locale-2.0.9-11.el7.noarch
tfm-rubygem-logging-1.8.2-4.el7.noarch
tfm-rubygem-net-ldap-0.15.0-1.el7.noarch
tfm-rubygem-net-ping-2.0.1-1.el7.noarch
tfm-rubygem-net-scp-1.1.0-6.el7.noarch
tfm-rubygem-net-ssh-4.0.1-2.el7.noarch
tfm-rubygem-netrc-0.7.7-7.el7.noarch
tfm-rubygem-oauth-0.5.1-1.el7.noarch
tfm-rubygem-parse-cron-0.1.4-2.fm1_11.el7.noarch
tfm-rubygem-passenger-4.0.18-9.11.el7.x86_64
tfm-rubygem-passenger-native-4.0.18-9.11.el7.x86_64
tfm-rubygem-passenger-native-libs-4.0.18-9.11.el7.x86_64
tfm-rubygem-pg-0.15.1-3.el7.x86_64
tfm-rubygem-powerbar-1.0.17-1.el7.noarch
tfm-rubygem-protected_attributes-1.1.3-3.el7.noarch
tfm-rubygem-qpid_messaging-0.34.1-1.el7.x86_64
tfm-rubygem-rabl-0.12.0-2.el7.noarch
tfm-rubygem-rack-jsonp-1.3.1-5.el7.noarch
tfm-rubygem-rails-i18n-4.0.9-1.el7.noarch
tfm-rubygem-rails-observers-0.1.2-7.el7.noarch
tfm-rubygem-rainbow-2.2.1-1.el7.noarch
tfm-rubygem-rbvmomi-1.10.0-1.el7.noarch
tfm-rubygem-responders-2.3.0-1.el7.noarch
tfm-rubygem-rest-client-1.8.0-1.el7.noarch
tfm-rubygem-roadie-3.2.1-1.el7.noarch
tfm-rubygem-roadie-rails-1.1.1-1.el7.noarch
tfm-rubygem-robotex-1.0.0-19.el7.noarch
tfm-rubygem-ruby2ruby-2.1.3-4.el7.noarch
tfm-rubygem-ruby_parser-3.6.3-4.el7.noarch
tfm-rubygem-runcible-1.12.0-1.el7.noarch
tfm-rubygem-safemode-1.3.2-1.el7.noarch
tfm-rubygem-scoped_search-4.1.0-1.el7.noarch
tfm-rubygem-secure_headers-3.4.1-1.el7.noarch
tfm-rubygem-sequel-4.20.0-5.el7.noarch
tfm-rubygem-sexp_processor-4.4.4-3.el7.noarch
tfm-rubygem-smart_proxy_dynflow_core-0.1.6-2.fm1_15.el7.noarch
tfm-rubygem-sshkey-1.9.0-1.el7.noarch
tfm-rubygem-trollop-2.1.2-1.el7.noarch
tfm-rubygem-unf-0.1.3-5.el7.noarch
tfm-rubygem-unf_ext-0.0.6-6.el7.x86_64
tfm-rubygem-unicode-display_width-1.0.5-1.el7.noarch
tfm-rubygem-useragent-0.16.8-1.el7.noarch
tfm-rubygem-validates_lengths_from_database-0.5.0-3.el7.noarch
tfm-rubygem-webpack-rails-0.9.8-1.el7.noarch
tfm-rubygem-wicked-1.1.0-4.el7.noarch
tfm-rubygem-will_paginate-3.1.5-1.el7.noarch
tfm-rubygem-x-editable-rails-1.5.5-1.el7.noarch
tfm-runtime-3.2-9.el7.x86_64

Greetings
Klaas

History

#1 Updated by Klaas D over 3 years ago

I work around the issue by simply restarting qdrouterd on crash; otherwise goferd will go wild on some clients (https://bugzilla.redhat.com/show_bug.cgi?id=1318015).

# cat /etc/systemd/system/qdrouterd.service.d/override.conf 
[Service]
Restart=always
RestartSec=3

#2 Updated by Justin Sherrill over 3 years ago

  • Assignee set to Eric Helms

#3 Updated by Eric Helms over 3 years ago

  • Status changed from New to Needs design

#4 Updated by Eric Helms about 3 years ago

  • Legacy Backlogs Release (now unused) set to 166
  • Status changed from Needs design to Resolved

This issue has been reported to the Qpid team and resolved by them. The fix will go out in the next router release so I am closing this issue out.

Also available in: Atom PDF