Bug #39518
opentls_ciphers default 'PROFILE=SYSTEM' fails on ruby 3.0.4 in smart-proxy-develop-source-release (ruby-build vendors non-RHEL OpenSSL)
Description
Error¶
smart-proxy-develop-source-release Jenkins job, ruby=3.0.4 test matrix leg, fails with:
Invalid tls_ciphers value 'PROFILE=SYSTEM': SSL_CTX_set_cipher_list: no cipher match (RuntimeError)
Origin¶
Introduced by #39405, commit 67f34e21eae6e17e92de1beb97a6b076d21c3775 (PR https://github.com/theforeman/smart-proxy/pull/947). lib/launcher.rb defaults tls_ciphers to 'PROFILE=SYSTEM' when /etc/crypto-policies/back-ends/openssl.config exists. PROFILE=SYSTEM is a Red Hat/Fedora OpenSSL patch (not in upstream OpenSSL) that reads that config file.
CI node facts¶
- Jenkins nodes for this job: RHEL/CentOS Stream 9 (theforeman/foreman-infra,
puppet/data/osfamily/RedHat-9.yaml,crb/epelinjenkins_node::unittests)./etc/crypto-policies/*is present by default on these nodes. - Ruby versions tested: 3.0.4, 3.1.0 (theforeman/foreman-jenkins-jobs,
pipelines/test/smart-proxy.groovy,pipelines/release/source/smart-proxy.groovy), each compiled from source via rbenv/ruby-build (jenkins_node::rbenvin foreman-infra). - System OpenSSL on CentOS Stream 9: 3.5.x only.
rbenv/ruby-builddefinitions (upstream, github.com/rbenv/ruby-build):share/ruby-build/3.0.4: buildsopenssl-1.1.1wfrom openssl.org source, gated by--if needs_openssl:1.0.1-1.x.xshare/ruby-build/3.1.0: buildsopenssl-3.0.18from github.com/openssl/openssl source, gated by--if needs_openssl:1.0.2-3.x.x
needs_openssl()(bin/ruby-build) compares only the system OpenSSL version against the given range; it does not check foropenssl-devel.- 3.5.x is outside
1.0.1-1.x.x-> ruby-build builds openssl-1.1.1w from source for the 3.0.4 leg, unconditionally, on this OS. - 3.5.x is inside
1.0.2-3.x.x-> ruby-build does not vendor for the 3.1.0 leg; that Ruby links the system's OpenSSL 3.x directly. - openssl-1.1.1w built from openssl.org source does not include Red Hat's crypto-policies patch, so it does not recognize
PROFILE=SYSTEMas a cipher-list value. - Reproduced locally:
openssl ciphers "PROFILE=SYSTEM"on stock Debian bookworm / Ubuntu 24.04 OpenSSL 3.0.x returns the same error text (SSL_CTX_set_cipher_list:no cipher match).
Scope¶
Fails only on the ruby=3.0.4 leg. The ruby=3.1.0 leg uses the system's RHEL-patched OpenSSL and is unaffected. Installing/ensuring openssl-devel on the Jenkins nodes does not change this outcome, since needs_openssl() gates on version range regardless of devel-package presence.
Fix options¶
- smart-proxy: in
lib/launcher.rb, do not infer OpenSSLPROFILE=SYSTEMsupport from the presence of/etc/crypto-policies/*alone. Rescue/validate theSSL_CTX_set_cipher_listcall and fall back to an explicit cipher list on failure. - CI-side workaround only (does not fix the underlying assumption): drop the ruby 3.0.4 leg from the test matrix.
References¶
- theforeman/smart-proxy:
lib/launcher.rb, commit 67f34e21eae6e17e92de1beb97a6b076d21c3775 - theforeman/foreman-jenkins-jobs:
theforeman.org/yaml/jobs/release/source/smart-proxy-develop-source-release.yaml,theforeman.org/pipelines/release/source/smart-proxy.groovy,theforeman.org/pipelines/lib/rbenv.groovy - theforeman/foreman-infra:
puppet/modules/jenkins_node/manifests/rbenv.pp,puppet/modules/rbenv/manifests/init.pp,puppet/data/osfamily/RedHat-9.yaml - rbenv/ruby-build:
share/ruby-build/3.0.4,share/ruby-build/3.1.0,bin/ruby-build(needs_openssl)
Updated by Ondřej Gajdušek about 1 month ago
- Related to Feature #39405: Add tls_min_version and tls_ciphers to smart proxy added
Updated by Ondřej Gajdušek about 1 month ago
Related prior finding, same class of bug, spotted during PR #947 review before merge (lhellebr, 2026-06-24): https://github.com/theforeman/smart-proxy/pull/947#issuecomment-4787973941
Quote: "The proxy refuses to start because undefined method 'ciphersuites=' for #<OpenSSL::SSL::SSLContext...> ... it seems that Red Hat's version of Ruby has this patched in earlier Ruby versions (https://gitlab.com/redhat/centos-stream/rpms/ruby/-/commit/42815870), which is why it works on RHEL with Ruby 3.0.7, but fails in CI on Ruby 3.0 and 3.1 (upstream, it got patched in 3.2 - https://github.com/ruby/openssl/pull/493)."
Same root pattern as this issue: RHEL-patched Ruby/OpenSSL behaves differently from vanilla upstream Ruby/OpenSSL on Ruby 3.0/3.1 (fixed upstream only in 3.2+). Different symptom (ciphersuites= missing method vs. PROFILE=SYSTEM cipher-list rejection reported here).
Updated by Ondřej Gajdušek about 1 month ago
Correction¶
Description names the wrong file for the code path: lib/launcher.rb's CRYPTO_POLICIES_CONFIG constant checks /etc/crypto-policies/back-ends/opensslcnf.config, not openssl.config. (Both files exist on real RHEL9/CentOS Stream 9 -- verified in a fresh quay.io/centos/centos:stream9 podman container -- but they're separate files/formats; the code only inspects opensslcnf.config.)
Sharper fix approach, verified end-to-end on real CentOS Stream 9¶
opensslcnf.config isn't just a "RHEL9 detected" signal file -- it contains the actual resolved cipher spec in plain, portable OpenSSL syntax, independent of the PROFILE= keyword:
CipherString = @SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:kRSAPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8 Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
Reproduced the exact failure and confirmed the fix directly on CentOS Stream 9 (not just inferred from ruby-build's source):
- Compiled vendored
openssl-1.1.1wfrom source in aquay.io/centos/centos:stream9container (same version/build methodruby-builduses for theruby=3.0.4leg pershare/ruby-build/3.0.4'sneeds_openssl:1.0.1-1.x.xgate). /opt/vendor-openssl/bin/openssl ciphers "PROFILE=SYSTEM"-> fails with the byte-identical error:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match./opt/vendor-openssl/bin/openssl ciphers "@SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:kRSAPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8"(the literalCipherStringfromopensslcnf.config) -> resolves a full, valid cipher list on that same vendored build.- Control check on real CentOS Stream 9 with the distro's own RHEL-patched
openssl-3.5.7package:openssl ciphers "PROFILE=SYSTEM"resolves correctly there, confirming the happy path is unaffected.
So instead of degrading fix option 1 all the way to a generic 'HIGH' default when PROFILE=SYSTEM isn't understood, resolve_tls_ciphers can parse and use this CipherString value directly -- same security intent as the active crypto policy, verified working on the exact vendored OpenSSL build the failing CI leg actually uses. Refines fix option 1; still falls back to 'HIGH' if parsing/probing that value also fails, so the auto-detected default can never crash startup.
Updated by The Foreman Bot about 1 month ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/smart-proxy/pull/953 added
Updated by The Foreman Bot about 1 month ago
- Pull request https://github.com/theforeman/smart-proxy/pull/954 added