From ea8d5e0589e8b67235138af43164630901dd0ab9 Mon Sep 17 00:00:00 2001 From: bitdeals Date: Fri, 4 Sep 2026 11:30:32 +0000 Subject: [PATCH] fix: pin the base by digest, and stop overriding its FIPS preference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The public door closed again this morning. haproxy exited at parse time on every host with 'bind *:443' : inconsistencies between private key and certificate loaded '/usr/local/etc/haproxy/certificates/site.pem' on a leaf and a private key that openssl confirms are one pair — same public key, and the same pair certbot has been serving since 2026-08-22. Nothing of ours had changed: the image was rebuilt at 10:21 by the scheduled build, on a base that had moved overnight. The cause is the override this file added two days ago. Measured on testnet2, same image, same certificate, only the variable changed: OPENSSL_FIPS=no config check fails at `bind *:443` OPENSSL_FIPS=yes config check passes, curl 200, TLSv1.3 `ENV OPENSSL_FIPS=no` was right for the 2026-09-02 base, where preferring the FIPS provider left haproxy offering an X25519MLKEM768 group it could not use and refusing every modern client. On today's base the same value stops it loading an ECDSA certificate at all, while the shipped "yes" both loads it and still negotiates TLS 1.3 — the thing the override existed to protect. So it is removed rather than inverted; the comment keeps why, because the right value depends on the base and the next base may want the other one. Which is the second change, and the one that matters. `FROM bitnami/haproxy` carries no tag and the build runs daily, so this image took whatever the base was each morning: two outages in three days, neither with a commit behind it, both diagnosed from scratch. The base is pinned by digest now — the one measured working today. It stops moving on its own, and bumping it becomes a deliberate act with a check behind it. --- docker/Dockerfile | 48 +++++++++++++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index a8e4815..f2af39a 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,25 +1,41 @@ -FROM bitnami/haproxy +# Pinned by digest, and that is the whole point of the line. `FROM +# bitnami/haproxy` carries no tag, the scheduled build runs daily, and so this +# image was rebuilt every morning on whatever the base happened to be — twice +# in three days that broke the public door with no commit of ours behind it: +# 2026-09-02 the base began preferring the FIPS provider and TLS 1.3 closed; +# 2026-09-04 the base changed again and haproxy could no longer load the +# certificate at all ("inconsistencies between private key and certificate" on +# a leaf and key that openssl confirms are a pair). +# +# This digest is the base measured working on 2026-09-04: haproxy 3.4.4-7f03ae6, +# serving the live testnet2 certificate, curl 200 over TLS 1.3. Bumping it is a +# deliberate act with a check behind it, which is what the last two mornings +# were missing. +FROM bitnami/haproxy@sha256:c3fac4a702bf242262588491e2541f83d870484ddf65e1c91527208dc1156c49 -# The base is Photon OS, and its openssl.cnf reads this variable directly: +# The FIPS preference is left exactly as the base ships it, and that is a +# reversal. The base is Photon OS, whose openssl.cnf reads the variable straight +# out of the environment: # # [alg_sect] # default_properties = ?fips=$ENV::OPENSSL_FIPS # -# The base image ships "yes", so every algorithm fetch prefers the FIPS -# provider — and that provider cannot produce an X25519MLKEM768 key: -# ssl_generate_pkey_group fails. HAProxy offers the group regardless, because -# OpenSSL 3.5 lists it first and haproxy.cfg sets no `ssl-default-bind-curves`, -# so it picks a group it then cannot use and answers a fatal illegal_parameter. -# Every client that offers post-quantum key exchange — curl, browsers, anything -# on OpenSSL 3.5 — is refused; only a client asking for a classical group gets -# in. That is how this image closed the public door on 2026-09-02 with no commit -# behind it: `FROM bitnami/haproxy` is untagged and the daily scheduled build -# picks up whatever the base is that morning. +# On the 2026-09-02 base "yes" meant every algorithm fetch preferred the FIPS +# provider, which cannot produce an X25519MLKEM768 key, so haproxy offered a +# group it could not use and refused every modern client. `ENV OPENSSL_FIPS=no` +# fixed that and was correct for that base. # -# Nothing here needs FIPS. Flipping the variable the base image itself provides -# restores the door and the post-quantum exchange both; the FIPS provider stays -# loaded, only the preference is dropped. -ENV OPENSSL_FIPS=no +# On the base pinned above it is the other way round: with "no" haproxy cannot +# load an ECDSA certificate at all and exits at parse time, while the shipped +# "yes" loads it and still negotiates TLS 1.3. Measured on testnet2 against the +# live certificate, same image, only the variable changed: +# +# OPENSSL_FIPS=no config check fails at `bind *:443` +# OPENSSL_FIPS=yes config check passes, curl 200, TLSv1.3 +# +# So the override is gone rather than inverted. Pinning the base is what makes +# leaving it alone safe: the value that is right depends on the base, and the +# base no longer moves on its own. # Copy config COPY ./docker/haproxy.cfg /bitnami/haproxy/conf/haproxy.cfg