feat: harden the edge and move the runtime API off TCP
The runtime API was an unauthenticated `level admin` channel on TCP 9999. `expose:` restricts nothing and docker networks have no per-port rules, so it was reachable by every container sharing a network — nginx included — and reaching it means installing your own certificate and key. It is now a unix socket on a volume shared with certbot alone, which also keeps the private key off the network on every renewal. `expose-fd listeners` is dropped: it hands the listening sockets themselves to a client of that socket and only serves seamless reloads, which this image never performs. HAProxy binds a unix socket by creating `<path>.<pid>.tmp` and renaming it, so it needs write access to the directory; the image now creates /var/lib/haproxy owned by uid 1001 and docker carries that onto an empty named volume. Without it HAProxy refuses to start. Also in this commit: - The visitor's address no longer leaves the process. `option forwardfor` is gone, X-Forwarded-For is deleted unconditionally, and X-Client-Id carries an HMAC of the address under the optional XFF_HMAC_KEY instead — one-to-one with the address, so a rate limit keyed on it is as precise, but not reversible. An empty key sends no header rather than one derived from an empty key. - Logging, which was absent entirely. To stdout for `docker logs`, errors-only via the existing dontlog-normal. Both log-format and error-log-format are hand-written: the built-in formats open with %ci:%cp and would have logged the addresses the rest of this works to avoid. The pseudonym is computed by a tcp-request connection rule so a refused handshake has one too. - TLS pinned: floor TLS 1.2, ECDHE-only in ECDSA and RSA variants, no session tickets, ALPN offering HTTP/2. - HTTP redirects to HTTPS, excepting the ACME challenge path, plus HSTS at one day — short deliberately, since the header cannot be recalled once sent. - timeout http-request, which `timeout client` cannot stand in for: that one resets on every byte, so a slow-drip client held a connection indefinitely. - Backends re-resolve through the declared `resolvers docker`, which nothing referenced. Names were resolved once at boot, so a container recreated on a new IP was never noticed. init-addr libc,none lets HAProxy start anyway. - The commented-out /ms deny rules are deleted. They would have blocked /ms/info and /ms/api/v2/whoami, both deliberately public. - compose: build context is the repository root, as the Dockerfile's COPY expects and as CI already did; restart policy added, since an empty certificate volume is a fatal start-up error; the volumes section, which was missing, so the file can actually come up. Verified on testnet2 against the real binary: config parses clean with the key set, empty, and rejects a non-base64 key at parse time; a live stack confirms the pseudonym is stable per address, differs between addresses, survives a forged header, and that the client address appears nowhere in the log.
This commit is contained in:
bitdeals
+136
-15
@@ -1,6 +1,40 @@
|
||||
global
|
||||
# Enable HAProxy runtime API
|
||||
stats socket :9999 level admin expose-fd listeners
|
||||
# The runtime API, through which certbot installs a renewed certificate.
|
||||
#
|
||||
# A unix socket on a volume shared with certbot alone, not a TCP port. This
|
||||
# is an unauthenticated `level admin` channel — whoever reaches it can
|
||||
# install their own certificate and private key, or point a backend
|
||||
# somewhere else — and a TCP port is reachable by every container sharing a
|
||||
# network, which here would include nginx. File permissions are the only
|
||||
# access control available; a docker network has none.
|
||||
#
|
||||
# No expose-fd listeners: that would additionally hand a client of this
|
||||
# socket the listening sockets themselves, and it is only needed for
|
||||
# seamless reloads, which this image never performs.
|
||||
stats socket /var/lib/haproxy/admin.sock mode 660 level admin
|
||||
|
||||
# To stdout, so `docker logs` collects it — a container has no syslog
|
||||
# daemon, and without any `log` directive HAProxy emits nothing at all:
|
||||
# no backend failures, no TLS handshake refusals, no 5xx.
|
||||
log stdout format raw local0
|
||||
|
||||
# TLS policy for every `bind ... ssl` below. Left unset, the floor is
|
||||
# whatever the image's OpenSSL happens to accept, which is not a decision
|
||||
# this repository gets to make once and keep.
|
||||
#
|
||||
# ECDHE only, no DHE: every client that speaks TLS 1.2 has done ECDHE for
|
||||
# over a decade, and admitting DHE would drag in the question of DH
|
||||
# parameter size for no gain. Both -ECDSA- and -RSA- variants are listed
|
||||
# because the certificate can be either — certbot issues ECDSA by default,
|
||||
# while the self-signed placeholder it falls back to is RSA.
|
||||
#
|
||||
# no-tls-tickets: a resumption ticket is sealed with a key held for the life
|
||||
# of the process, so a ticket captured today is readable by anyone who later
|
||||
# obtains that key. Dropping tickets keeps forward secrecy whole, at the
|
||||
# cost of a full handshake on resumption.
|
||||
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
|
||||
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305
|
||||
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
|
||||
|
||||
defaults
|
||||
mode http
|
||||
@@ -9,43 +43,130 @@ defaults
|
||||
timeout connect 5s
|
||||
timeout client 1m
|
||||
timeout server 1m
|
||||
# An absolute cap on the header-reading phase. `timeout client` cannot
|
||||
# replace it: that one is an inactivity timeout and resets on every byte, so
|
||||
# a client dripping one byte at a time holds a connection open for as long
|
||||
# as it likes.
|
||||
timeout http-request 10s
|
||||
|
||||
log global
|
||||
|
||||
# A hand-written format, never `option httplog`: that one's default begins
|
||||
# with `%ci:%cp` and would put every visitor's address into `docker logs`,
|
||||
# undoing the pseudonym the frontends go to the trouble of minting. The
|
||||
# pseudonym takes that first field instead, and with no key configured the
|
||||
# field is a bare `-` rather than something derived from an empty key.
|
||||
#
|
||||
# The request line is logged as method and path only. `%{+Q}r` would carry
|
||||
# the query string too, and a token that ever appears in one would be
|
||||
# written down for as long as the log is kept.
|
||||
# Two loggers, and the second is easy to miss: a connection that dies before
|
||||
# a transaction exists — a refused TLS handshake, and TLS 1.2 is now the
|
||||
# floor — is written by the error logger, whose default format *also* opens
|
||||
# with %ci:%cp. Leaving it alone would log the address of exactly the
|
||||
# clients this configuration turns away.
|
||||
.if !streq("${XFF_HMAC_KEY}","")
|
||||
log-format "%[var(sess.cid)] [%tr] %ft %b/%s %ST %B %TR/%Ta %HM %HP"
|
||||
error-log-format "%[var(sess.cid)] [%tr] %ft %ac/%fc %[fc_err_str]"
|
||||
.else
|
||||
log-format "- [%tr] %ft %b/%s %ST %B %TR/%Ta %HM %HP"
|
||||
error-log-format "- [%tr] %ft %ac/%fc %[fc_err_str]"
|
||||
.endif
|
||||
|
||||
# Kept: with logging now on, this is what makes the log errors-only rather
|
||||
# than a full access log. Remove it deliberately if every request is wanted.
|
||||
option dontlog-normal
|
||||
option tcp-smart-accept
|
||||
option tcp-smart-connect
|
||||
#option forwardfor
|
||||
# No `option forwardfor`, deliberately: it would write the visitor's real
|
||||
# address into X-Forwarded-For, and the address must not leave this process.
|
||||
# Each frontend deletes that header and mints X-Client-Id instead — see
|
||||
# XFF_HMAC_KEY below.
|
||||
option http-keep-alive
|
||||
|
||||
http-reuse safe
|
||||
|
||||
frontend http
|
||||
bind *:80
|
||||
#http-request redirect scheme https code 301
|
||||
|
||||
.if !streq("${XFF_HMAC_KEY}","")
|
||||
# A pseudonym instead of the address: one-to-one with it, so a rate limit
|
||||
# keyed on it is exactly as precise, but not reversible without the key.
|
||||
# HMAC rather than a bare digest — IPv4 is 2^32 values, and an unkeyed hash
|
||||
# of an address is brute-forced in seconds. Unset key means no pseudonym at
|
||||
# all, never one derived from an empty key.
|
||||
#
|
||||
# Computed on accept rather than in the http phase: a TLS handshake that
|
||||
# fails never reaches an http-request rule, and the error logger still needs
|
||||
# something to write in place of the address. `sess` scope so it outlives
|
||||
# the transaction. Written above the http-request rules because that is the
|
||||
# order it runs in, and HAProxy warns when the two disagree.
|
||||
tcp-request connection set-var(sess.cid) src,hmac(sha256,"${XFF_HMAC_KEY}"),base64
|
||||
.endif
|
||||
|
||||
# The visitor's address stops here — X-Forwarded-For is dropped rather than
|
||||
# filled in, so nothing downstream can log an address it was never given.
|
||||
# Both deletes run unconditionally: a header a client sent must never be
|
||||
# mistaken for one this proxy minted.
|
||||
http-request del-header X-Forwarded-For
|
||||
http-request del-header X-Client-Id
|
||||
.if !streq("${XFF_HMAC_KEY}","")
|
||||
http-request set-header X-Client-Id %[var(sess.cid)]
|
||||
.endif
|
||||
http-request set-header X-Forwarded-Proto http
|
||||
|
||||
# ACL
|
||||
acl certbot path_beg /.well-known/acme-challenge/
|
||||
use_backend certbot if certbot
|
||||
|
||||
# # Fool-proof: MS has no auth — never expose /ms publicly.
|
||||
# http-request deny deny_status 404 if { path -i /ms } || { path -i -m beg /ms/ }
|
||||
# Everything except the ACME challenge goes to HTTPS. The `unless` is
|
||||
# load-bearing: Let's Encrypt validates over plain HTTP on port 80, so
|
||||
# redirecting that path would break every renewal. Written above
|
||||
# use_backend because that is the order it runs in — http-request rules are
|
||||
# evaluated before backend selection whatever the order in the file, and
|
||||
# HAProxy warns when the two disagree.
|
||||
http-request redirect scheme https code 301 unless certbot
|
||||
|
||||
use_backend certbot if certbot
|
||||
|
||||
frontend https
|
||||
bind *:443 ssl crt /usr/local/etc/haproxy/certificates/site.pem
|
||||
http-request add-header X-Forwarded-Proto https
|
||||
# alpn: without it HAProxy negotiates HTTP/1.1 only. The backend stays
|
||||
# HTTP/1.1 — HAProxy translates — so this is a client-side upgrade alone.
|
||||
bind *:443 ssl crt /usr/local/etc/haproxy/certificates/site.pem alpn h2,http/1.1
|
||||
|
||||
# Same as the http frontend — see the comments there.
|
||||
.if !streq("${XFF_HMAC_KEY}","")
|
||||
tcp-request connection set-var(sess.cid) src,hmac(sha256,"${XFF_HMAC_KEY}"),base64
|
||||
.endif
|
||||
|
||||
http-request del-header X-Forwarded-For
|
||||
http-request del-header X-Client-Id
|
||||
.if !streq("${XFF_HMAC_KEY}","")
|
||||
http-request set-header X-Client-Id %[var(sess.cid)]
|
||||
.endif
|
||||
http-request set-header X-Forwarded-Proto https
|
||||
|
||||
# HSTS. max-age is deliberately one day, not the customary year: this is a
|
||||
# one-way door — a browser that has seen the header refuses plain HTTP to
|
||||
# this host until it expires, and no server-side change can call it back.
|
||||
# One day keeps a mistake (a lapsed certificate, a host that must serve
|
||||
# HTTP again) recoverable within a day. Raise it in steps once renewals have
|
||||
# been seen to work for a while: 86400 -> 2592000 -> 31536000.
|
||||
# No includeSubDomains and no preload: both would bind names this proxy does
|
||||
# not serve, and preload is effectively permanent.
|
||||
http-response set-header Strict-Transport-Security "max-age=86400"
|
||||
|
||||
# ACL
|
||||
acl certbot path_beg /.well-known/acme-challenge/
|
||||
use_backend certbot if certbot
|
||||
|
||||
# # Fool-proof: MS has no auth — never expose /ms publicly.
|
||||
# http-request deny deny_status 404 if { path -i /ms } || { path -i -m beg /ms/ }
|
||||
|
||||
backend default-backend-http
|
||||
http-request set-header X-Forwarded-Proto https if { ssl_fc }
|
||||
server main nginx:80 check
|
||||
# resolvers: without it the name is resolved once at start-up and kept, so a
|
||||
# recreated nginx container on a new IP is never noticed. init-addr libc,none
|
||||
# lets HAProxy start even when nginx is not up yet.
|
||||
server main nginx:80 check resolvers docker resolve-prefer ipv4 init-addr libc,none
|
||||
|
||||
backend certbot
|
||||
server certbot certbot:380
|
||||
server certbot certbot:380 resolvers docker resolve-prefer ipv4 init-addr libc,none
|
||||
|
||||
resolvers docker
|
||||
nameserver dns1 127.0.0.11:53
|
||||
|
||||
Reference in New Issue
Block a user