The chown at startup was the only thing that ever needed root here, and it
served a case this project does not have: keys.dat arriving from a bind mount
owned by somebody else. Every deployment uses a named volume, which takes its
ownership from the image. So the chown goes, and everything that existed to
survive it goes with it.
USER bitmessage in the Dockerfile, from PID 1 onwards. drop_privs.py is
deleted, the supervisor no longer re-executes itself with a trimmed bounding
set, run.sh has no privileged prologue and no wrapper around the eight
commands that used to run through one. What is left of run.sh differs from
the version before any of this by eleven lines: two chowns gone, seven `gosu
bitmessage` prefixes gone, one comment reworded.
keys.dat gets its mode 600 at build time instead of on every start, because
on every start there is now no root to set it. Its mode and ownership reach a
fresh volume from the image, and every volume in service already carries them
-- checked on all four live nodes: nothing under /home/bitmessage is owned by
anyone but 2000.
The setuid strip stays. It is two lines and it closes the one way a taken-over
daemon could still have climbed.
What the caller sets changes too, and in the right direction: `cap_drop: ALL`
with nothing added back, where the previous commit needed seven capabilities
handed in. Confinement that used to be split between the image and the caller
now sits in one place. The image gives up defending itself when run with no
options at all, which is the trade named in the README along with the bind
mount it costs.
Ubuntu 18.04 does not ship setpriv. The program exists in util-linux 2.31,
but Debian only began installing it at 2.32, and this image is on bionic
because PyBitmessage is Python 2. The build assertion added in the last
commit caught it, which is what it is for: `/bin/sh: 1: setpriv: not found`,
exit 127, no image pushed.
drop_privs.py does the same work with what the image already has. It sets
no_new_privs, drops the bounding set with PR_CAPBSET_DROP while CAP_SETPCAP
is still held, then optionally becomes the daemon's user. Two shapes, both in
run.sh: keep four capabilities and stay root, for the supervisor; keep none
and become uid 2000, for the daemon and everything run on its behalf.
It is better than setpriv would have been in one respect. The bounding set is
walked up to the kernel's own cap_last_cap instead of a list of names, so a
capability this image has never heard of goes too -- and the "-all" spelling
that bionic's setpriv refuses under a newer kernel is not needed at all.
Verified in a user namespace, in the arrangement run.sh uses: the outer drop
leaves 00000000000001e0, the inner leaves every capability set at zero with
no_new_privs set. The build assertion checks the same two things.
PyBitmessage is a Python 2 daemon parsing untrusted data from an open port,
so the question is not whether it can be taken over but what is left once it
has been. Until now: uid 2000, but the full fourteen capabilities Docker
hands a container in the bounding set, no_new_privs off, and a root PID 1
holding all fourteen for the life of the container.
run.sh now needs root exactly once. keys.dat can arrive from a bind mount
owned by anyone, so it is chowned, given mode 600 and read first; the block
that does it ends by re-executing the file with a bounding set of four --
SETUID and SETGID to start the daemon as its own user, KILL for the fallback
stop, SETPCAP to drop the rest. Every start of the daemon then goes through
setpriv rather than gosu: uid 2000, every capability set empty, no_new_privs
on. gosu changed the user and left everything else alone; moreutils went with
it, nothing here ever called any of its tools.
No file in the image carries a setuid or setgid bit any more, which is what
makes no_new_privs worth having: there is nothing left to climb.
Both setpriv lines are checked at build time, in the arrangement run.sh uses,
against uid, capability sets and no_new_privs read back from /proc. The base
image and the PyBitmessage clone are both unpinned, so an option that
quietly changed meaning would otherwise ship as a container that looks
confined and is not. Two things that check caught while it was being written:
Ubuntu 18.04's setpriv refuses the "all" keyword under a kernel that knows
more capabilities than its headers did (40 against 37), and capability names
there carry no cap_ prefix. Hence the lists written out by hand.
A caller that sets cap_drop: ALL now needs seven back, not six: SETPCAP joins
CHOWN, DAC_OVERRIDE, FOWNER, SETUID, SETGID and KILL, because dropping a
bounding set takes it. The example compose file and both READMEs say so, and
say what else only a caller can set: read_only with tmpfs, and pids_limit.
A PyBitmessage daemon that has dropped to zero network connections does not find
its way back. It can sit there for days — testnet1 did, and every BitDeals deal
completed on that stand since 2026-08-30 left its escrow unspent, because the
guarantor never received a CH3 the node could no longer publish. A daemon that
has just started, by contrast, dials hard and reconnects within seconds. The
cure was already known; what was missing was anything to notice and apply it.
Docker will not: it reacts to a process exiting, never to a healthcheck, and
health-driven restarts exist only under Swarm. Doing it from outside means
handing a container the docker socket, which is root on the host and a poor
trade for a relay with a published port. So the container supervises itself.
`run.sh` no longer ends at `exec gosu bitmessage pybitmessage -d`. That made the
daemon PID 1, and it is a bad PID 1: daemonize() double-forks and parks the
grandfather in `while True: time.sleep(1)`, the final child SIGTERMs it to say
"ready", and PID 1 drops that signal for want of a handler. The grandfather slept
for ever; `docker stop` therefore reached the real daemon only as the SIGKILL ten
seconds later, cutting a startup VACUUM in half — the way a node gets trapped
retrying one it can never finish; and a daemon that died on its own left the
container Up around a corpse, because what PID 1 was doing had nothing to do with
whether the daemon lived.
Away from PID 1 that grandfather does die on the ready signal — measured here:
the start call returns at once with status 143 and leaves exactly one
pybitmessage process behind. That makes starting the daemon an ordinary blocking
call, and the supervisor ordinary shell: a trap that stops the daemon through its
own API, a restart when it is gone, and the peer rule.
What the supervisor does not do is act on a daemon whose API is not answering at
all. That is the trapped-VACUUM node; a restart does not cure it and cuts the
next VACUUM short as well. watchdog.py reports that case as its own exit code so
the loop can leave it to a person. The healthcheck is untouched: it reports, and
does not act.
Four settings, on by default: BITMESSAGE_WATCHDOG, and _PERIOD, _AFTER,
_COOLDOWN. All validated at start, where a typo is visible, rather than hours
later as a supervisor that spins or one that never acts.
Callers must raise the stop grace period — 90s in compose, or --stop-timeout 90.
A clean shutdown took 17.5 s on a small database and grows with it, so under
Docker's default ten the daemon is killed mid-write anyway and the supervisor
buys nothing. An image cannot set this for itself.
A private Bitmessage contour cannot be assembled by letting the nodes find
each other. The sybil check in connectionpool refuses a candidate whose /16 is
already among the outbound connections, and every container of a compose
project shares one /16 -- so each node keeps a single outbound connection to a
randomly chosen peer, and the contour splits into components on some runs and
not on others.
Three variables make the topology explicit instead:
BITMESSAGE_TRUSTED_PEER trustedpeer = host:port
BITMESSAGE_SEND_OUTGOING sendoutgoingconnections = True/False
BITMESSAGE_KNOWN_NODES host:port,... -> knownnodes.dat
With them a star is one line of config per node: the hub takes
SEND_OUTGOING=False and only accepts, the spokes take TRUSTED_PEER=<hub>:8444.
Details worth knowing:
- trustedpeer is absent from the stock keys.dat, so a substitution alone would
be a silent no-op. The key is added the same way maxtotalconnections is,
and it is added even when the value is empty -- that is how a node that was
pinned before can be unpinned. Its anchors stop at "=" rather than "= ",
because an empty value leaves no trailing space to match.
- knownnodes.dat is rewritten on every start, not only when missing. "Only
when missing" would never have fired: the image ships one, built by the
`pybitmessage -t` run in the Dockerfile, and a named volume inherits it.
Seeding it is also what stops the DNS bootstrap -- deserialising any peer
that is neither a DEFAULT_NODE nor "self" raises knownNodesActual, and
startBootstrappers only runs while that flag is down.
- Both peer variables are validated here. PyBitmessage does check trustedpeer,
but with a sys.exit() from a constructor in the network thread, which reads
as a container that died for no stated reason.
The daemon has always listened on 8444 and nothing published it, so every
node built from this image was outbound-only. Publishing it is now a
documented choice rather than an omission -- including the two things that
are not obvious from the config: peers are told the port from `port` in
keys.dat rather than the one you mapped it to (so only 8444:8444 works),
and the node's own address is never configured at all, because every peer
replaces the hardcoded 127.0.0.1 in the version message with the IP it
sees on the socket.
An open port needs a brake, hence BITMESSAGE_MAXTOTALCONNECTIONS. It
defaults to the PyBitmessage stock 200, so nothing changes for existing
users of the image. The key is inserted when the stock config lacks it
instead of trusting the substitution: the PyBitmessage clone is unpinned,
and a silent no-op would ship a node that looks capped and is not.
The API port in the example compose moves to loopback, which is what the
README already prescribed.
Same pairing the bitdeals-ng repositories use: each file points at the
other under its first heading, and both examples are byte-identical across
the two so they cannot drift.
The examples were not safe to copy. They published the API on every
interface next to the default credentials it documents, carried no volume
-- so keys.dat, the node's Bitmessage identity, was lost on every image
update -- and the docker run one was missing a line continuation and did
not run at all. Ports are now on loopback, the volume is there, and the
run was checked verbatim on a host.
Also: stopresendingafterxdays is documented as 30, which is what run.sh
actually defaults to, not 60; the seed phrase is noted as regenerated per
start; and a short Notes section records what cost time to find out -- '%'
is unusable in the API password, clients have to percent-encode
credentials, and the health check waits for a network connection.
The sed pass that configures the daemon punished anyone setting a strong
password: '&' in a replacement means the whole match, so a password
containing one was silently rewritten into something else, and the '|'
delimiter made sed exit with 'unknown option to s'. Every setting rode in
one invocation and the script had no set -e, so that failure applied none
of them -- not apipassword, not apienabled, not apiinterface -- and the
daemon came up on whatever it had before without saying so.
esc() now escapes backslash, '&' and the delimiter via printf. The
expressions are anchored to the start of the line and name their key in the
replacement instead of using \1, so no backreference is involved and
nothing in another section can match. keys.dat also holds privsigningkey
and privencryptionkey for this node's Bitmessage identities; editing in
place leaves every byte outside [bitmessagesettings] untouched, which is
why this stays a line edit rather than a parse-and-rewrite.
The clients needed the other half of this: credentials go into an XML-RPC
URL, where '@' splits the userinfo and '#' truncates the rest, so a strong
password wrote correctly and still failed to connect. Both now
percent-encode. A '%' in the password remains unusable -- it breaks
PyBitmessage's own config reader and every API call returns 500.
Also here, all found while making the above safe:
- set -eu, with the keys.dat chown guarded. A misconfiguration now stops
the container instead of passing unnoticed.
- The seed-address retry loop used bash brace expansion under CMD ["sh"],
where /bin/sh is dash and {1..4} is a literal, so it ran once, not four
times.
- apt-get update shared a layer with nothing, letting a cached update feed
install months-stale package lists.
- HEALTHCHECK gained a start period; the startup VACUUM takes tens of
seconds on a large messages.dat and the container reported unhealthy for
all of it.
- Removed the AppImage systemd unit, AppArmor profile and updater script.
Nothing referenced them -- not the image, the deployment or the ansible
roles -- and the updater fetched a binary with no signature or checksum
check.
bitdeals
bitdeals git user
private-user