Commit Graph
3 Commits
Author SHA1 Message Date
bitdeals c66cea2009 feat: the daemon keeps no privilege, and the container drops what it can
Build docker image and push to registry.bitdeals.org / main-build-job (push) Failing after 3m41s
PyBitmessage is a Python 2 daemon parsing untrusted data from an open port,
so the question is not whether it can be taken over but what is left once it
has been. Until now: uid 2000, but the full fourteen capabilities Docker
hands a container in the bounding set, no_new_privs off, and a root PID 1
holding all fourteen for the life of the container.

run.sh now needs root exactly once. keys.dat can arrive from a bind mount
owned by anyone, so it is chowned, given mode 600 and read first; the block
that does it ends by re-executing the file with a bounding set of four --
SETUID and SETGID to start the daemon as its own user, KILL for the fallback
stop, SETPCAP to drop the rest. Every start of the daemon then goes through
setpriv rather than gosu: uid 2000, every capability set empty, no_new_privs
on. gosu changed the user and left everything else alone; moreutils went with
it, nothing here ever called any of its tools.

No file in the image carries a setuid or setgid bit any more, which is what
makes no_new_privs worth having: there is nothing left to climb.

Both setpriv lines are checked at build time, in the arrangement run.sh uses,
against uid, capability sets and no_new_privs read back from /proc. The base
image and the PyBitmessage clone are both unpinned, so an option that
quietly changed meaning would otherwise ship as a container that looks
confined and is not. Two things that check caught while it was being written:
Ubuntu 18.04's setpriv refuses the "all" keyword under a kernel that knows
more capabilities than its headers did (40 against 37), and capability names
there carry no cap_ prefix. Hence the lists written out by hand.

A caller that sets cap_drop: ALL now needs seven back, not six: SETPCAP joins
CHOWN, DAC_OVERRIDE, FOWNER, SETUID, SETGID and KILL, because dropping a
bounding set takes it. The example compose file and both READMEs say so, and
say what else only a caller can set: read_only with tmpfs, and pids_limit.
2026-09-09 10:29:32 +00:00
bitdeals 88b5b896e1 feat: supervise the daemon, and restart one that has lost every peer
Build docker image and push to registry.bitdeals.org / main-build-job (push) Successful in 7m12s
A PyBitmessage daemon that has dropped to zero network connections does not find
its way back. It can sit there for days — testnet1 did, and every BitDeals deal
completed on that stand since 2026-08-30 left its escrow unspent, because the
guarantor never received a CH3 the node could no longer publish. A daemon that
has just started, by contrast, dials hard and reconnects within seconds. The
cure was already known; what was missing was anything to notice and apply it.

Docker will not: it reacts to a process exiting, never to a healthcheck, and
health-driven restarts exist only under Swarm. Doing it from outside means
handing a container the docker socket, which is root on the host and a poor
trade for a relay with a published port. So the container supervises itself.

`run.sh` no longer ends at `exec gosu bitmessage pybitmessage -d`. That made the
daemon PID 1, and it is a bad PID 1: daemonize() double-forks and parks the
grandfather in `while True: time.sleep(1)`, the final child SIGTERMs it to say
"ready", and PID 1 drops that signal for want of a handler. The grandfather slept
for ever; `docker stop` therefore reached the real daemon only as the SIGKILL ten
seconds later, cutting a startup VACUUM in half — the way a node gets trapped
retrying one it can never finish; and a daemon that died on its own left the
container Up around a corpse, because what PID 1 was doing had nothing to do with
whether the daemon lived.

Away from PID 1 that grandfather does die on the ready signal — measured here:
the start call returns at once with status 143 and leaves exactly one
pybitmessage process behind. That makes starting the daemon an ordinary blocking
call, and the supervisor ordinary shell: a trap that stops the daemon through its
own API, a restart when it is gone, and the peer rule.

What the supervisor does not do is act on a daemon whose API is not answering at
all. That is the trapped-VACUUM node; a restart does not cure it and cuts the
next VACUUM short as well. watchdog.py reports that case as its own exit code so
the loop can leave it to a person. The healthcheck is untouched: it reports, and
does not act.

Four settings, on by default: BITMESSAGE_WATCHDOG, and _PERIOD, _AFTER,
_COOLDOWN. All validated at start, where a typo is visible, rather than hours
later as a supervisor that spins or one that never acts.

Callers must raise the stop grace period — 90s in compose, or --stop-timeout 90.
A clean shutdown took 17.5 s on a small database and grows with it, so under
Docker's default ten the daemon is killed mid-write anyway and the supervisor
buys nothing. An image cannot set this for itself.
2026-09-04 09:53:07 +00:00
bitdeals 58e3f22982 let the topology be pinned: trusted peer, outgoing switch, known nodes
Build docker image and push to registry.bitdeals.org / main-build-job (push) Successful in 1m50s
A private Bitmessage contour cannot be assembled by letting the nodes find
each other. The sybil check in connectionpool refuses a candidate whose /16 is
already among the outbound connections, and every container of a compose
project shares one /16 -- so each node keeps a single outbound connection to a
randomly chosen peer, and the contour splits into components on some runs and
not on others.

Three variables make the topology explicit instead:

  BITMESSAGE_TRUSTED_PEER    trustedpeer = host:port
  BITMESSAGE_SEND_OUTGOING   sendoutgoingconnections = True/False
  BITMESSAGE_KNOWN_NODES     host:port,... -> knownnodes.dat

With them a star is one line of config per node: the hub takes
SEND_OUTGOING=False and only accepts, the spokes take TRUSTED_PEER=<hub>:8444.

Details worth knowing:

- trustedpeer is absent from the stock keys.dat, so a substitution alone would
  be a silent no-op. The key is added the same way maxtotalconnections is,
  and it is added even when the value is empty -- that is how a node that was
  pinned before can be unpinned. Its anchors stop at "=" rather than "= ",
  because an empty value leaves no trailing space to match.
- knownnodes.dat is rewritten on every start, not only when missing. "Only
  when missing" would never have fired: the image ships one, built by the
  `pybitmessage -t` run in the Dockerfile, and a named volume inherits it.
  Seeding it is also what stops the DNS bootstrap -- deserialising any peer
  that is neither a DEFAULT_NODE nor "self" raises knownNodesActual, and
  startBootstrappers only runs while that flag is down.
- Both peer variables are validated here. PyBitmessage does check trustedpeer,
  but with a sys.exit() from a constructor in the network thread, which reads
  as a container that died for no stated reason.
2026-08-06 14:51:50 +00:00