Commit Graph
2 Commits
Author SHA1 Message Date
bitdeals 88b5b896e1 feat: supervise the daemon, and restart one that has lost every peer
Build docker image and push to registry.bitdeals.org / main-build-job (push) Successful in 7m12s
A PyBitmessage daemon that has dropped to zero network connections does not find
its way back. It can sit there for days — testnet1 did, and every BitDeals deal
completed on that stand since 2026-08-30 left its escrow unspent, because the
guarantor never received a CH3 the node could no longer publish. A daemon that
has just started, by contrast, dials hard and reconnects within seconds. The
cure was already known; what was missing was anything to notice and apply it.

Docker will not: it reacts to a process exiting, never to a healthcheck, and
health-driven restarts exist only under Swarm. Doing it from outside means
handing a container the docker socket, which is root on the host and a poor
trade for a relay with a published port. So the container supervises itself.

`run.sh` no longer ends at `exec gosu bitmessage pybitmessage -d`. That made the
daemon PID 1, and it is a bad PID 1: daemonize() double-forks and parks the
grandfather in `while True: time.sleep(1)`, the final child SIGTERMs it to say
"ready", and PID 1 drops that signal for want of a handler. The grandfather slept
for ever; `docker stop` therefore reached the real daemon only as the SIGKILL ten
seconds later, cutting a startup VACUUM in half — the way a node gets trapped
retrying one it can never finish; and a daemon that died on its own left the
container Up around a corpse, because what PID 1 was doing had nothing to do with
whether the daemon lived.

Away from PID 1 that grandfather does die on the ready signal — measured here:
the start call returns at once with status 143 and leaves exactly one
pybitmessage process behind. That makes starting the daemon an ordinary blocking
call, and the supervisor ordinary shell: a trap that stops the daemon through its
own API, a restart when it is gone, and the peer rule.

What the supervisor does not do is act on a daemon whose API is not answering at
all. That is the trapped-VACUUM node; a restart does not cure it and cuts the
next VACUUM short as well. watchdog.py reports that case as its own exit code so
the loop can leave it to a person. The healthcheck is untouched: it reports, and
does not act.

Four settings, on by default: BITMESSAGE_WATCHDOG, and _PERIOD, _AFTER,
_COOLDOWN. All validated at start, where a typo is visible, rather than hours
later as a supervisor that spins or one that never acts.

Callers must raise the stop grace period — 90s in compose, or --stop-timeout 90.
A clean shutdown took 17.5 s on a small database and grows with it, so under
Docker's default ten the daemon is killed mid-write anyway and the supervisor
buys nothing. An image cannot set this for itself.
2026-09-04 09:53:07 +00:00
bitdeals 58e3f22982 let the topology be pinned: trusted peer, outgoing switch, known nodes
Build docker image and push to registry.bitdeals.org / main-build-job (push) Successful in 1m50s
A private Bitmessage contour cannot be assembled by letting the nodes find
each other. The sybil check in connectionpool refuses a candidate whose /16 is
already among the outbound connections, and every container of a compose
project shares one /16 -- so each node keeps a single outbound connection to a
randomly chosen peer, and the contour splits into components on some runs and
not on others.

Three variables make the topology explicit instead:

  BITMESSAGE_TRUSTED_PEER    trustedpeer = host:port
  BITMESSAGE_SEND_OUTGOING   sendoutgoingconnections = True/False
  BITMESSAGE_KNOWN_NODES     host:port,... -> knownnodes.dat

With them a star is one line of config per node: the hub takes
SEND_OUTGOING=False and only accepts, the spokes take TRUSTED_PEER=<hub>:8444.

Details worth knowing:

- trustedpeer is absent from the stock keys.dat, so a substitution alone would
  be a silent no-op. The key is added the same way maxtotalconnections is,
  and it is added even when the value is empty -- that is how a node that was
  pinned before can be unpinned. Its anchors stop at "=" rather than "= ",
  because an empty value leaves no trailing space to match.
- knownnodes.dat is rewritten on every start, not only when missing. "Only
  when missing" would never have fired: the image ships one, built by the
  `pybitmessage -t` run in the Dockerfile, and a named volume inherits it.
  Seeding it is also what stops the DNS bootstrap -- deserialising any peer
  that is neither a DEFAULT_NODE nor "self" raises knownNodesActual, and
  startBootstrappers only runs while that flag is down.
- Both peer variables are validated here. PyBitmessage does check trustedpeer,
  but with a sys.exit() from a constructor in the network thread, which reads
  as a container that died for no stated reason.
2026-08-06 14:51:50 +00:00