feat: supervise the daemon, and restart one that has lost every peer

A PyBitmessage daemon that has dropped to zero network connections does not find
its way back. It can sit there for days — testnet1 did, and every BitDeals deal
completed on that stand since 2026-08-30 left its escrow unspent, because the
guarantor never received a CH3 the node could no longer publish. A daemon that
has just started, by contrast, dials hard and reconnects within seconds. The
cure was already known; what was missing was anything to notice and apply it.

Docker will not: it reacts to a process exiting, never to a healthcheck, and
health-driven restarts exist only under Swarm. Doing it from outside means
handing a container the docker socket, which is root on the host and a poor
trade for a relay with a published port. So the container supervises itself.

`run.sh` no longer ends at `exec gosu bitmessage pybitmessage -d`. That made the
daemon PID 1, and it is a bad PID 1: daemonize() double-forks and parks the
grandfather in `while True: time.sleep(1)`, the final child SIGTERMs it to say
"ready", and PID 1 drops that signal for want of a handler. The grandfather slept
for ever; `docker stop` therefore reached the real daemon only as the SIGKILL ten
seconds later, cutting a startup VACUUM in half — the way a node gets trapped
retrying one it can never finish; and a daemon that died on its own left the
container Up around a corpse, because what PID 1 was doing had nothing to do with
whether the daemon lived.

Away from PID 1 that grandfather does die on the ready signal — measured here:
the start call returns at once with status 143 and leaves exactly one
pybitmessage process behind. That makes starting the daemon an ordinary blocking
call, and the supervisor ordinary shell: a trap that stops the daemon through its
own API, a restart when it is gone, and the peer rule.

What the supervisor does not do is act on a daemon whose API is not answering at
all. That is the trapped-VACUUM node; a restart does not cure it and cuts the
next VACUUM short as well. watchdog.py reports that case as its own exit code so
the loop can leave it to a person. The healthcheck is untouched: it reports, and
does not act.

Four settings, on by default: BITMESSAGE_WATCHDOG, and _PERIOD, _AFTER,
_COOLDOWN. All validated at start, where a typo is visible, rather than hours
later as a supervisor that spins or one that never acts.

Callers must raise the stop grace period — 90s in compose, or --stop-timeout 90.
A clean shutdown took 17.5 s on a small database and grows with it, so under
Docker's default ten the daemon is killed mid-write anyway and the supervisor
buys nothing. An image cannot set this for itself.

docker/watchdog.py

@@ -0,0 +1,75 @@
+#!/usr/bin/python
+
+"""Two errands the supervisor in run.sh cannot do in shell: ask, and stop.
+
+`peers` reports how many network connections the daemon has, as an exit code the
+shell can branch on. `shutdown` asks the daemon to stop through its own API,
+which runs doCleanShutdown -- the database is closed rather than cut off.
+
+The streak, the cooldown and the restart itself are deliberately not here. This
+process is short-lived, so a counter would need a file to live in, and then two
+places would decide one thing. run.sh counts; this only reports.
+
+Exit codes are the interface:
+
+    0  the daemon answered and has peers
+    1  the daemon answered and has none      (`peers`)
+       or the shutdown was asked for         (`shutdown`)
+    2  the daemon did not answer at all
+
+2 is not 1 on purpose. A daemon whose API is refusing connections is the trapped
+node of the startup-VACUUM failure, and restarting it neither cures that nor is
+harmless: it drops the next VACUUM half-done as well. The supervisor must be
+able to tell "no peers" from "no answer", and only act on the first.
+"""
+
+import json
+import os
+import sys
+import urllib
+import xmlrpclib
+
+# Credentials go into a URL, so they must be percent-encoded: '@' splits the
+# userinfo, '#' truncates the rest, '/' and ':' change what is parsed as host
+# and port. healthy_check.py carries the same six lines for the same reason --
+# these are one-shot scripts with no package around them to share a module.
+API_USER = os.getenv('BITMESSAGE_API_USER', 'bitmessage_api_user')
+API_PASSWORD = os.getenv('BITMESSAGE_API_PASSWORD', 'bitmessage_api_password')
+API_PORT = os.getenv('BITMESSAGE_API_PORT', '8442')
+
+API_LINK = "http://{}:{}@127.0.0.1:{}/".format(
+    urllib.quote(API_USER, safe=''), urllib.quote(API_PASSWORD, safe=''), API_PORT)
+
+NO_ANSWER = 2
+
+
+def peers(api):
+    """How many peers the daemon holds, printed for the container log."""
+    count = json.loads(api.clientStatus())['networkConnections']
+    print "networkConnections:", count
+    return 0 if count > 0 else 1
+
+
+def shutdown(api):
+    """Ask the daemon to stop itself cleanly (api.py: HandleShutdown)."""
+    api.shutdown()
+    print "shutdown requested"
+    return 1
+
+
+COMMANDS = {'peers': peers, 'shutdown': shutdown}
+
+
+def main(argv):
+    if len(argv) != 2 or argv[1] not in COMMANDS:
+        print >> sys.stderr, "usage: watchdog.py peers|shutdown"
+        return NO_ANSWER
+    try:
+        return COMMANDS[argv[1]](xmlrpclib.ServerProxy(API_LINK))
+    except Exception as exc:  # the API is down, or answering something else
+        print "no answer from the daemon:", exc
+        return NO_ANSWER
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv))