Expand description
L3 node daemon: a minimal git smart-HTTP server (DECISIONS.md D12).
v1 wraps git http-backend (git’s own CGI) over bare repositories, so
the daemon is a thin, self-hostable shell over git plumbing, and
platform behavior (sequencer, queue, identity) layers on top.
ForgeMark benchmarks this surface directly.
Authentication is per-actor basic auth (AuthTable, --auth-file),
plus the credentials self-service has issued (accounts,
--accounts-file); the platform API (platform) additionally
verifies ed25519 op signatures. The bind stays loopback-only: beyond
localhost you still need TLS or an SSH tunnel so tokens aren’t sent in
the clear.
§Where this sits
docs/architecture.md is the map of the whole workspace.
This crate is L3, the daemon that serves both git smart-HTTP and the platform API.
It builds on choir_fs, choir_hash, choir_identity, choir_oplog, choir_sequencer and choir_view.
The operator’s guide to this daemon:
§Running a node
One binary serves two protocols on one port: git smart-HTTP for clone,
fetch and push, and the platform API for signed operations. This page is
how to start it and what each policy file means.
Using a node: docs/using/cli.md. The policy behind each flag:
docs/operating/authorization.md, docs/operating/limits.md,
docs/operating/webhooks.md, docs/operating/observability.md,
docs/operating/transports.md.
Default port 8417. A repository is sequenced only when it carries a
pre-receive hook: create with choir repo create or --create; a bare
repository that arrives any other way is adopted and hooked at the next
start. With no arguments the binary uses ./repos and port 8417;
configured invocations must supply both <repo-root> and <port> before
any flags.
§Getting the binaries
One command, no Rust toolchain and no OpenSSL headers. macOS and Linux, x86-64 and arm64:
curl -fsSL https://choirs.dev/download/install.sh | shThat installs both packages, which is what this page needs: every command
here is choir, and choir node serve execs the daemon. The four
binaries land in $CARGO_HOME/bin (~/.cargo/bin by default) as
choir-node, choir-ssh, choir and choir-mcp; each archive is
checked against the SHA256 beside it on the shelf. When that directory
is not on your PATH, the installer adds one line to your shell’s
startup file so new terminals find it, and prints the export for the
terminal you ran it in.
The node renders that installer from the Host you asked it on, so the
script and the archives come from the same origin and no address is
configured anywhere. Any node with a shelf serves its own:
https://<node>/download/.
From source instead, needing only rustup:
cargo install --git https://choirs.dev/choir/choir.git choir-cli choir-nodeNeither package pulls in choir-actor, the one crate that needs this
workspace’s LIBSQLITE3_FLAGS.
From a checkout you already have: the toolchain and headers the README lists, then:
cargo build --release -p choir-node -p choir-cli
export PATH="$PWD/target/release:$PATH"choir node status and choir --version report the built commit from a
stamp the release workflow sets.
§One command
choir host| you have | run | you get |
|---|---|---|
| a laptop, or a box nobody else reaches | choir host | http://127.0.0.1:8417, in seconds, no certificate |
| a name pointing at this box | choir host --domain node.example | https://node.example:8417 |
| a VPS and no name | choir host --public | https://<this-ip>.sslip.io:8417 |
It prints the URL people use and, with --invite, one invite link:
choir host --repo me/thing.git --invite AdaThe two public modes take two commands. Obtaining a certificate is
privileged and choir host does not run sudo: it prints the line to
paste and exits 3. Paste it, run the same choir host again, and it
continues.
$ choir host --domain node.example
ok state /home/you/.choir — credential, key, trusted keys
next: a certificate for node.example has to be issued as root.
certbot writes /etc/letsencrypt, and the renewal hook that keeps
this working for the next two years lives there too. Paste this:
sudo choir node tls node.example --user you --port 8417
sudo ufw allow 8417/tcp && sudo ufw allow 80/tcp
then: choir host --domain node.exampleDry-run the certificate step without spending a Let’s Encrypt rate-limit slot:
sudo choir node tls node.example --user "$(id -un)" --dry-run§Why a public bind needs a certificate at all
choir-node refuses a non-loopback bind without TLS
(invariant 9). There is no flag to soften it.
§--public, and why the address is ugly
choir host --public uses an sslip.io name built from
the box’s address, 203-0-113-7.sslip.io for 203.0.113.7, so a
certificate can be issued with no DNS. Replace it any time:
choir host --domain the-name-you-bought.examplesslip.io and nip.io share one Let’s Encrypt rate-limit pool. If issuance
fails for that reason, --public-name <name> takes any name pointing at
this box.
§What it did
Every step is a command you could run yourself:
| step | what | undo |
|---|---|---|
| state | choir init: ~/.choir with the credential, actor key, trusted keys, and .choir/config | delete ~/.choir |
| acl + accounts | ~/.choir/acl granting the operator everything, and an empty ~/.choir/accounts.jsonl | delete either file |
| certificate | choir node tls: certbot, a renewal deploy hook, the pair projected where the node can read it, and ~/.choir/tls.enabled | sudo rm /etc/letsencrypt/renewal-hooks/deploy/choir-tls, sudo certbot delete |
| address | ~/.choir/public-url, and .choir/config pointed here | edit either |
| supervised | choir node install: a launchd agent or a systemd --user unit | choir node uninstall |
| healthy | polls /healthz | |
| repository | choir repo create, with --repo | |
| invited | choir invite, with --invite | choir revoke |
Two things it prints but never runs:
- the firewall.
ufworfirewalld, opening the serving port, and port 80 when HTTP-01 is the challenge (renewals rebind it every ~60 days). - linger. Without
loginctl enable-linger, systemd stops a--userunit at logout.choir hoststops with the line to paste;--yesaccepts a node that dies at logout.
§TLS, and what happens every 60 days
The daemon terminates TLS itself with --tls-cert and --tls-key. An
nginx or Caddy front is a valid topology; it is not required.
The daemon reads its certificate once, at bind. The certbot deploy hook
choir node tls installs re-projects the pair into ~/.choir/tls/ and
restarts the unit. Renewal runs on certbot’s timer.
Root for certbot, an unprivileged account for the node; the node’s copy of the pair is a copy for that reason.
The ACME account is registered without an email. sudo certbot renew --dry-run is the manual check; choir doctor reports the expiry date.
§Checking it
choir doctorOn a hosting machine it adds six rows: bind address, TLS on, certificate expiry, linger, unit loaded and running, and whether the public URL answers (asked from the box, so it proves name and certificate, not the firewall).
§Upgrading
choir node upgrade --from https://<node> # that node's shelf, its installer run here
choir node upgrade --source ~/choir-build # a checkout: cargo build --release thereEither way: the newer binaries are placed over the ones in the directory
this choir runs from (--into <dir> names another), by rename, so a
build that fails leaves the old ones untouched; the supervised node is
restarted; and the build stamp is read back from the running daemon’s
/api/view and compared with what was installed. A node that reports a
different commit than the binary just placed exits 1 with both. --from
wants a directory called bin, since that is where an installer writes;
--dry-run prints the plan. The command runs on the node host and never
runs ssh; the hop that types it is yours.
§Uninstalling
choir node uninstallRemoves the unit and stops the node. Keeps ~/.choir: keys,
repositories and the op log. If a renewal hook is installed it prints the
two sudo lines that remove it and the certificate.
§In a container
Dockerfile at the repository root builds from source and runs the daemon
as an unprivileged user with /var/lib/choir as a volume. Loopback mode and
bring-your-own-certificate mode; no ACME client in the daemon. No compose
file: the volume and the port are one flag each.
§By hand
choir init # the same layout, without a service manager
choir node serve # runs here, in this terminal
choir node serve -- --reviewers-file ~/.choir/reviewers # any daemon flag, after `--`choir node serve derives root, port, credential and trusted-key file from
what choir init wrote, then execs the daemon. Three files switch
behaviour by existing:
| file | effect |
|---|---|
~/.choir/tls.enabled | two lines, cert path then key path: binds 0.0.0.0 with that pair |
~/.choir/acl | --acl-file |
~/.choir/accounts.jsonl | --accounts-file (refused without an ACL) |
Supervised, step by step:
choir init # mint ~/.choir: credential, key, trusted keys, config
choir node install # hand it to launchd (macOS) or systemd (Linux)
choir node status # health, the commit serving, sequencer position
choir repo create me/thing.git # a repository on the running node
choir repo url me/thing.git # the clone URL, and the git config to go with it
choir node logs # the tail of the daemon log
# choir node restart | stop | uninstall -- uninstall keeps ~/.choirchoir node install writes a unit that runs choir node serve. Daemon
flags go after -- and are recorded in the unit:
choir node install -- --acl-file ~/.choir/acl --rate-limit-api 60The daemon can be run directly:
choir-node /tmp/choir-repos 8417 \
--create owner/demo.git \
--auth-file ~/.choir/auth \
--keys-file ~/.choir/keysUseful flags: --bind, --tls-cert / --tls-key, --acl-file <file> (required before a second credential), --request-log <file> and --rate-limit-api / --rate-limit-git (also required before a second credential), --quota-push-bytes / --quota-workspaces, --api-body-limit, --batch-limit, --ready-min-free-bytes, --read-only-browser, --journal <file>, --require-assignment, --protected-refs <file>, --require-review, --reviewer-conflict-graph <file> with --reviewer-conflict-distance <hops>, --review-retention <count>, and --review-lapse-after-secs <seconds>. Authenticated operations endpoints: /healthz, /readyz, /metrics. Flag reference: crates/choir-node/src/main.rs module docs, or AGENTS.md.
File formats (all mode 0600)
| File | Format |
|---|---|
--auth-file | user:token per line; authentication only, pair with --acl-file |
--acl-file | <user> <repo|*|@node> <level> per line; levels read < propose < write < own, and auditor on @node |
--keys-file | <64-hex> or <channel> <64-hex> (bound key) |
--reviewers-file | channel name per line; re-read on each draw |
--protected-refs | owner/repo.git:refs/heads/main (trailing * ok) |
--reviewer-conflict-graph | undirected operator operator edges; pair with an explicit maximum hop distance |
Hot-reload: trusted keys, channel bindings, push-certificate signers, reviewers, and ACL grants take effect on the next request.
[!WARNING] Without
--acl-file, every credential reaches every repository. A seconduser:tokenline can clone every repo, push to any unprotected ref and provision workspaces anywhere. Do not issue a second credential without an ACL.choir hostwrites one.
§Serving your own binaries
--downloads-dir <dir> publishes one directory at /download/, with no
credential (D79). Three things are served there and only three: an index
of the directory, any file in it by exact name, and install.sh, which
the node renders from the Host of the request rather than reading off
the shelf. That is what lets the script fetch from the node it came from
without anybody configuring an address.
The installers run ./choirctl deploy with the flag whenever
~/.choir/downloads exists, so creating the directory is the switch:
./choirctl publish-release # or: publish-release v0.1.0
./choirctl deploypublish-release reads owner/name from ~/.choir/release-repo and
copies that release’s archives and digests onto the shelf. It
deliberately leaves two things behind: the packaging tool’s own
*-installer.sh, which carries the release host baked in at build time
and would send a reader straight back to it, and source.tar.gz, which
the node already serves over git at a revision a reader can name.
The node is not the builder. CI builds and publishes; this changes where a reader fetches from. The digests served beside the archives prove the transfer, not the build, so the trust boundary is TLS plus whoever runs the node.
Names are one path segment of [A-Za-z0-9._-], dotfiles are refused and
symlinks are not followed, so the shelf can only hand out a regular file
an operator put directly in it.
§Backups
On the node host:
choir backup take /srv/backups/choir # now, then verified
choir backup schedule /srv/backups/choir # every hour; --every <seconds> for another interval
choir backup unscheduleA backup is the log, the node fingerprint, the attestation, the policy
files as one archive and one git bundle per repository, in the shape
choir backup verify reads and choir backup restore unpacks, never a
key or a credential. Each take checks that the node’s log extends the
copy already in the directory, is contiguous from seq 0 and verifies as
a chain, keeps a bundle whose refs have not moved, and writes everything
into a sibling directory before moving it into place, so a run cut short
leaves the previous backup as it was. The directory must not be inside
the state directory. The timer is a launchd job or a systemd timer
running the same command; its runs are logged to backup.log beside the
node’s own. Copying the directory off the machine is still the operator’s
job, and docs/runbook-restore.md is what to do with it.
§Followers
A follower is a remote on a bare repository under the root that every landing is pushed to (D21): a mirror on a forge, a second node’s seed input, a bare repository on another disk. On the node host:
choir repo follower add me/thing.git mirror git@forge.example:me/thing.git
choir node restart # once; the marker makes `node serve` pass --followers
choir repo follower list
choir repo follower push # by hand, every repository, every remoteBranches and tags are pushed, never forced and never pruned, so a
follower that diverged from the node is a push that refuses and a line
in <root>/.choir/followers.jsonl, not a rewrite. Nothing is fetched:
the node is canonical and a follower copies it. The daemon pushes only
when started with --followers, which choir node serve derives from
the marker add writes; without it a remote is inert, which is what a
choir-bridge mirror with an upstream origin relies on.
§Behind a TLS proxy
For the private beta, bind 127.0.0.1 and terminate TLS at a hardened
reverse proxy. Do not use the legacy direct-TLS installer. The beta service
renderer requires an ACL, protected-ref review policy, scoped operations,
operator-issued auth, and the read-only browser.
docs/private-beta-runbook.md covers the
network hold, renderers, backups, CI packaging, staging promotion,
monitoring, rollback and go-live receipts.
scripts/flip/RUNBOOK.md is the operator’s own dogfood procedure; not the
page to start from.
§A copy is a seed
A node that dies takes its repositories with it, unless a copy of them exists somewhere else. The cheapest copy needs no choir process at all:
choir-node --export <root> <dir> # on the node: the log, one git bundle per repository, a manifest
# copy <dir> anywhere: another disk, an object store, a static web host
choir-node --verify-export <dir> # on the copy, with the same binary--verify-export folds the log and requires every ref the view names to
be in that repository’s bundle at the same oid (D61), so a copy that
verifies is the node’s history and the objects that history points at,
checked on the machine that holds it. choir-node --import <dir> <root>
turns it back into a node.
What the copy proves, and what it does not:
- It proves integrity. The log is a hash chain, each entry carries its author’s signature, and the bundles hold every commit the log names.
- It does not prove freshness. A copy is the node as of the moment it was taken. Nothing in it says the node has not moved on since, or that the copy you are holding is the latest one taken.
- It does not witness. A copy made by the node’s own operator is the node’s own word. It cannot tell you whether a second reader was shown the same history. That needs a seed run by somebody else; see SYNC.md for what a seed signs and how two statements are compared.
Secrets and policy files are never in an export, by construction and then
by inspection; docs/runbook-restore.md covers what a restore needs
beside it.
§Run a seed
A seed is a live copy of another node’s log, its home (D80). It
reads the home’s log page by page, checks every page before keeping any
of it, fetches the git objects the log names, serves what it verified,
and signs a statement about the attestation it saw. It writes nothing:
every write sent to it, a git push included, is answered 421 not_home with the home’s address. SYNC.md is the
contract.
One command, run twice, the same shape as choir host:
choir seed https://<home-host> --name seed-aThe first run mints the seed’s node key where the daemon will look for it and stops, exit 3, printing what the home’s operator pastes:
keys file: seed-a <64-hex>
binding: choir bind https://<home-host> <home-root>/.choir/node.key seed-a <64-hex>
auth file: seed-a:<a token minted there>
acl file: seed-a @node auditor
seed-a * readOn the home, nothing seed-specific, only the files it already has:
the keys line registers the key; the binding makes the key on its
statements a name; the credential and the two grants admit it. @node auditor is the node-wide read /api/log needs, and there is no
anonymous seed. The log grant does not reach git, so * read is what
lets it fetch; name repositories instead of * to seed only some.
The second run, with the credential the home issued as a 0600 file:
choir seed https://<home-host> --name seed-a --credential ./seed-a.authIt copies the credential into the state directory, mints a credential
and an ACL for the seed’s own readers, writes the marker choir node serve reads, installs the unit under launchd or systemd, waits for the
seed to answer, and prints its URL. --port <n> picks the port,
--foreground execs the daemon instead of installing a unit,
--archival makes a seed that binds nothing: it replicates, stores,
fetches git and signs statements into its root, and serves nobody. The
laptop, the CI sandbox, the machine behind NAT. After that choir node status|logs|restart|stop|uninstall and choir doctor treat it as the
node it is.
By hand, the daemon lines the command composes. The credential file
holds the one seed-a:<token> line, mode 0600; it reaches curl on stdin
and git through its environment, never an argv or a URL. A serving seed
takes its own readers’ --auth-file and --acl-file like any node:
choir-node /srv/choir-seed 8418 \
--seed https://<home-host> \
--seed-credential ~/.choir/seed-credential \
--auth-file ~/.choir/seed-auth \
--acl-file ~/.choir/seed-acl
choir-node /srv/choir-archive --seed https://<home-host> --seed-credential ~/.choir/seed-credential # archivalA seed that serves beyond loopback is a node like any other and needs
--bind with --tls-cert and --tls-key; it refuses otherwise. It
takes no --keys-file (it learns every key from the home’s
/api/signers) and no --create. On first contact it pins the home’s
node key at .choir/home.fingerprint and refuses a home that later
signs with another one.
Where it stands is replica in its GET /api/view, and one line in its
page header: seed of <home>, N behind. An entry that fails a check, or
that its build cannot fold, halts replication there; the seed keeps
serving what it verified and says halted with the seq and the reason.
--seed-strict exits nonzero instead, for a supervisor to notice.
Seeds upgrade before homes. A seed whose binary cannot fold an op the home admitted halts rather than skipping it, because a copy that drops what it does not understand is a fork. Upgrading the seeds first means a new op kind never reaches a seed that cannot read it.
Readers point choir at seeds with seeds = <url>[, <url>] beside
node = in .choir/config; choir doctor then compares each seed’s
statement with what the home shows them, fails on a fork, and warns on a
seed that halted, with the seq and the reason.
Re-exports§
pub use platform::Platform;
Modules§
- accounts
- Account and credential self-service (D36).
- acl
- Per-repository authorization (D29).
- followers
- Followers (D21): after a ref lands, push it to the remotes the operator added to the bare repository.
- hooks
- Outbound ref-landed webhooks (D32): “something moved, go run this”.
- limits
- Request accounting and admission control (D33): who did what, and how much of it they may do.
- platform
- The platform API: signed op submission and view queries over HTTP.
- portable
- Exporting a node root, importing one, and settling what an export claims (§E).
- profile
- Who an actor is, from what the log already says about them (D63).
- provision
- Instant workspace provisioning (
POST /api/workspace) — the D21 “productized provisioning” wedge feature. - queue
- The node’s own merge queue (D68).
- queue_
api - Driving the node’s merge queue from outside (D5, D68).
- quota
- Per-user quotas (D37): how much of the node one credential may hold.
- reject
- Rejections that name the repair.
- replica
- A seed (D80): a node whose log is a copy of another node’s, taken one way, verified before it is kept, and never written to by anybody here.
- ssh
- SSH transport for git (D31): the host’s
sshd, a forced command, and this shim.
Structs§
- Node
- A running node daemon serving repos under a root directory.
- Trusted
Key - One line of the trusted-keys file: a public key the node accepts, and optionally the channel name its holder is allowed to speak as.
Constants§
- BUILD_
COMMIT - The commit this binary was built from, or the literal
unknownwhen the build had no way to find out. Seebuild.rs. - BUILD_
DIRTY - Whether the build tree had uncommitted changes. Only meaningful under
BUILD_SOURCE == "git", and even then best-effort: cargo cannot rerun the build script on every source edit, so this can be stale whereBUILD_COMMITcannot. - BUILD_
SOURCE - Where
BUILD_COMMITcame from:env(the installer passedCHOIR_GIT_HEAD, the sound path),git(best-effort at build time), orunavailable(no commit could be determined). - DEFAULT_
API_ BODY_ BYTES - Default maximum body size for every
/api/...request: 1 MiB. - DEFAULT_
READY_ MIN_ FREE_ BYTES - Readiness refuses when the filesystem reports less than 1 GiB free.
Functions§
- build_
json - The build stamp as served under
/api/view.build. - build_
line - One line naming the running binary’s provenance, for the startup log.
- git_
invocations - How many times this process has shelled out to git while serving.
- parse_
keys_ file - Parses a trusted-keys file. One key per line,
#comments and blank lines skipped, in either form: - ssh_
ed25519_ pubkey - Renders a raw ed25519 public key in OpenSSH
ssh-ed25519 <b64>form (the wire blob is two length-prefixed strings: key type, key bytes). - ui_
tokens_ css - The design tokens from
ui.css, alone, for the documentation book. - write_
allowed_ signers - Writes
<root>/.choir/allowed_signers— the file git’s ssh signature verification checks push certificates against. One line per actor: principal (the actor id) followed by the OpenSSH public key.
Type Aliases§
- Auth
Table - Per-actor credentials: username → token, checked as HTTP basic auth (the standard git-over-HTTP shape; every forge client speaks it).