Skip to main content

Crate choir_node

Crate choir_node 

Source
Expand description

L3 node daemon: a minimal git smart-HTTP server (DECISIONS.md D12).

v1 wraps git http-backend (git’s own CGI) over bare repositories, so the daemon is a thin, self-hostable shell over git plumbing, and platform behavior (sequencer, queue, identity) layers on top. ForgeMark benchmarks this surface directly.

Authentication is per-actor basic auth (AuthTable, --auth-file), plus the credentials self-service has issued (accounts, --accounts-file); the platform API (platform) additionally verifies ed25519 op signatures. The bind stays loopback-only: beyond localhost you still need TLS or an SSH tunnel so tokens aren’t sent in the clear.

§Where this sits

docs/architecture.md is the map of the whole workspace. This crate is L3, the daemon that serves both git smart-HTTP and the platform API.

It builds on choir_fs, choir_hash, choir_identity, choir_oplog, choir_sequencer and choir_view.

The operator’s guide to this daemon:

§Running a node

One binary serves two protocols on one port: git smart-HTTP for clone, fetch and push, and the platform API for signed operations. This page is how to start it and what each policy file means.

Using a node: docs/using/cli.md. The policy behind each flag: docs/operating/authorization.md, docs/operating/limits.md, docs/operating/webhooks.md, docs/operating/observability.md, docs/operating/transports.md.

Default port 8417. A repository is sequenced only when it carries a pre-receive hook: create with choir repo create or --create; a bare repository that arrives any other way is adopted and hooked at the next start. With no arguments the binary uses ./repos and port 8417; configured invocations must supply both <repo-root> and <port> before any flags.

§Getting the binaries

One command, no Rust toolchain and no OpenSSL headers. macOS and Linux, x86-64 and arm64:

curl -fsSL https://choirs.dev/download/install.sh | sh

That installs both packages, which is what this page needs: every command here is choir, and choir node serve execs the daemon. The four binaries land in $CARGO_HOME/bin (~/.cargo/bin by default) as choir-node, choir-ssh, choir and choir-mcp; each archive is checked against the SHA256 beside it on the shelf. When that directory is not on your PATH, the installer adds one line to your shell’s startup file so new terminals find it, and prints the export for the terminal you ran it in.

The node renders that installer from the Host you asked it on, so the script and the archives come from the same origin and no address is configured anywhere. Any node with a shelf serves its own: https://<node>/download/.

From source instead, needing only rustup:

cargo install --git https://choirs.dev/choir/choir.git choir-cli choir-node

Neither package pulls in choir-actor, the one crate that needs this workspace’s LIBSQLITE3_FLAGS.

From a checkout you already have: the toolchain and headers the README lists, then:

cargo build --release -p choir-node -p choir-cli
export PATH="$PWD/target/release:$PATH"

choir node status and choir --version report the built commit from a stamp the release workflow sets.

§One command

choir host
you haverunyou get
a laptop, or a box nobody else reacheschoir hosthttp://127.0.0.1:8417, in seconds, no certificate
a name pointing at this boxchoir host --domain node.examplehttps://node.example:8417
a VPS and no namechoir host --publichttps://<this-ip>.sslip.io:8417

It prints the URL people use and, with --invite, one invite link:

choir host --repo me/thing.git --invite Ada

The two public modes take two commands. Obtaining a certificate is privileged and choir host does not run sudo: it prints the line to paste and exits 3. Paste it, run the same choir host again, and it continues.

$ choir host --domain node.example

  ok  state           /home/you/.choir — credential, key, trusted keys

  next: a certificate for node.example has to be issued as root.
  certbot writes /etc/letsencrypt, and the renewal hook that keeps
  this working for the next two years lives there too. Paste this:

    sudo choir node tls node.example --user you --port 8417
    sudo ufw allow 8417/tcp  &&  sudo ufw allow 80/tcp

  then: choir host --domain node.example

Dry-run the certificate step without spending a Let’s Encrypt rate-limit slot:

sudo choir node tls node.example --user "$(id -un)" --dry-run

§Why a public bind needs a certificate at all

choir-node refuses a non-loopback bind without TLS (invariant 9). There is no flag to soften it.

§--public, and why the address is ugly

choir host --public uses an sslip.io name built from the box’s address, 203-0-113-7.sslip.io for 203.0.113.7, so a certificate can be issued with no DNS. Replace it any time:

choir host --domain the-name-you-bought.example

sslip.io and nip.io share one Let’s Encrypt rate-limit pool. If issuance fails for that reason, --public-name <name> takes any name pointing at this box.

§What it did

Every step is a command you could run yourself:

stepwhatundo
statechoir init: ~/.choir with the credential, actor key, trusted keys, and .choir/configdelete ~/.choir
acl + accounts~/.choir/acl granting the operator everything, and an empty ~/.choir/accounts.jsonldelete either file
certificatechoir node tls: certbot, a renewal deploy hook, the pair projected where the node can read it, and ~/.choir/tls.enabledsudo rm /etc/letsencrypt/renewal-hooks/deploy/choir-tls, sudo certbot delete
address~/.choir/public-url, and .choir/config pointed hereedit either
supervisedchoir node install: a launchd agent or a systemd --user unitchoir node uninstall
healthypolls /healthz
repositorychoir repo create, with --repo
invitedchoir invite, with --invitechoir revoke

Two things it prints but never runs:

  • the firewall. ufw or firewalld, opening the serving port, and port 80 when HTTP-01 is the challenge (renewals rebind it every ~60 days).
  • linger. Without loginctl enable-linger, systemd stops a --user unit at logout. choir host stops with the line to paste; --yes accepts a node that dies at logout.

§TLS, and what happens every 60 days

The daemon terminates TLS itself with --tls-cert and --tls-key. An nginx or Caddy front is a valid topology; it is not required.

The daemon reads its certificate once, at bind. The certbot deploy hook choir node tls installs re-projects the pair into ~/.choir/tls/ and restarts the unit. Renewal runs on certbot’s timer.

Root for certbot, an unprivileged account for the node; the node’s copy of the pair is a copy for that reason.

The ACME account is registered without an email. sudo certbot renew --dry-run is the manual check; choir doctor reports the expiry date.

§Checking it

choir doctor

On a hosting machine it adds six rows: bind address, TLS on, certificate expiry, linger, unit loaded and running, and whether the public URL answers (asked from the box, so it proves name and certificate, not the firewall).

§Upgrading

choir node upgrade --from https://<node>      # that node's shelf, its installer run here
choir node upgrade --source ~/choir-build     # a checkout: cargo build --release there

Either way: the newer binaries are placed over the ones in the directory this choir runs from (--into <dir> names another), by rename, so a build that fails leaves the old ones untouched; the supervised node is restarted; and the build stamp is read back from the running daemon’s /api/view and compared with what was installed. A node that reports a different commit than the binary just placed exits 1 with both. --from wants a directory called bin, since that is where an installer writes; --dry-run prints the plan. The command runs on the node host and never runs ssh; the hop that types it is yours.

§Uninstalling

choir node uninstall

Removes the unit and stops the node. Keeps ~/.choir: keys, repositories and the op log. If a renewal hook is installed it prints the two sudo lines that remove it and the certificate.

§In a container

Dockerfile at the repository root builds from source and runs the daemon as an unprivileged user with /var/lib/choir as a volume. Loopback mode and bring-your-own-certificate mode; no ACME client in the daemon. No compose file: the volume and the port are one flag each.

§By hand

choir init                                  # the same layout, without a service manager
choir node serve                            # runs here, in this terminal
choir node serve -- --reviewers-file ~/.choir/reviewers   # any daemon flag, after `--`

choir node serve derives root, port, credential and trusted-key file from what choir init wrote, then execs the daemon. Three files switch behaviour by existing:

fileeffect
~/.choir/tls.enabledtwo lines, cert path then key path: binds 0.0.0.0 with that pair
~/.choir/acl--acl-file
~/.choir/accounts.jsonl--accounts-file (refused without an ACL)

Supervised, step by step:

choir init                      # mint ~/.choir: credential, key, trusted keys, config
choir node install              # hand it to launchd (macOS) or systemd (Linux)
choir node status               # health, the commit serving, sequencer position
choir repo create me/thing.git  # a repository on the running node
choir repo url me/thing.git     # the clone URL, and the git config to go with it
choir node logs                 # the tail of the daemon log
# choir node restart | stop | uninstall   -- uninstall keeps ~/.choir

choir node install writes a unit that runs choir node serve. Daemon flags go after -- and are recorded in the unit:

choir node install -- --acl-file ~/.choir/acl --rate-limit-api 60

The daemon can be run directly:

choir-node /tmp/choir-repos 8417 \
  --create owner/demo.git \
  --auth-file ~/.choir/auth \
  --keys-file ~/.choir/keys

Useful flags: --bind, --tls-cert / --tls-key, --acl-file <file> (required before a second credential), --request-log <file> and --rate-limit-api / --rate-limit-git (also required before a second credential), --quota-push-bytes / --quota-workspaces, --api-body-limit, --batch-limit, --ready-min-free-bytes, --read-only-browser, --journal <file>, --require-assignment, --protected-refs <file>, --require-review, --reviewer-conflict-graph <file> with --reviewer-conflict-distance <hops>, --review-retention <count>, and --review-lapse-after-secs <seconds>. Authenticated operations endpoints: /healthz, /readyz, /metrics. Flag reference: crates/choir-node/src/main.rs module docs, or AGENTS.md.

File formats (all mode 0600)

FileFormat
--auth-fileuser:token per line; authentication only, pair with --acl-file
--acl-file<user> <repo|*|@node> <level> per line; levels read < propose < write < own, and auditor on @node
--keys-file<64-hex> or <channel> <64-hex> (bound key)
--reviewers-filechannel name per line; re-read on each draw
--protected-refsowner/repo.git:refs/heads/main (trailing * ok)
--reviewer-conflict-graphundirected operator operator edges; pair with an explicit maximum hop distance

Hot-reload: trusted keys, channel bindings, push-certificate signers, reviewers, and ACL grants take effect on the next request.

[!WARNING] Without --acl-file, every credential reaches every repository. A second user:token line can clone every repo, push to any unprotected ref and provision workspaces anywhere. Do not issue a second credential without an ACL. choir host writes one.

§Serving your own binaries

--downloads-dir <dir> publishes one directory at /download/, with no credential (D79). Three things are served there and only three: an index of the directory, any file in it by exact name, and install.sh, which the node renders from the Host of the request rather than reading off the shelf. That is what lets the script fetch from the node it came from without anybody configuring an address.

The installers run ./choirctl deploy with the flag whenever ~/.choir/downloads exists, so creating the directory is the switch:

./choirctl publish-release            # or: publish-release v0.1.0
./choirctl deploy

publish-release reads owner/name from ~/.choir/release-repo and copies that release’s archives and digests onto the shelf. It deliberately leaves two things behind: the packaging tool’s own *-installer.sh, which carries the release host baked in at build time and would send a reader straight back to it, and source.tar.gz, which the node already serves over git at a revision a reader can name.

The node is not the builder. CI builds and publishes; this changes where a reader fetches from. The digests served beside the archives prove the transfer, not the build, so the trust boundary is TLS plus whoever runs the node.

Names are one path segment of [A-Za-z0-9._-], dotfiles are refused and symlinks are not followed, so the shelf can only hand out a regular file an operator put directly in it.

§Backups

On the node host:

choir backup take /srv/backups/choir          # now, then verified
choir backup schedule /srv/backups/choir      # every hour; --every <seconds> for another interval
choir backup unschedule

A backup is the log, the node fingerprint, the attestation, the policy files as one archive and one git bundle per repository, in the shape choir backup verify reads and choir backup restore unpacks, never a key or a credential. Each take checks that the node’s log extends the copy already in the directory, is contiguous from seq 0 and verifies as a chain, keeps a bundle whose refs have not moved, and writes everything into a sibling directory before moving it into place, so a run cut short leaves the previous backup as it was. The directory must not be inside the state directory. The timer is a launchd job or a systemd timer running the same command; its runs are logged to backup.log beside the node’s own. Copying the directory off the machine is still the operator’s job, and docs/runbook-restore.md is what to do with it.

§Followers

A follower is a remote on a bare repository under the root that every landing is pushed to (D21): a mirror on a forge, a second node’s seed input, a bare repository on another disk. On the node host:

choir repo follower add me/thing.git mirror git@forge.example:me/thing.git
choir node restart                            # once; the marker makes `node serve` pass --followers
choir repo follower list
choir repo follower push                      # by hand, every repository, every remote

Branches and tags are pushed, never forced and never pruned, so a follower that diverged from the node is a push that refuses and a line in <root>/.choir/followers.jsonl, not a rewrite. Nothing is fetched: the node is canonical and a follower copies it. The daemon pushes only when started with --followers, which choir node serve derives from the marker add writes; without it a remote is inert, which is what a choir-bridge mirror with an upstream origin relies on.

§Behind a TLS proxy

For the private beta, bind 127.0.0.1 and terminate TLS at a hardened reverse proxy. Do not use the legacy direct-TLS installer. The beta service renderer requires an ACL, protected-ref review policy, scoped operations, operator-issued auth, and the read-only browser.

docs/private-beta-runbook.md covers the network hold, renderers, backups, CI packaging, staging promotion, monitoring, rollback and go-live receipts.

scripts/flip/RUNBOOK.md is the operator’s own dogfood procedure; not the page to start from.

§A copy is a seed

A node that dies takes its repositories with it, unless a copy of them exists somewhere else. The cheapest copy needs no choir process at all:

choir-node --export <root> <dir>        # on the node: the log, one git bundle per repository, a manifest
# copy <dir> anywhere: another disk, an object store, a static web host
choir-node --verify-export <dir>        # on the copy, with the same binary

--verify-export folds the log and requires every ref the view names to be in that repository’s bundle at the same oid (D61), so a copy that verifies is the node’s history and the objects that history points at, checked on the machine that holds it. choir-node --import <dir> <root> turns it back into a node.

What the copy proves, and what it does not:

  • It proves integrity. The log is a hash chain, each entry carries its author’s signature, and the bundles hold every commit the log names.
  • It does not prove freshness. A copy is the node as of the moment it was taken. Nothing in it says the node has not moved on since, or that the copy you are holding is the latest one taken.
  • It does not witness. A copy made by the node’s own operator is the node’s own word. It cannot tell you whether a second reader was shown the same history. That needs a seed run by somebody else; see SYNC.md for what a seed signs and how two statements are compared.

Secrets and policy files are never in an export, by construction and then by inspection; docs/runbook-restore.md covers what a restore needs beside it.

§Run a seed

A seed is a live copy of another node’s log, its home (D80). It reads the home’s log page by page, checks every page before keeping any of it, fetches the git objects the log names, serves what it verified, and signs a statement about the attestation it saw. It writes nothing: every write sent to it, a git push included, is answered 421 not_home with the home’s address. SYNC.md is the contract.

One command, run twice, the same shape as choir host:

choir seed https://<home-host> --name seed-a

The first run mints the seed’s node key where the daemon will look for it and stops, exit 3, printing what the home’s operator pastes:

  keys file:   seed-a <64-hex>
  binding:     choir bind https://<home-host> <home-root>/.choir/node.key seed-a <64-hex>
  auth file:   seed-a:<a token minted there>
  acl file:    seed-a @node auditor
               seed-a * read

On the home, nothing seed-specific, only the files it already has: the keys line registers the key; the binding makes the key on its statements a name; the credential and the two grants admit it. @node auditor is the node-wide read /api/log needs, and there is no anonymous seed. The log grant does not reach git, so * read is what lets it fetch; name repositories instead of * to seed only some.

The second run, with the credential the home issued as a 0600 file:

choir seed https://<home-host> --name seed-a --credential ./seed-a.auth

It copies the credential into the state directory, mints a credential and an ACL for the seed’s own readers, writes the marker choir node serve reads, installs the unit under launchd or systemd, waits for the seed to answer, and prints its URL. --port <n> picks the port, --foreground execs the daemon instead of installing a unit, --archival makes a seed that binds nothing: it replicates, stores, fetches git and signs statements into its root, and serves nobody. The laptop, the CI sandbox, the machine behind NAT. After that choir node status|logs|restart|stop|uninstall and choir doctor treat it as the node it is.

By hand, the daemon lines the command composes. The credential file holds the one seed-a:<token> line, mode 0600; it reaches curl on stdin and git through its environment, never an argv or a URL. A serving seed takes its own readers’ --auth-file and --acl-file like any node:

choir-node /srv/choir-seed 8418 \
  --seed https://<home-host> \
  --seed-credential ~/.choir/seed-credential \
  --auth-file ~/.choir/seed-auth \
  --acl-file ~/.choir/seed-acl
choir-node /srv/choir-archive --seed https://<home-host> --seed-credential ~/.choir/seed-credential   # archival

A seed that serves beyond loopback is a node like any other and needs --bind with --tls-cert and --tls-key; it refuses otherwise. It takes no --keys-file (it learns every key from the home’s /api/signers) and no --create. On first contact it pins the home’s node key at .choir/home.fingerprint and refuses a home that later signs with another one.

Where it stands is replica in its GET /api/view, and one line in its page header: seed of <home>, N behind. An entry that fails a check, or that its build cannot fold, halts replication there; the seed keeps serving what it verified and says halted with the seq and the reason. --seed-strict exits nonzero instead, for a supervisor to notice.

Seeds upgrade before homes. A seed whose binary cannot fold an op the home admitted halts rather than skipping it, because a copy that drops what it does not understand is a fork. Upgrading the seeds first means a new op kind never reaches a seed that cannot read it.

Readers point choir at seeds with seeds = <url>[, <url>] beside node = in .choir/config; choir doctor then compares each seed’s statement with what the home shows them, fails on a fork, and warns on a seed that halted, with the seq and the reason.

Re-exports§

pub use platform::Platform;

Modules§

accounts
Account and credential self-service (D36).
acl
Per-repository authorization (D29).
followers
Followers (D21): after a ref lands, push it to the remotes the operator added to the bare repository.
hooks
Outbound ref-landed webhooks (D32): “something moved, go run this”.
limits
Request accounting and admission control (D33): who did what, and how much of it they may do.
platform
The platform API: signed op submission and view queries over HTTP.
portable
Exporting a node root, importing one, and settling what an export claims (§E).
profile
Who an actor is, from what the log already says about them (D63).
provision
Instant workspace provisioning (POST /api/workspace) — the D21 “productized provisioning” wedge feature.
queue
The node’s own merge queue (D68).
queue_api
Driving the node’s merge queue from outside (D5, D68).
quota
Per-user quotas (D37): how much of the node one credential may hold.
reject
Rejections that name the repair.
replica
A seed (D80): a node whose log is a copy of another node’s, taken one way, verified before it is kept, and never written to by anybody here.
ssh
SSH transport for git (D31): the host’s sshd, a forced command, and this shim.

Structs§

Node
A running node daemon serving repos under a root directory.
TrustedKey
One line of the trusted-keys file: a public key the node accepts, and optionally the channel name its holder is allowed to speak as.

Constants§

BUILD_COMMIT
The commit this binary was built from, or the literal unknown when the build had no way to find out. See build.rs.
BUILD_DIRTY
Whether the build tree had uncommitted changes. Only meaningful under BUILD_SOURCE == "git", and even then best-effort: cargo cannot rerun the build script on every source edit, so this can be stale where BUILD_COMMIT cannot.
BUILD_SOURCE
Where BUILD_COMMIT came from: env (the installer passed CHOIR_GIT_HEAD, the sound path), git (best-effort at build time), or unavailable (no commit could be determined).
DEFAULT_API_BODY_BYTES
Default maximum body size for every /api/... request: 1 MiB.
DEFAULT_READY_MIN_FREE_BYTES
Readiness refuses when the filesystem reports less than 1 GiB free.

Functions§

build_json
The build stamp as served under /api/view.build.
build_line
One line naming the running binary’s provenance, for the startup log.
git_invocations
How many times this process has shelled out to git while serving.
parse_keys_file
Parses a trusted-keys file. One key per line, # comments and blank lines skipped, in either form:
ssh_ed25519_pubkey
Renders a raw ed25519 public key in OpenSSH ssh-ed25519 <b64> form (the wire blob is two length-prefixed strings: key type, key bytes).
ui_tokens_css
The design tokens from ui.css, alone, for the documentation book.
write_allowed_signers
Writes <root>/.choir/allowed_signers — the file git’s ssh signature verification checks push certificates against. One line per actor: principal (the actor id) followed by the OpenSSH public key.

Type Aliases§

AuthTable
Per-actor credentials: username → token, checked as HTTP basic auth (the standard git-over-HTTP shape; every forge client speaks it).