Production runs on GPU racks, rented or on-prem; a laptop install is onboarding. One install is one supervised process
tree started by one command. main is the control plane: it owns the database, the jobs, the policies and the
configuration. Gateways and engines are the serving plane: they answer inference requests from a cached configuration,
so they keep serving while main restarts. Workers are the batch plane: they run data preparation, training and eval.
Workers and engines talk to main only over HTTP, so a GPU box elsewhere is the same program with a URL, a token and a
pool name.
| Doc |
Covers |
| Security |
Policies, trust zones, ports |
| state.md |
Failure modes, database, artifacts, several mains |
| Serving |
Planes, request path, engines, routing, compute pools, machine heartbeats and drain |
| Models |
Catalogue, Add from Hugging Face, sources, import formats |
| jobs.md |
Job contract, lifecycle, retries |
Endpoints: web/openapi.json. Tables: migrations.
| Role |
Plane |
Local (minibrain serve) |
Deployed |
Scales by |
Database |
| main |
Control |
The parent process |
Its own service; two or more with Postgres |
Copies behind a load balancer |
The only reader and writer |
| gateway |
Serving |
Inside main |
minibrain gateway |
Copies behind a load balancer |
Cached config snapshot |
| engine |
Serving |
A supervised child per model |
A child of a worker in the deployment’s pool, or minibrain engine --pool … |
Replicas per deployment, across boxes |
HTTP to main |
| worker |
Batch |
A supervised child |
minibrain worker --pool … on each GPU box |
More boxes per pool |
HTTP to main |
| training / eval |
Batch |
A child of the worker |
Same, on the worker’s box |
One per job |
None |
Same roles, same code; only where they run changes. The three pictures share one layout: callers on the left, the
inference path across the top, main and its state in the middle, GPU pools on the right. Colour is the plane: blue
control, green serving, amber batch. diagrams/topology.html is the interactive version,
with failure drills.
|
One machine |
Local main, remote GPUs |
Production |
| main |
minibrain serve |
minibrain serve on a laptop or small VM |
Two or more behind a load balancer |
| Gateways |
Inside main |
Inside main |
minibrain gateway ×N behind a load balancer |
| Workers and engines |
Local children, pool “This machine” |
On GPU boxes, one pool per rack |
GPU racks, several pools |
| Database |
SQLite |
SQLite |
Postgres, highly available |
| Artifacts |
Folder |
Folder |
S3-compatible |
| Stage |
Who starts workers and engines |
| v1 |
People or their tooling: minibrain worker and minibrain engine on each box (systemd, Compose, Slurm) |
| v2 |
A Kubernetes launcher: main creates Jobs for training and Deployments for engine replicas, autoscaled |
| Later |
A cloud VM launcher for rented GPUs |
| Decision |
Why |
Cost |
| One control plane: main and one database |
One source of truth, simple operations |
Changes need main; HA needs Postgres and several mains |
| Workers and engines pull from main over HTTP |
Outbound-only GPU boxes, no database credentials outside main, one SQLite writer |
Artifacts flow through main; base weights in a bucket come straight from it on links main signs, so GPU boxes also reach the bucket |
| HTTP, not gRPC |
The job channel carries a few requests a second and inference is GPU-bound; HTTP streaming serves both |
Larger messages than protobuf |
| Own jobs table |
Leases and fencing in one table, no broker, works on SQLite. Procrastinate is Postgres-only, Huey keeps a second record without leases, Celery / RQ / Dramatiq need a broker, Temporal / Hatchet / Prefect are separate servers |
We own the bugs; DBOS Transact is the candidate for multi-step pipelines |
| Jobs found by polling |
No broker |
Up to one poll interval of latency; Postgres LISTEN/NOTIFY later |
| No coordination service |
The database coordinates jobs; a Postgres advisory lock picks the leader |
HA depends on Postgres HA (Patroni or managed) |
| Config up to ~1 s stale |
No database read per inference request |
A revoked key works for up to a second |
| One base model per engine process |
Simple lifecycle, many adapters per model |
More processes; replicas scale out, autoscaled on Kubernetes |
| One package for every role |
One install, one version |
A large install; split extras later |
| Python gateway |
Same code and models as main |
More replicas than a native proxy; Go or Rust only if ever needed |
| No Compose or Kubernetes requirement |
Docker on macOS has no GPU; uvx minibrain serve works on a laptop and air-gapped |
Workers and engines started by hand in v1; a Kubernetes launcher in v2 |