Producer / Reader Split

kahn beginner 4 min read

ELI5

A factory has two roles: a stamping machine that punches metal blanks onto a conveyor (the orchestrator), and a clipboard-carrying inspector who only watches what comes by (Scope). The inspector never re-stamps a blank; the stamper never asks the inspector for permission.

Technical Deep Dive

Post-Taskset-2 inversion: the orchestrator is the producer, Scope (backend/kahn/) is a pure reader of .kahn/archive/. There is no .kahn/state/ watcher anymore — the archive volume is the only shared surface.

Roles

Side	Path	Responsibility
Producer	`core/orchestrator.py`	Walks the DAG, runs Ralph, emits via `contracts.kahn_emit`
Reader	`backend/kahn/server.py`	FastAPI; reads `.kahn/archive/runs/<id>/{transitions.jsonl, graph.json, summary.json}` and reduces in-memory

Data Flow

flowchart LR
    O["core/orchestrator.py"] -->|append| T["transitions.jsonl"]
    O -->|once at run_start| G["graph.json"]
    O -->|once at run_end| S["summary.json"]
    subgraph Archive [".kahn/archive/runs/&lt;run_id&gt;/"]
      T
      G
      S
    end
    Archive -->|read-only scan| R["backend/kahn/server.py"]
    R -->|HTTP/WS| C["frontend SPA"]

Invariants

I-1: Scope writes only under .kahn/archive/. Verified by an strace invariant test.
I-9: OSS mode has no database — JSON file read + in-memory reduce only.
I-8: Storage backend swap (OSS file vs cloud Postgres) does not change aggregator/diagnostics/server layers.

Key Terms

Producer → A process that emits transitions.jsonl lines (orchestrator, audit-runner, sister-repo agents).
Scope → The read-only observability service in backend/kahn/.
Archive volume → .kahn/archive/, the single shared surface between producer and reader.

Q&A

Q: Why was the previous .kahn/state/ watcher removed? A: It coupled Scope to the orchestrator’s working directory and forced two write surfaces. The Taskset 2 inversion made transitions.jsonl the canonical event log so Scope only needs the archive.

Q: Can Scope mutate a run from the UI? A: No. Tenet “Read-only” — to kill a run you Ctrl-C the orchestrator or rm state files. The UI reflects, it does not act.

Q: What guarantees Scope never writes outside .kahn/archive/? A: Invariant I-1, enforced by an strace test that asserts every file write Scope performs lives under that prefix.

Examples

When audit-runner runs in choco-hq, it appends events to .kahn/archive/runs/<run_id>/transitions.jsonl. Scope, possibly running on another machine via NFS or S3-fuse, reads the same file. No RPC, no shared DB, no lockfile.