WebSocket Event Stream

ELI5

The Engine’s WebSocket is a kitchen pass-through window: every time a chef finishes a dish (compress, extract, prune, cache update, metrics tick), they ring a bell with the dish name on it. Every browser holding a ticket hears every bell.

Technical Deep Dive

Source: nestr-tools/engine internal/server/websocket.go:14-30,60-62,100-130,149, internal/server/rest.go:138,529-545, internal/auth/middleware.go:34-51, internal/server/pellet_handlers.go:185,226,289 @e9180e4, read 2026-10-25. Event names re-verified; the emitters, wire shape and drop behaviour below were corrected in this revision.

engine/internal/server/websocket.go defines a fan-out hub. The handler upgrades the HTTP connection (after auth — see nestr-006) and registers a client; the hub keeps a set of clients and broadcasts JSON-encoded events to all of them.

Event Catalogue

Broadcast Sequence

sequenceDiagram
    participant H as REST Handler
    participant S as pellets.Store
    participant Hub as ws.Hub
    participant C1 as Client A
    participant C2 as Client B
    H->>S: Compress(source, level)
    S-->>H: *Pellet
    H->>Hub: Broadcast type pellet:compressed with payload
    Hub-->>C1: JSON frame
    Hub-->>C2: JSON frame
    H-->>H: respondJSON(success)

Client Lifecycle

• Upgrade succeeds → hub.register(client); client gets a buffered send channel.
• Client read pump: ignored payloads (server-to-client only in this revision).
• Client write pump: drains its 256-slot send channel into the socket. When a client’s buffer is full the hub skips that message for that client; the client stays connected (websocket.go:110-114).
• Broadcast itself is non-blocking: when the hub’s 256-slot broadcast channel is full the event is dropped for everyone (websocket.go:123-128).
• Disconnect → hub.unregister(client).

The wire format is JSON: {"type": "pellet:compressed", "payload": {...}} (WebSocketEvent, websocket.go:24-27); there is no timestamp field, and payload is omitted when empty. Unlike systems with a versioned envelope (e.g. choco’s EventEnvelope), nestr’s WS messages are not versioned — clients are expected to ignore unknown type values.

Key Terms

• Hub → a single in-process actor owning the client set and the broadcast channel.
• Send buffer → per-client bounded channel (256); overflow drops the message for that client, not the client.
• Event type → a string constant; the wire form is the same string with no namespace prefix.

Q&A

Q: How many event types does the hub emit? A: Five are declared (pellet:compressed, pellet:extracted, pellet:pruned, cache:updated, metrics:updated), but only the three pellet events are emitted anywhere in the engine.

Q: What carries the event payload — JSON, protobuf, or text? A: JSON: each frame is a marshalled WebSocketEvent, {"type": ..., "payload": ...}.

Q: Who calls hub.Broadcast — handlers or the cache layer? A: The REST handlers, after the store call returns: compress, extract and prune in pellet_handlers.go. The store itself never broadcasts.

Q: How does a browser authenticate the socket? A: With ?token=<jwt> on the upgrade URL (or an Authorization header from non-browser clients). The same EdDSA validator runs before the upgrade; with JWKS_URL unset, no check runs at all.

Q: What happens to events emitted while no clients are connected? A: They are dropped. The hub has no replay log; it is fan-out only, not pub/sub with retention.

Q: Can a client subscribe to only one event type? A: No — there is no subscription filter. Filtering is the client’s responsibility.

Q: Is delivery ordered across event types? A: Per-client yes (single goroutine writes the socket); cross-client ordering is not guaranteed because each client has its own send goroutine.

Examples

A client that wants live pellet updates connects to /ws?token=<jwt>, switches on type, and refetches GET /api/pellets on pellet:compressed or pellet:pruned. The web dashboard at 6f42cca does not do this: its only WebSocket client is the Olly chat (src/api/olly.ts), and the useWebSocket.ts hook the previous revision cited is no longer in src/hooks/. Pellet lists rely on polling.