Service Topology Overview

ELI5

Sparki is a workshop with one main bench (api-engine) and a few specialist stations around it: a security desk (auth-shield), a deployer crane (deploy-loco), an observability rig (observability-storm), and a contract printer (smithy). Each station does one job and talks to the others over queues and HTTP.

Technical Deep Dive

The repo is a polyglot monorepo (go.work at the root) built around six backend surfaces under services/, plus apps, libs and infra.

Service Map

Topology

flowchart LR
  subgraph Clients
    W[web-app Next.js]
    T[sparki-tui Bubbletea]
    M[mobile-ios / mobile-android]
  end
  subgraph Backend
    AE[api-engine Go]
    AS[auth-shield Django]
    DL[deploy-loco Rust]
    SM[smithy Go]
    OS[observability-storm]
  end
  subgraph Infra
    PG[(PostgreSQL)]
    RMQ[(RabbitMQ)]
    RD[(Redis)]
  end
  W & T & M -->|REST/WS| AE
  W & M -->|OIDC/SAML| AS
  AE -->|JWT verify| AS
  AE -->|publish builds/deployments| RMQ
  DL -->|consume deployments| RMQ
  AE -->|state| PG
  DL -->|state| PG
  AS -->|state| PG
  AE -->|cache| RD
  AE -. metrics/traces/logs .-> OS
  DL -. metrics/traces/logs .-> OS
  AS -. metrics/traces/logs .-> OS

Why Loco Is Its Own Service

deploy-loco is a Rust binary (see services/deploy-loco/Cargo.toml: axum 0.7, sqlx 0.8, jsonwebtoken 9). It is split out because deployment is the slowest, most failure-prone path and benefits from process isolation: an api-engine restart should not abort an in-flight deploy. It consumes the deployments queue from RabbitMQ and writes to the shared deployments table in PostgreSQL.

Why Shield Is Python

auth-shield is a Django app (manage.py, config.settings) with apps/ for accounts, mfa, oauth, saml, workspaces, audit. Django was chosen for the auth surface because it ships SAML/OAuth ecosystem libraries that Go would have to assemble. Other services trust JWTs minted by shield.

Key Terms

• api-engine → the Go core service; everything pipeline-shaped lives here
• subsystem → a domain module under api-engine/subsystems/ (bind, link, loco-client, scan, score, forgery, run, etc.)
• deploy-loco → the Rust deployment orchestrator, separate process
• auth-shield → the Django auth service that mints JWTs the engine verifies
• observability-storm → the metrics/trace/log aggregation stack

Q&A

Q: Why is there a loco subsystem inside api-engine AND a deploy-loco service? A: The subsystem (api-engine/subsystems/loco) is the engine-side API client and orchestration glue. The standalone deploy-loco service is the actual worker that performs the deployment. They share data via PostgreSQL and RabbitMQ.

Q: What is build-forgery versus subsystems/forgery? A: services/build-forgery/ is a placeholder containing only release-please / renovate config; the real implementation lives at api-engine/subsystems/forgery/ (orchestrator, generator, scanner, validator, adapter).

Q: Does the engine talk to shield on every request? A: No. Shield issues signed JWTs; api-engine verifies signatures locally (jsonwebtoken-equivalent) and only calls shield for token introspection or session-management actions.

Examples

A push-to-deploy lifecycle crosses every service: web-app → api-engine REST (auth via shield-issued JWT) → engine enqueues builds → worker runs the pipeline → engine enqueues deployments → deploy-loco consumes, calls platform adapter (Railway, Render, Fly), writes back to the deployments row → storm scrapes metrics from each hop.