Pipeline Config: Contract Stages, Stored Steps, Triggers

ELI5

You describe a pipeline as stages of steps, like chapters of a recipe. The engine files it away as one flat list of steps with a note of which chapter each came from, and that flat list is what the build worker cooks from. Triggers say when to cook: on a push, by hand, or on a schedule.

Technical Deep Dive

Source: sparki-tools/api-contracts openapi/sparki-api.yaml:1182-1246, schemas/pipeline-config.json:1-89 @2b45241; sparki-tools/api-engine pkg/api/handlers/contract.go:21-79,191-197,231-263, internal/pipeline/generator.go:24-47, pkg/workerapi/workerapi.go:72-80 @0b0309f, read 2026-10-25.

Two shapes, one pipeline

contract.go converts both ways: on write it parses timeout with time.ParseDuration and rounds up to whole seconds (contract.go:257-263); on read it renders timeout_seconds back as a Go duration string (contract.go:196-197).

classDiagram
  class PipelineConfig {
    +string version
    +PipelineStage[] stages
    +Trigger[] triggers
    +map environment
  }
  class Trigger {
    +string type
    +map conditions
  }
  class PipelineStage {
    +string name
    +PipelineStep[] steps
  }
  class PipelineStep {
    +string name
    +string command
    +string image
    +string timeout
    +int retry
    +map env
  }
  PipelineConfig --> Trigger : triggers
  PipelineConfig --> PipelineStage : 1..*
  PipelineStage --> PipelineStep : 1..*

Triggers

OpenAPI Trigger is {type, conditions} with type required (example git_push) and free-form conditions (sparki-api.yaml:1202-1212). The engine’s pipeline.Trigger types conditions as map[string]string and names git_push, manual, schedule (generator.go:44-47). Legacy object-shaped triggers from the demo seed ({"push": {...}}) are normalised to [{type: "git_push", ...}] on read (contract.go:88-120).

Write path

flowchart LR
  REQ["POST /api/v1/pipelines contract stages"] --> CONV["contract.go: stages to flat steps"]
  CONV -->|"timeout via time.ParseDuration"| SEC["timeout_seconds rounded up"]
  CONV -->|"retry below 0"| REJ["400"]
  SEC --> STORE[("pipelines.configuration")]
  STORE --> CLAIM["worker claim: workerapi.Step"]
  CLAIM --> RUN["step runs in container with min of step timeout and job deadline"]

Where the schemas disagree

• schemas/pipeline-config.json pins timeout to ^[0-9]+[smh]$ and retry to 0..5. The engine accepts any time.ParseDuration string (so 5m30s passes) and rejects only negative retries.
• OpenAPI PipelineConfig requires triggers; the JSON schema does not mention triggers at all and requires version and stages.
• retry is persisted but workerapi.Step has no retry field, so no step is retried.

Key Terms

• stage_index → 1-based marker on each stored step so adjacent stages with the same name stay apart.
• timeout_seconds → the stored and wire form of a step timeout; 0 means the worker default.
• Trigger → {type, conditions}; git_push, manual or schedule in the engine’s generator.

Q&A

Q: What shape does the API accept and return, and what shape is stored and run? A: The API speaks contract stages. The engine stores and the worker runs a flat list of steps, each tagged with its stage index; contract.go maps between them.

Q: Is 5m30s a valid step timeout, and does the answer depend on where you check it? A: Yes, it depends. The JSON schema regex ^[0-9]+[smh]$ rejects it; the engine parses it with time.ParseDuration, accepts it, and stores 330 seconds.

Q: Does the worker honour retry? A: No. The value is stored and echoed back, but the worker job carries no retry field and a failed step fails the build.

Examples

version: v1
triggers:
  - type: git_push
    conditions: { branches: "main" }
stages:
  - name: build
    steps:
      - name: compile
        image: golang:1.24
        command: go build ./...
        timeout: 5m
  - name: test
    steps:
      - name: unit
        command: go test ./...
        timeout: 10m

Stored, this becomes two flat steps with stage_index 1 and 2 and timeout_seconds 300 and 600; the second runs in the worker default image node:20-bookworm-slim because it names none.