Pipeline Config Schema

ELI5

A pipeline is a list of stages; each stage is a list of steps; each step is one shell command. The schema (libs/api-contracts/schemas/pipeline-config.json) is the contract every YAML file is checked against before the engine will run it.

Technical Deep Dive

The canonical schema is libs/api-contracts/schemas/pipeline-config.json, JSON-Schema draft-07, $id: https://sparki.tools/schemas/pipeline-config.json. The Go side mirrors it in internal/pipeline/yaml.go (PipelineConfig.ToYAML, Validate).

Top-Level Shape

PipelineStage

PipelineStep

Class Diagram

classDiagram
  class PipelineConfig {
    +string version
    +PipelineStage[] stages
    +map~string,string~ environment
    +Validate() error
    +ToYAML() string
  }
  class PipelineStage {
    +string name
    +PipelineStep[] steps
  }
  class PipelineStep {
    +string name
    +string command
    +map~string,string~ env
    +int retry
    +string timeout
  }
  PipelineConfig --> PipelineStage : 1..*
  PipelineStage --> PipelineStep : 1..*

Validation Flow

flowchart LR
  YAML[YAML payload] --> PARSE[parse to PipelineConfig]
  PARSE --> SCHEMA[JSON-Schema draft-07]
  SCHEMA -->|name empty| FAIL[reject]
  SCHEMA -->|stages 0| FAIL
  SCHEMA -->|retry > 5| FAIL
  SCHEMA -->|timeout pattern miss| FAIL
  SCHEMA -->|ok| GO[PipelineConfig.Validate]
  GO --> ENQ[enqueue BuildJob]

Key Terms

• stage → a named bag of steps; stages run in array order
• step → a single shell command with optional retry/timeout/env
• retry → integer 0–5; the schema caps retries at 5 to bound runaway loops
• timeout pattern → ^[0-9]+[smh]$ — only seconds, minutes, or hours

Q&A

Q: Can a stage have zero steps? A: No. The schema sets minItems: 1 on PipelineStage.steps. A zero-step stage fails validation.

Q: What is the highest value retry accepts? A: 5. retry is integer, minimum 0, maximum 5, default 0.

Q: Is 5m30s a valid timeout? A: No. The pattern is ^[0-9]+[smh]$ — exactly one numeric run followed by one of s, m, h. Compound durations like 5m30s fail validation.

Q: Can two stages share an environment override? A: Yes via top-level environment; per-step env overrides shadow the global map for that step only.

Examples

version: v1
environment:
  CI: "true"
stages:
  - name: build
    steps:
      - name: compile
        command: go build ./...
        timeout: 5m
        retry: 1
  - name: test
    steps:
      - name: unit
        command: go test ./...
        timeout: 10m