CI/CD Pipeline & Schema Propagation

iris beginner 4 min read

ELI5

The CI/CD pipeline is like an automated quality control system in a factory. Every time someone submits a design change (pull request), robots check: does it follow the blueprint? Does it fit with existing parts? Does it break any machines? If everything passes, the change is approved for the production line. When the blueprint (schema) changes, all downstream factories automatically get notified to update their tools.

Technical Deep Dive

Repository Structure

flowchart TB
  subgraph specs["iris-specs"]
    schema["JSON Schema / OpenAPI / Protobuf"]
  end
  subgraph core["Tier 1 — Core"]
    py_sdk["iris-sdk-python<br/>PyPI: iris-sdk"]
    ts_sdk["iris-sdk-typescript<br/>NPM: iris-hq/sdk"]
    iris_service["iris-service<br/>Docker"]
  end
  subgraph bridge["Tier 2 — Bridge"]
    mcp["iris-mcp-server<br/>CF Workers"]
    web["iris-web<br/>Vercel"]
  end
  subgraph federation["Tier 3 — Federation"]
    meridian["iris-meridian-adapter<br/>gRPC"]
  end
  schema --> py_sdk
  schema --> ts_sdk
  schema --> iris_service
  schema --> mcp
  schema --> meridian

CI/CD Workflow (iris-service)

flowchart TD
    A["Pull Request opened"] --> B["GitHub Actions triggered"]
    B --> C["Lint & Format"]
    C --> D["black / ruff / mypy"]
    D --> E{"Pass?"}
    E -->|No| F["Block merge"]
    E -->|Yes| G["Unit Tests"]
    G --> H["pytest + pytest-asyncio"]
    H --> I{"Pass?"}
    I -->|No| F
    I -->|Yes| J["Integration Tests"]
    J --> K["pytest integration/"]
    K --> L{"Pass?"}
    L -->|No| F
    L -->|Yes| M["Schema Validation"]
    M --> N["validate against iris.schema.json"]
    N --> O{"Pass?"}
    O -->|No| F
    O -->|Yes| P["Contract Compatibility"]
    P --> Q["OpenAPI conformance check"]
    Q --> R{"Pass?"}
    R -->|No| F
    R -->|Yes| S["Maintainer Review"]
    S --> T{"Approved?"}
    T -->|No| F
    T -->|Yes| U["Merge to main"]
    U --> V["Deploy to staging"]

GitHub Actions Workflow

File: .github/workflows/iris-service-ci.yml

name: IRIS Service CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.11', '3.12']
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: pip install -e ".[dev]"
      - name: Lint
        run: ruff check src/ && black --check src/
      - name: Type check
        run: mypy src/
      - name: Unit tests
        run: pytest tests/unit/ -v --cov=src --cov-report=xml
      - name: Integration tests
        run: pytest tests/integration/ -v
      - name: System tests (Docker)
        run: docker compose -f docker-compose.test.yaml up --abort-on-container-exit

Tool Configurations

Tool	Config File	Purpose
black	`pyproject.toml`	Code formatting; line-length 100; target py311+py312
ruff	`pyproject.toml`	Linting; selects E/W/F/I/C/B/UP rules; target py311
mypy	`pyproject.toml`	Type checking; strict optional; check untyped defs
pytest	`pytest.ini`	Test runner; async mode auto; custom markers

Schema Propagation Flow

flowchart LR
    A["iris-specs change<br/>iris.schema.json updated"] --> B["Downstream repos detect change"]
    B --> C["iris-sdk-python"]
    B --> D["iris-sdk-typescript"]
    B --> E["iris-service"]
    B --> F["iris-mcp-server"]
    B --> G["iris-meridian-adapter"]
    C --> H["Update bundled schemas"]
    D --> I["Regenerate types"]
    E --> J["Update validation logic"]
    F --> K["Update tool schemas"]
    G --> L["Update protobuf"]

Propagation mechanism:

iris-specs PR is merged
Downstream repos update their iris-specs submodule or dependency
CI in each downstream repo runs schema validation tests
Breaking changes require: major version bump, migration guide, 2-week deprecation notice

Test Pyramid

flowchart TD
    A["Test Suite"] --> B["Unit Tests<br/>~60% of suite"]
    A --> C["Integration Tests<br/>~30% of suite"]
    A --> D["System Tests<br/>~10% of suite"]

    B --> B1["Sprite validation"]
    B --> B2["Fingerprint computation"]
    B --> B3["Chain execution logic"]
    B --> B4["Gate evaluation"]

    C --> C1["API endpoint testing"]
    C --> C2["Registry operations"]
    C --> C3["SDK client tests"]

    D --> D1["Docker Compose stack"]
    D --> D2["End-to-end chain execution"]
    D --> D3["Observability integration"]

Docker Test Stack

services:
  iris-service-test:
    build: .
    environment:
      - ENV=test
      - DATABASE_URL=postgresql+asyncpg://iris:iris@db:5432/iris_test
    depends_on:
      - db
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: iris_test

Key Terms

CI/CD → Continuous Integration / Continuous Deployment; automated build, test, and deployment pipeline
Schema propagation → The process of distributing schema changes from iris-specs to downstream repositories
Contract compatibility → Ensuring API changes don’t break existing clients (OpenAPI conformance)
Test pyramid → Unit tests (majority) → Integration tests → System tests (smallest, slowest)
Linting → Automated code style and quality checks (black, ruff, mypy)
Deprecation notice → 2-week warning before removing or changing breaking API behaviour

Q&A

Q: How do I run tests locally? A: pytest tests/ for unit/integration tests. docker compose -f docker-compose.test.yaml up for system tests.

Q: What Python versions are tested in CI? A: Python 3.11 and 3.12. The pyproject.toml targets these versions explicitly.

Q: How do schema changes trigger downstream CI? A: Currently manual — downstream repos update their iris-specs reference. Future automation could use GitHub repository dispatch events or a central schema registry.

Q: What is the 2-week deprecation notice? A: Per GOVERNANCE.md, breaking contract changes require a 2-week notice period so downstream consumers can update. This applies to OpenAPI endpoint changes, not internal implementation changes.

Q: Can I skip CI for documentation changes? A: The current workflow triggers on all PRs. You can add path filters to .github/workflows/*.yml to skip CI for *.md changes.

Examples

The CI/CD pipeline is like a restaurant’s kitchen workflow:

Linting (black, ruff) = The prep cook washing and chopping vegetables to exact specifications
Type checking (mypy) = The sous chef verifying every ingredient is what the recipe says it is
Unit tests = Tasting each sauce individually before it goes on a dish
Integration tests = Assembling the full dish and making sure flavours work together
System tests (Docker) = Serving the dish to a test table during a full dinner service simulation
Schema propagation = When the head chef updates the menu, all stations (grill, pastry, bar) get new recipe cards
Breaking changes notice = Putting a “New Menu Coming Soon” sign outside so regulars aren’t surprised

neighbors on the map

Docker Compose & Observability Stack deploying iris-service locally or in production
Python SDK & CLI writing Python scripts that interact with IRIS