Per-IP Fixed-Window Rate Limiter (engine only, as deployed)

ELI5

Each visiting IP gets a ticket with N punches, and a fresh ticket prints every minute. Run out early and the door says 429 and tells you when the next ticket prints. In production only the engine has this door. The MCP server has the same door in a file that production never runs.

Technical Deep Dive

Source: traceo-cat/traceo_mcp_server/traceo_mcp_server/rate_limit.py:20-132, config.py:98-105, app.py:108-111, server.py:88-160 (no limiter), engine/src/engine/server.py:162-168; live headers from GET https://engine.traceo.cat/jobs and GET https://mcp.traceo.cat/api/requirements; traceo-cat origin/main 7d647ab, read 2026-10-25

Where it is installed

The v1.0 card described a 100/min MCP limiter. It exists in code, but no deployed process installs it (see traceo-020). The only throttle on the MCP surface is whatever Railway’s edge applies, which was not examined for this card.

Configuration

_parse_rate_limit turns "50/minute" into (50, 60). The unit maps to second/minute/hour/day, and anything unrecognised falls back to 60 s. The 429 body still says “per minute” whatever the unit is.

Algorithm

flowchart TD
  REQ["request from IP X"] --> EX{"path exempt?"}
  EX -- yes --> PASS["skip, no headers"]
  EX -- no --> B["bucket = _buckets[X]"]
  B --> RS{"now - window_start >= window (60 s default)?"}
  RS -- yes --> NEW["count = 0, window_start = now"] --> INC["count += 1"]
  RS -- no --> INC
  INC --> CHK{"count <= limit?"}
  CHK -- yes --> OK["call next, add X-RateLimit-* headers"]
  CHK -- no --> DENY["429 + Retry-After"]

Engine exempt paths are /health, /ready and /metrics (server.py:166-168), so probes and Prometheus scrapes never touch a bucket.

Order relative to auth

FastAPI runs the last-added middleware first. The engine adds the limiter after AuthMiddleware, so the limiter is outermost and runs before auth. An unauthenticated request spends budget and still receives rate-limit headers on its 401, as the live probe shows.

sequenceDiagram
  participant C as Client
  participant RL as RateLimit (outermost)
  participant A as AuthMiddleware
  participant H as /jobs handler
  C->>RL: GET /jobs (no token)
  RL->>RL: count += 1 for client IP
  RL->>A: call next
  A-->>RL: 401
  RL-->>C: 401 + x-ratelimit-remaining 49

Storage

_buckets is an in-process defaultdict keyed by request.client.host. It does not survive a restart and does not coordinate across replicas. Both Railway services run numReplicas = 1. Behind a proxy, request.client.host may be the proxy’s address rather than the caller’s. Whether Railway forwards the real client IP to this field was not verified (UNVERIFIED).

Key Terms

• Fixed window → a counter that hard-resets every 60 seconds. It is cheaper than a token bucket but lets bursts through at the boundary.
• Bucket → the (count, window_start) pair kept for each key.
• Exempt → a path the limiter skips entirely, without counting it or adding headers.

Q&A

Q: What is the window length in seconds? A: 60 with the shipped defaults. _parse_rate_limit("50/minute") returns (50, 60), and setting ENGINE_RATE_LIMIT=10/second would make it 1.

Q: Which deployed service actually enforces a rate limit? A: Only the engine. The MCP limiter is wired in app.py, which production does not run. The live MCP host returns no x-ratelimit-* headers.

Q: Where does the limiter sit relative to auth on the engine? A: Outside it. The limiter was added last, so it runs first. Unauthenticated requests consume budget, and their 401s carry rate-limit headers.

Q: Where is the bucket state stored? A: In process memory (_buckets). It is lost on restart and not shared between replicas.

Examples

An attacker sends 200 unauthenticated requests to the engine at the 0:59→1:00 boundary. Fifty pass the limiter in the first window and are refused by auth. Fifty more pass in the next window. The rest get 429. The same 200 requests to the MCP server all reach AuthMiddleware, because there is no limiter in front of it.