eva doctor Validation Rules

ELI5

eva doctor is the home inspector for every prompt in the library: it walks each room, checks the labels, makes sure every {{var}} mentioned in the recipe is declared in the meta, that the door verbs (aliases) don’t collide, and that the description card is the right shape to ship as an Anthropic skill.

Technical Deep Dive

Per-Prompt Checks (bin/eva:329-455)

flowchart TD
  start["iter_prompts()"] --> req["REQUIRED_META present and non-empty?"]
  req --> idm["meta.id == dirname?"]
  idm --> stat["status in {draft,tested,ready}?"]
  stat --> xml["prompt.xml exists and non-empty?"]
  xml --> vars["every {{var}} in prompt.xml declared in meta.inputs?"]
  vars --> reqv["each required input referenced in prompt.xml?"]
  reqv --> al["aliases match ^[A-Za-z_][A-Za-z0-9_-]*$?"]
  al --> col["alias not owned by another prompt?"]
  col --> ev["eval.yml schema (cases or triggering, names unique, inputs/inputs_from present, assert or rubric)?"]
  ev --> sk["_check_skill_export_shape: id tokens, description length/chars, triggers/not_for shape, body word counts"]
  sk --> rep["PASS / FAIL + warnings"]

Skill-Shape Subset (bin/eva:287-326)

Aggregation

A run with any problem returns exit 1; warnings are reported but do not change the exit code. alias_owners is a per-run dict mapping alias → first-seen prompt id, so a duplicate verb is reported on the second prompt that declares it (bin/eva:380-383).

Key Terms

• REQUIRED_META — the seven keys at bin/eva:49.
• alias_owners — in-memory map that detects cross-prompt verb collisions.
• skill-shape problem vs. warning — problems make doctor (and export-skill without --force) fail; warnings only annotate.

Q&A

Q: What two checks tie {{vars}} in prompt.xml to meta.inputs? A: (1) Every {{var}} parsed by VAR_RE must appear in meta.inputs[].name; (2) every inputs[] entry with required: true must be referenced in prompt.xml (bin/eva:362-369).

Q: What body-size threshold is a warning vs. a hard problem for skill export? A: ≥ 3500 words on prompt.xml is a yellow warning; ≥ 5000 is a hard problem that blocks export-skill without --force (bin/eva:282-283, 317-321).

Q: How are duplicate aliases across prompts reported? A: The second prompt to declare an already-claimed verb gets alias '<x>' collides with <other-id> as a problem; the alias is otherwise still owned by the first claimant for the rest of the run (bin/eva:380-383).

Examples

$ eva doctor
PASS refactor-with-constraints
FAIL crumb-flashcard-extractor
  - prompt.xml references undeclared var: {{ unknown }}
  - alias 'refactor' collides with refactor-with-constraints
2 problem(s) across 1 prompt(s)