Run the repair loop with the prompt as the initial user turn.
Inspect the produced bundle against the prompt's expect.
Record pass/fail + attempts + duration.
The harness does NOT call into Anthropic — the caller injects
step to either point at a real LLM, a fixture-replay, or a stub.
CI uses a stub that returns canned bundles; product owners run the
real-LLM variant before each release.
Drive the benchmark. For each prompt:
The harness does NOT call into Anthropic — the caller injects
stepto either point at a real LLM, a fixture-replay, or a stub. CI uses a stub that returns canned bundles; product owners run the real-LLM variant before each release.