Twelve incidents, four from real meridian-ops runs and eight authored to be hostile, each labelled with bounds rather than answers: a severity floor, permitted categories, and whether the security floor must hold. Both brains that run in production take the same set: Claude inside the fleet, and gpt-4.1-mini behind an Azure Function. rendering…
Its unit tests found a live production bug pre-run: the security regex ended in a trailing word boundary that never matched "phishing" or "ransomware". It had survived three deployments across two codebases, masked by fixtures that matched other branches. Fixed and regression-tested in all three implementations, then redeployed to both clouds.
The first live run then caught two more: coverage drift between the deployed regexes, and the harness's own token starvation, where the model's thinking block ate an 800-token budget before the verdict. And the "unterminated string" errors were not truncation at all: the model was quoting the ticket inside its reasoning field, breaking strict parsing. The parser now repairs newlines and falls back to field extraction, because malformed prose in a commentary field must not cost the model the case.
CI runs the set nightly and on demand: an absolute floor, plus no brain below its own committed baseline minus one case of tolerance, because run-to-run judgment variance means a baseline from one good run lies. Fork pull requests run the gate logic offline, so they are never silently unverified.
Every scored case also emits a Langfuse trace from the run: the incident in, the verdict out, the security-floor decision as its own span, with the share id committed into the row. The instrumentation is evals/tracing.py, env-gated so offline runs never depend on it, and each row of the committed results.json carries its trace URL.