Eval-Driven Development

Here's a pattern that keeps showing up. You ask an AI agent to build something complex — an algorithm, a data pipeline, a parser, a workflow. The agent delivers code that looks right. It runs without errors. The output seems reasonable. You glance at it, nod, and move on.

Two weeks later, you discover the output was subtly wrong the entire time.

The instinct at this point is to use another AI to check the first one. And that's not wrong — running multiple agents to review each other's work catches real mistakes. I do it myself. But it's not sufficient. AI models share correlated failure modes. The same architectural blind spots that caused the first model to miss something can cause the second model to miss it too. Multi-agent review is a useful layer. It's not a ground truth.

Deterministic checks can provide repeatable evidence for a stated property. They are not ground truth merely because they are code: a test can encode a wrong expectation, omit a case or share the implementation’s mistake. AI can help write tests, but their requirements and expected results need independent scrutiny.

The Concept

Eval-driven development makes acceptance criteria explicit and builds checks suited to the output. Use deterministic code where a property can be tested reliably, alongside the review needed for requirements that are not fully captured by those checks.

A passing check establishes that the tested condition held for the tested input and environment. Formal proof requires a different argument and its own assumptions; ordinary tests do not become proofs by being repeatable.

This takes many forms:

• Assertions — the output must satisfy specific mathematical properties, boundary conditions, or invariants
• Schema validators — the structure must match an exact specification, every field present, every type correct
• Diff checks — the output must produce identical results to a known-good reference implementation on the same inputs
• Property-based tests — the output must hold true across thousands of randomly generated inputs, not just the three examples you thought of
• Integration tests — the output must actually work when plugged into the real system, not just in isolation
• Snapshot tests — the output must match a previously approved baseline, and any deviation gets flagged for human review

These are established testing techniques. They remain useful regardless of who wrote the implementation. AI assistance makes it especially important to check that the implementation and tests have not repeated the same unsupported assumption.

Why This Works

Models can produce fluent incorrect output and can also express uncertainty. Confidence in the explanation is insufficient evidence either way. Inspect the artifact and the behavior the task requires.

Executable checks answer narrower, explicit questions. Their value depends on their specification, coverage and implementation.

• An assertion either passes or fails. There's no "probably passes."
• A schema validator doesn't care how confident the AI was. The field is there or it isn't.
• A diff check doesn't negotiate. The output matches or it doesn't.
• A property-based test doesn't get tired after 50 cases. It runs 10,000 and reports every failure.

A test program does not generate a new natural-language assertion each time it runs, but it can still be wrong. Validate the oracle, challenge the boundaries and use independent expected results where available.

The New Development Loop

Traditional development: write code → write tests → run tests → fix code.

Eval-driven development flips the order: write the eval first. Define what correct looks like in code before the agent writes a single line. Then let the agent generate. Then run the eval. If it fails, feed the failure back to the agent and iterate.

The loop looks like this:

1. Define the contract: requirements, invariants, representative inputs and known limitations.
2. Write and review checks: AI may assist in authoring; execution of a deterministic check is distinct from an AI judge.
3. Implement under the agreed requirements and permissions.
4. Run the checks and preserve actual results and environment details.
5. Investigate failures, including the possibility that a check is wrong; review changes to either code or tests.

If this sounds like Test-Driven Development, that's because it is. Kent Beck pioneered TDD decades ago, and the core insight — define correctness before writing implementation — is more relevant now than when he first proposed it. The implementation author changed from human to AI. The need for upfront correctness criteria didn't.

When AI-as-Judge Is Fine (and When It's Not)

A fresh agent review can add useful perspectives. It remains advisory evidence that requires reconciliation; neither a separate model nor a deterministic test is automatically an infallible oracle.

For subjective qualities, combine human judgment and structured review with any objective constraints that can be checked. A model judge is one option, not necessarily the best or only evaluator.

For verifiable properties, executable checks are often an efficient part of the evidence:

• Code generation — does it compile? Do the tests pass? Does it handle the edge cases?
• Data transformations — does the output schema match? Are the row counts right? Do the aggregations sum correctly?
• API integrations — does the request match the spec? Does the response parse correctly?
• Algorithms — does it produce the correct output for known inputs? Does it satisfy the time complexity requirement?
• Configuration — is the YAML/JSON valid? Do all references resolve? Are there no circular dependencies?

Automate a check when it is useful, reliable and proportionate to the risk. Retain review and other evidence for requirements the check does not cover.

Closing the Verification Gap

I wrote previously about the verification gap — the growing distance between what AI agents can produce and what humans can verify. Agents generate code faster than anyone can review it. The output volume exceeds human attention bandwidth. The gap widens with every model improvement.

Reusable checks can reduce repeated verification work. They do not catch every failure or remove the need to inspect consequential changes. Test execution time and the amount of review required depend on the system.

A layered workflow combines clear requirements, representative tests, code review and operational evidence. The result is bounded confidence that can be strengthened when new failure cases appear.

Making It Practical

If you're working with AI agents daily, here's how to start:

• Prioritize checks for consequential and plausible failures.
• Make acceptance criteria available without allowing tests to be weakened merely to pass.
• Review AI-generated tests against requirements and independently known examples.
• Run relevant checks in the delivery workflow and retain failure output.
• Treat failures as evidence to investigate, not as a reason to let an agent change expectations without review.

The Boring Part Is the Important Part

Writing and maintaining good checks is engineering work. Give the test suite the same attention to clarity and correctness as the implementation.

They're also the part that keeps everything from quietly falling apart.

As generation becomes faster, verification needs explicit ownership. Expand checks when a new risk or observed failure justifies them, and keep their limitations visible.

Let AI write the code. Write the code that checks the code. That's eval-driven development. That's the discipline that makes everything else sustainable.