Why agentic eval loops fail

The first eval loop you ship will tell you your agent is great. The second one — built by someone who wasn't on the original team — will tell you it's mediocre. The third one, with hand-graded golden examples, will tell you what's actually happening.

This is a field guide to building the third kind on the first try.

The problem with confident scores

When you grade an agent with the same model that powers it, you don't measure quality — you measure preference. The model has opinions about its own outputs, and those opinions get baked into the score.

The pattern is depressingly common: weeks of prompt tuning push the reported number up. Swap the grader to a different model family and the number falls back. The gap between the two scores is the bias you've been baking in — and you only see it when you measure it.

What a good trace looks like

Before we talk about scoring, look at what we're scoring. An agent run is a sequence of decisions, tool calls, and observations — not a single output. Your eval needs to grade the whole trace, not just the final answer.

A bad eval scores only the final string. A good eval scores: did the agent search before answering? Did it verify the source? Did it return a date? An eval is the contract you write between you and your future agent.

Three approaches to grading

There are three families of grader. Each has a place; mixing them is usually best.

const score = await grader.generate({
  model: 'gpt-4o',
  messages: [{
    role: 'user',
    content: `Rubric: ${rubric}\nAgent output: ${output}\nReturn 0-100.`,
  }],
});

const valid = JSON_SCHEMA.safeParse(output).success
  && output.includes(expectedDate)
  && output.length < 500;

const golden = readJSON('evals/golden-set.json');
const score = compareAgainst(output, golden[caseId]);

How they compare

The right answer is rarely "pick one." It's almost always: deterministic checks first, LLM-as-judge for what's left, anchored to a hand-graded golden set you trust.

Naive vs production loops

The shape of the loop matters as much as the grader.

const result = await agent.run(input);
const score = await grader.score(result);
return score;

const result = await withRetry(
  () => agent.run(input),
  { timeout: 30_000, maxAttempts: 3 }
);
const score = await grader.score(result);
const goldenScore = compareAgainst(result, golden[caseId]);
log.info({ caseId, score, goldenScore, trace: result.trace });
return { score, goldenScore };

Implementing it

1. 1

   Build the golden set first

   Pick 5–10 inputs whose ideal outputs you can write by hand. Store them in evals/golden/. Re-grade them yourself every quarter — your standards drift.

2. 2

   Layer deterministic checks

   Schema validation, length bounds, presence of required fields, regex for forbidden content. These run in milliseconds and catch the embarrassing failures.

3. 3

   Add LLM-as-judge for what's left

   Pick a different model family than your agent. Score against an explicit rubric, not "is this good?" — the rubric reveals disagreement faster than the score does.

4. 4

   Run on every PR

   A 50-case eval that takes 90 seconds is a CI gate. A 5,000-case eval that takes an hour is a once-a-day batch. Both are useful for different things.

Caveats and edge cases

Visualizing the full loop