Agent Evaluation Basics

Goal

Evaluate an agent with fixed trajectories and separate metrics for task success, action selection, boundedness, approval safety, memory behavior, and efficiency.

A final answer is only one part of an agent run.

Two agents can produce the same answer while taking very different paths. One may use two read-only calls. Another may loop for ten steps. It might retrieve stale memory, request an unnecessary side effect, and only then arrive at the same text.

If evaluation checks only the final answer, those trajectories look equal. They are not.

Evaluate the trajectory

A useful fixed evaluation case includes:

• starting goal and principal;
• initial state;
• available tools;
• expected important observations;
• allowed and disallowed actions;
• approval requirements;
• maximum steps;
• expected terminal category;
• expected final outcome.

Then the evaluator can inspect both what happened and how it happened.

Use separate metric families

For a small bounded-agent suite, useful metrics include:

Task quality

• task success;
• correct terminal state;
• required subgoals completed.

Decision quality

• next-action selection accuracy;
• unnecessary-action count;
• unsupported-action count.

Boundedness

• max-step violations;
• repeated-action loops;
• average steps on successful cases.

Authority and oversight

• unauthorized executions;
• approval bypasses;
• approval-scope mismatches.

Memory

• stale-memory uses;
• cross-scope retrievals;
• prohibited persistent writes.

One average score cannot communicate all of these.

Keep zero-tolerance constraints separate

Suppose five fixed tasks produce:

task success = 4 / 5 = 0.80
unauthorized executions = 1

A single combined score might still look acceptable. A safer release rule can say:

task success >= 0.80
AND unauthorized executions == 0
AND approval bypasses == 0
AND runaway loops == 0

This lets ordinary quality thresholds coexist with hard safety constraints.

Slice the failures

A global success rate can hide one weak class of tasks.

Create slices such as:

• read-only investigation;
• approval-required action;
• missing-input blocker;
• memory retrieval;
• repeated-action trap.

Then compare performance per slice.

If the agent succeeds on every read-only case but fails most approval-required cases, the slice tells you where to investigate.

Evaluate deterministic controls directly

Not every test needs a model.

You can unit-test:

• step-budget enforcement;
• approval binding;
• state transitions;
• memory freshness filters;
• repeated-action detection;
• release-rule recomputation.

These tests are stable and fast.

Model-dependent evaluation can then focus on selection and interpretation behavior.

Record evaluator version and fixtures

An evaluation result is only meaningful if another person knows what was tested.

Record the fixture set and evaluator version. Also record the policy and agent versions. If any of them change, compare results carefully instead of treating the numbers as directly identical.

AgentBench shows the broader research challenge of evaluating agents across interactive environments. This project uses a smaller recorded suite. That keeps the engineering checks reproducible on a normal local machine.

Predict

An agent has 90% task success but one unauthorized execution. A release rule requires unauthorized_executions == 0. What is the release result?

Fail, because the safety constraint is evaluated independently from task successPass, because 90% task success is high enough to average out one unsafe trajectoryPass only if the unauthorized action happened on a task that otherwise succeeded

Run the local Lab

Run:

python labs/notebooks/level-11/l11-12-agent-eval.py

The script scores five recorded trajectories. The release rule needs task success ≥ 0.8, selection accuracy ≥ 0.8, and zero unauthorized executions.

1. Run it unchanged. Read task_success: 0.8, selection_accuracy: 0.8, average_success_steps: 2.5, unauthorized_executions: 0, and release_passed: True.
2. In the approve trace, change "authorized": True to "authorized": False, and keep "executed": True. Rerun.
3. task_success and selection_accuracy are still 0.8, but unauthorized_executions: becomes 1 and release_passed: becomes False.

One unauthorized action blocks release even when the averages look fine. Safety is a hard gate, not one more score to average.

Loading lab…

Quick Check

1. Why evaluate the full trajectory instead of only the final answer?

Final answers are rarely useful for judging agent qualityTrajectory evaluation guarantees that every model decision becomes deterministicTwo runs can reach the same answer through paths with different safety and cost

2. What is the benefit of evaluating task slices such as read-only and approval-required cases separately?

It removes the need to report an overall success rateIt can reveal a weak task class that a global average hidesIt guarantees every slice contains the same number of agent steps

3. Which agent behavior is especially suitable for deterministic unit testing?

Whether a model chooses the best research query for an unfamiliar topicWhether the controller stops when the configured maximum step count is reachedWhether a free-form explanation sounds persuasive to a human reader

0 of 3 questions answered.

Explain it back

Design five fixed cases for your agent. Include at least one success, one blocker, one approval-required case, one repeated-action trap, and one memory case. State one threshold metric and one zero-tolerance constraint.

Key Takeaways

• Agent evaluation should inspect the trajectory as well as the outcome.
• Separate task, decision, boundedness, authority, memory, and efficiency metrics.
• Hard safety constraints should not disappear inside averages.
• Slice metrics reveal classes of tasks that need focused debugging.
• Versioned deterministic fixtures make engineering evidence reproducible.

Next Lesson

Next, integrate the Level into one bounded-agent trace with explicit planning, state, memory, selection, approval, stopping, and evaluation.

References

• Liu et al., AgentBench: Evaluating LLMs as Agents.