What is Evals?

Automated tests that measure whether a change made your agent better, not just different — a set of inputs with known-good expectations, run on every change.

How it works

Without evals you're guessing whether a prompt or model change helped. Evals turn 'it feels smarter' into a number you can track. They're the difference between engineering an agent and vibing one.

Example

In practiceFifty questions with reference answers; after a prompt change you re-run and compare accuracy before shipping.

When should I add evals?

As soon as the agent does real work. Retro-fitting them after drift is far harder.

Go deeper: Deploy Your First AI Agent

The complete, step-by-step guide — every command, config, and the reasoning behind it.

Get the guide