Agent evaluation & observability training
Know whether your AI Agent works — including after a change.
Know whether your AI Agent works — including after a change. Work on a bounded use case of your own. We agree level, tooling and programme with your team beforehand; product access and available features determine the exercises.
What you learn
- Define representative tasks, edge cases and acceptance criteria.
- Measure tool use, source use and task completion; use model-based grading critically.
- Inspect model and tool steps for errors, latency and usage.
- Include evaluations in development and define escalation for regressions.
Who it's for
- 01Agentic Engineers, QA engineers, data scientists and technical product owners
Programme
Design a test set
Define representative tasks, edge cases and acceptance criteria.
Evaluate
Measure tool use, source use and task completion; use model-based grading critically.
Tracing
Inspect model and tool steps for errors, latency and usage.
Regression & operations
Include evaluations in development and define escalation for regressions.
What you take away
What you take away
- An initial evaluation set and monitoring plan for your AI Agent.
- An overview of risks, prerequisites and next steps.
Prerequisites
- An agent prototype and basic Python or TypeScript experience.
Frequently asked questions
Is the training tailored to our team?+
Yes. We discuss your experience, environment and learning goals during a no-obligation intake, then provide a programme and written quote.
How long is this training?+
Duration depends on the level and exercises. It is agreed in the programme beforehand; there is no fixed duration for this topic.
Where does the training take place?+
In-company, at our Utrecht office or remotely, as agreed with your team.
Schedule this training
Discuss your learning goals during a no-obligation intake. Afterwards, you receive a tailored programme and written quote.