Learning path
Agent Evals and Tracing
Test sets for agents, LLM-as-judge and tracing, with animated films and runnable Python. Part of the Agentic AI Engineer premium track.
-
1
Test Sets for Agents
Evaluate agents like a pro: tasks with checkable end states, trajectory checks on which tools were used, efficiency in steps, running each case several times, pass@k versus pass^k,…
-
2
LLM-as-Judge
Use a model to grade agent work you can't check with code: rubric judges with critical items, pairwise comparisons, catching position bias by swapping order, and calibrating the…
-
3
Tracing and Observability
See inside every agent run: traces and spans, recording model calls, tool calls, tokens, latency and errors, building a tiny tracer, finding the slowest and costliest steps, and…