Agent Evaluation and Reliability
Series · 1 posts
-
Agents Are Systems, Not Models: Why Agent Evaluation Must Go Beyond Model Scores
Advanced Agent runtime, safety, and evaluationThis AI-for-science study varies task information, reasoning, verification, runtime budget, and backbone. In its main experiments, repeat-run noise accounts for about 54% of score variance among genuine attempts. The result is important—but covers only four specialist-model tasks, not every kind of agent.
Understand it in 90 seconds
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact