Agent Trace Observability

Series · 1 posts

← Back to Paper Reading

  • Parsing the Stream: Long-Horizon Agents Need Auditable Live State, Not Just Memory

    Advanced Agent runtime, safety, and evaluation
    Agent Trace Observability: Part 1 , Note: Sep 7, 2026 , Paper: 2026 , AI Engineering

    A critical reading of Pakhomov and Nijkamp's Parsing the Stream (arXiv:2609.01466): an append-only trace is folded into typed RunState and compiled into observer and worker views. The paper reports gains on specific accumulation tasks and monitoring costs, but does not show that fixed aggregates replace every form of trace memory.

    Understand it in 90 seconds
    Problem
    A long-horizon Agent trace outgrows both of its consumers. A human observer needs to know what is happening, what has settled, and what is still missing; the worker Agent must put the same growing trace back into a bounded context window. A tail-only view loses early facts, while replaying the full history on every turn makes tokens, cost, and failure risk grow together.
    Core insight
    Instead of building two unrelated summarizers for the worker and observer, write the trace as an append-only typed ledger, fold it into a RunState with source and coverage information, and compile consumer-specific views from that state.
    Strongest evidence
    In COMPREHEND, using 12 real transcripts and 70 monitoring questions per condition, the compiled view reaches 0.871 accuracy for Sonnet 5 and 0.850 for Haiku 4.5; raw tails reach only 0.479 and 0.476. In CONTINUE's clean 120-link protocol, the curated fold scores 30/30, the scratchpad scores 30/30, and full context scores 8/30 (Tables 1–2, Figures 2–3).
    Main boundary
    The evidence is conditional on schema coverage and task shape. The authors show that the fold loses its advantage on an alternating-sign chain, and they acknowledge benchmark–system co-evolution, a single vendor, a fixed schema, single-session traces, and untested prompt injection, secret redaction, and multi-agent ledgers.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact