Agent Safety & Governance
33 reading notes
Examine control boundaries for agents in real execution environments: risk injection, auditing, permissions, and rollback.
Reader question
Beyond task completion, how do we show that an agent’s process, memory, and side effects remain controlled?
Reading library
Read this topic in depth
Every paper reading currently filed under this research topic.
-
StepGuard Paper Reading: Guarding Agent Actions Step by Step
Intermediate Agent runtime, safety, and evaluationA close reading of how StepGuard trains a 4B runtime guard from prefix-aligned synthetic trajectories and uses Balance-GRPO to adjust safety and task utility. We examine the 77.3% relative ASR reduction and 2.8-point utility decrease on AgentDojo and AgentDyn, along with synthetic supervision, failure modes, latency, and the absence of a formal guarantee.
Understand it in 90 seconds
-
PACT: Can Enterprise AI Assistants Be Trusted Under Pressure? A Reading on Rule Following
Advanced Agent runtime, safety, and evaluationPACT tests whether enterprise AI assistants follow rules through ordinary pressure, multi-turn pushback, and cases where a rule does not apply. This reading explains its six evaluation axes, reported results, and limits as evidence.
Understand it in 90 seconds
-
An Empirical Study of Harness Design for Coding Agents: Planning, Tools, and Context Management
Advanced Agent runtime, safety, and evaluationWhen does a coding agent benefit from planning, a structured tool interface, or context management, and when do these components only add cost? Read the conditional effects across four models, two benchmarks, and 176 matched settings.
Understand it in 90 seconds
-
Refuse, Decompose, Refresh: Claim Boundaries in Closed-Loop AI Evaluation
Advanced Agent runtime, safety, and evaluationA close reading of an evaluation protocol that abstains when comparisons lack observable support, reports separate decisions, and treats distribution shift as reference invalidation. The article explains its simulator thresholds, denominators, held-out evidence, and deployment boundary.
Understand it in 90 seconds
-
Finding the Right Fit: How Models, Harnesses, and Tasks Change Agent Rankings
Advanced Agent runtime, safety, and evaluationA comparison of 66 model–harness configurations across three task collections, with score reversals, cost trade-offs, and failure-feedback traces. What these observations can—and cannot—tell us about deployment choices.
Understand it in 90 seconds
-
MAGS Paper Reading: Formal Verification and the Boundary of Safety Specs
Advanced Agent runtime, safety, and evaluationA deep reading of how MAGS translates agent-generated programs into Dafny, proves them against frozen specifications, and compiles them back; the reading separates 220 formal certificates from independent safety and functional results.
Understand it in 90 seconds
-
Closed-World Resolution: Why Tool Identity Must Precede Authorization
Intermediate Agent runtime, safety, and evaluationA close reading of Closed-World Resolution Against Tool Hallucination in LLM Agents: how fabricated tools, invalid arguments, and MCP namespace collisions can bypass ordinary gates, and where the evidence stops.
Understand it in 90 seconds
-
Loopjacking: How Human Approval Loses Its Binding to an Action
Advanced Agent runtime, safety, and evaluationA deep reading of Loopjacking: how incomplete approval representations and post-approval state substitution detach a human decision from the executed side effect, and what version-pinned product traces do and do not establish.
Understand it in 90 seconds
-
Agents Are Systems, Not Models: Why Agent Evaluation Must Go Beyond Model Scores
Advanced Agent runtime, safety, and evaluationThis AI-for-science study varies task information, reasoning, verification, runtime budget, and backbone. In its main experiments, repeat-run noise accounts for about 54% of score variance among genuine attempts. The result is important—but covers only four specialist-model tasks, not every kind of agent.
Understand it in 90 seconds
-
When Upstream Messages Override Correct Answers: How Wrong Answers Displace Downstream Judgment
Intermediate Agent runtime, safety, and evaluationA controlled handoff study across five benchmarks and five receivers tests when an incorrect upstream answer displaces a downstream answer that was independently correct—and what the experiment does not establish.
Understand it in 90 seconds
-
Raven Paper Reading: How a Harness of Harnesses Plans Multi-Agent Work
Advanced Agent runtime, safety, and evaluationRaven composes model–harness pairs as specialists and asks a Host Agent to plan a dependency DAG. This reading unpacks harness composition, the MAOB benchmark, and what its +10.4/+10.5 percentage-point result actually measures: graph planning match, not successful worker execution.
Understand it in 90 seconds
-
Who Signs Off? Let Specifications, Not Agents, Decide When a Task Is Done
Intermediate Agent runtime, safety, and evaluationA deep reading of how SpecHarness compiles agent-visible instructions into source-linked obligations and commits state only from qualified evidence, with an examination of SkillsBench and GuideBench results, runtime costs, and limits on attributing gains to sign-off alone.
Understand it in 90 seconds
-
Can Authorization Hold After an Agent Requests Payment? A Deep Reading of APort Vault
Advanced Agent runtime, safety, and evaluationA deep reading of APort Vault that separates payment requests, successful payments, policy decisions, recipient membership, and unpermitted transfers while examining what a pre-action authorization layer does and does not establish.
Understand it in 90 seconds
-
How Can a Vulnerability Report Become Executable Evidence? A Deep Read of MobileCybench
Advanced Agent runtime, safety, and evaluationA deep read of MobileCybench: 13 Android apps and 495 executable security probes used to evaluate five coding agents, with careful distinctions among probe triggers, vulnerability attribution, maintainer confirmation, and the limits of the evidence.
Understand it in 90 seconds
-
Prime Agent: How a Self-Improving RLM Harness Keeps Long-Run State Outside the Model
Advanced Agent runtime, safety, and evaluationA deep read of Prime Agent (arXiv 2608.23552 v1): its L0–L3 state hierarchy, persistent REPL, recursive subagents, and Continual Harness—alongside author-reported evaluations, a specification exploit retained through refinement, and the unsandboxed execution boundary.
Understand it in 90 seconds
-
SIFT: Turning Expensive Self-Improvement Evaluation into Cheap Ranking and Deferred Verification
Advanced Agent runtime, safety, and evaluationA deep reading of Self Improvement via Fast Tree-search (arXiv:2609.19526): pairwise LLM judging, regularized Bradley–Terry ranking, and asynchronous tree search decide which agent patches deserve expensive benchmark verification.
Understand it in 90 seconds
-
Learning When to Trust: Measuring and Training Selective Trust with Selective Context Preference Optimization
Advanced Retrieval, memory, and production RAGA deep read of Learning When to Trust via Selective Context Preference Optimization: MIST holds each reasoning item fixed while varying clean, misleading, correct-context, and irrelevant-context signals; SC2W isolates clean-correct flips, and SCOPE reduces them with balanced DPO preference quartets while retaining the paper's text-only, contamination, and deployment-prevalence boundaries.
Understand it in 90 seconds
-
Bounded Agents: When Agent Security Is an Authorization-Architecture Problem
Advanced Agent runtime, safety, and evaluationA deep read of Bounded Agents (arXiv:2608.15888 v1): the Agentic Principal Chain, six conjunctive authorization conditions, and composition closure for constraining delegated authority and cross-step side effects—together with the complete-restriction, serialized-admission, enforcement, and utility boundaries.
Understand it in 90 seconds
-
After the Party: Governing What a Viral Agent-Skill Ecosystem Left Behind
Advanced Agent runtime, safety, and evaluationA deep reading of After the Party’s OpenClaw and ClawHub ecosystem study: 91 days of explosive growth, download concentration, the reviewability gap, privilege evidence, scanner disagreement, and the governance method that can transfer without pretending the rates do.
Understand it in 90 seconds
-
RAGSieve: Detecting RAG Knowledge-Poisoning Promotion with Self-Referenced Local Contrast
Advanced Retrieval, memory, and production RAGA deep read of RAGSieve: query-local and corpus-local references expose suspicious retrieval promotion without a trusted clean corpus, while the paper's boundary remains essential—promotion detection is not truth verification.
Understand it in 90 seconds
-
ACE: Let a Canvas Agent Understand Structure Before It Corrects Itself
Advanced Agent runtime, safety, and evaluationA deep reading of ACE (arXiv:2608.24103 v1): hierarchical scene graphs, CARE routing, and an instruction-following judge turn multi-slide editing into a scoped, diffable, rollback-aware loop, with explicit limits around benchmarks, human raters, mock mode, and live reproduction.
Understand it in 90 seconds
-
Tool Call Succeeds, Workflow Fails: External-Effect Anomalies at the Agent–Tool Boundary
Advanced Agent runtime, safety, and evaluationA deep reading of the effect-history model behind Agent–Tool Boundary: why a successful tool response cannot guarantee a coherent external world state, and what MCP annotations and transactional contracts still leave unspecified.
Understand it in 90 seconds
-
Corrupt Plans, Clean Traces: How Plan Injection Evades Chain-of-Thought Monitoring
Advanced Agent runtime, safety, and evaluationA deep reading of Plan Injection: when an adversarial plan enters context and an actor rewrites it into plausible Chain-of-Thought, why the monitor’s clean trace can disconnect from behavioral causality, and where APPS, Bio-Math, and budget-sweep evidence stops.
Understand it in 90 seconds
-
K-Bench: Why Agentic Unlearning Cannot Be Certified from the Final Answer Alone
Advanced Agent runtime, safety, and evaluationA deep read of Yu et al.'s K-Bench (arXiv:2609.12808 v1): an end-to-end agent deployment benchmark that evaluates six observable channels across four memory substrates, using OR-of-channels leakage, collapse-aware K-Scores, and pre-registered statistical tests to separate forgetting from channel migration and agent collapse.
Understand it in 90 seconds
-
DRACO: Sending Long-Horizon Agent Credit Back to the Steps
Advanced Agent runtime, safety, and evaluationA source-grounded reading of DRACO (arXiv:2609.04094): dynamic per-trajectory rubrics create an outcome-blind reward, then a closed-form rule redistributes GRPO advantage to the steps cited by the judge.
Understand it in 90 seconds
-
Indirect Prompt Injection: Web Pages and Tool Returns Become Instruction Channels, but 2023 Cases Do Not Represent Later Guard Products
Intermediate Agent runtime, safety, and evaluationA source-grounded reading of Greshake et al., arXiv:2302.12173 v2: when LLM-integrated apps retrieve web pages, email, or tool output, untrusted data enters the prompt as if it were instructions. The authors demonstrate indirect prompt injection on Bing Chat, GitHub Copilot, and synthetic GPT-4 apps and give a computer-security threat taxonomy. This is 2023 control-plane evidence, not a Llama-Guard, Constitutional AI, OWASP Top-10, or jailbreak-benchmark product SLA.
Understand it in 90 seconds
-
ADIAS: Turning Agent Self-Improvement into Traceable Issue Repair
Advanced Agent runtime, safety, and evaluationA deep reading of ADIAS: persistent issue state organizes failure evidence across optimization rounds so a full-code agent designer can remember what was tried, what regressed, and when a repair is actually confirmed.
Understand it in 90 seconds
-
Agentic Configuration Management: Treating Agent Systems as Governed Configuration, Not Just One Execution
Advanced Agent runtime, safety, and evaluationA deep reading of how ACM uses a framework-independent Configuration Graph, immutable revisions, dependency-aware impact propagation, and runtime provenance to govern heterogeneous agent configurations across LangGraph, CrewAI, and the OpenAI Agents SDK.
Understand it in 90 seconds
-
A²E: A Traceable, Re-Evaluable Engine for Agent Auditing
Intermediate Agent runtime, safety, and evaluationA deep reading of A²E: ATP aligns benchmarks with agent harnesses, span-based traces preserve execution causality, and lifecycle-aligned metrics analyze correctness, tools, cost, and safety.
Understand it in 90 seconds
-
Argus Deep Read: Long-Running Agents Need a Runtime, Not a Longer Prompt
Advanced Agent runtime, safety, and evaluationA critical reading of Argus's Manager–Planner–Engineer–Reviewer runtime, durable state, verification-gated evolution, and rollback, separating benchmark results from author-operated case studies and unproven self-learning claims.
Understand it in 90 seconds
-
AgentS4D Deep Read: The Task Finished—Is the Runtime Safe?
Advanced Agent runtime, safety, and evaluationA critical reading of how AgentS4D places workspace-agent risk entry, induction strategy, target harm, and lifecycle evidence in one sandbox benchmark, and why completion rate cannot stand in for safety.
Understand it in 90 seconds
-
Real-Time Detection and Repair of LLM Agent Failures: A Deep Read of AgentTrajectorySentinel
Advanced Agent runtime, safety, and evaluationA critical reading of AgentTrajectorySentinel's low-cost healthy-only temporal monitor, deterministic verification, and rollback-and-retry loop, separating measured detection and repair gains from calibration dependence, content blind spots, and reproducibility limits.
Understand it in 90 seconds
-
OSReward Deep Read: Why Agent Success Cannot Be Judged by Another Model Alone
Advanced Agent runtime, safety, and evaluationA complete reading of OSReward's data construction, 27 VLM judges, Hard and Multi subsets, error and cost analyses, OS-Shepherd-100K training, and a deployable hybrid verification architecture.
Understand it in 90 seconds
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact