PAPERS / REPRODUCTION / ENGINEERING NOTES
Paper reading for working engineers
Structured readings that extract architecture decisions, reproducible methods, evaluation limits, and what changes when research meets a real system.
- reading notes
- 106
- research topics
- 8
- learning paths
- 4
READING LIBRARY
Paper archive — page 2
Showing notes 25–48 of 106.
-
Agents Are Systems, Not Models: Why Agent Evaluation Must Go Beyond Model Scores
Advanced Agent runtime, safety, and evaluationThis AI-for-science study varies task information, reasoning, verification, runtime budget, and backbone. In its main experiments, repeat-run noise accounts for about 54% of score variance among genuine attempts. The result is important—but covers only four specialist-model tasks, not every kind of agent.
Understand it in 90 seconds
-
When Upstream Messages Override Correct Answers: How Wrong Answers Displace Downstream Judgment
Intermediate Agent runtime, safety, and evaluationA controlled handoff study across five benchmarks and five receivers tests when an incorrect upstream answer displaces a downstream answer that was independently correct—and what the experiment does not establish.
Understand it in 90 seconds
-
Raven Paper Reading: How a Harness of Harnesses Plans Multi-Agent Work
Advanced Agent runtime, safety, and evaluationRaven composes model–harness pairs as specialists and asks a Host Agent to plan a dependency DAG. This reading unpacks harness composition, the MAOB benchmark, and what its +10.4/+10.5 percentage-point result actually measures: graph planning match, not successful worker execution.
Understand it in 90 seconds
-
EfficientAgent Reading: KV-Cache Offloading for Concurrent Agents
Advanced Agent runtime, safety, and evaluationEfficientAgent asks when moving a KV cache from GPU memory to host memory actually pays off. The answer depends not on one request alone, but on the reuse working set created by concurrent agents between uses. This reading examines capacity prediction, write admission, SWE-bench replay, and hardware boundaries.
Understand it in 90 seconds
-
Stale-Document Poisoning: How RAG Can Recognize Evidence That No Longer Applies
Advanced Retrieval, memory, and production RAGAuthentic, once-correct documents can make RAG overturn an answer the model already got right. This reading examines 317 cross-domain knowledge reversals, the difference between dates and validity intervals, causal interventions, and a reranker whose gains depend on trustworthy metadata.
Understand it in 90 seconds
-
Completed Pairs Hide Capped Failures: Stopping Rules and Unknown Outcomes in Paired Evaluation
Advanced Agent runtime, safety, and evaluationA single ReVerPi source-reading campaign shows how a runner can suppress a companion arm after the first arm reaches its request cap. Finite-frame bounds and stratified cost accounting reveal what completed-pair summaries omit, without making a population claim about context projection.
Understand it in 90 seconds
-
Who Signs Off? Let Specifications, Not Agents, Decide When a Task Is Done
Intermediate Agent runtime, safety, and evaluationA deep reading of how SpecHarness compiles agent-visible instructions into source-linked obligations and commits state only from qualified evidence, with an examination of SkillsBench and GuideBench results, runtime costs, and limits on attributing gains to sign-off alone.
Understand it in 90 seconds
-
Can LLM Agents Tamper With Their Own Traces? Who Controls the Evidence Boundary
Intermediate Agent runtime, safety, and evaluationA close reading of arXiv v1: ten local coding-agent harness pairs, four controlled routes to trace tampering, and the distinct evidentiary roles of native traces, an external file observer, and tool-effect records.
Understand it in 90 seconds
-
Do Agent Skills Help with Version-Specific Plugin Migration? Scores and Contract Audits in a Retrospective Study
Intermediate Agent runtime, safety, and evaluationA close reading of the arXiv v1 comparison of a dsh plugin migration skill: what the static diagnostic scores show, how a contract counterexample qualifies them, and where task concentration, LLM judges, and artifact coverage set limits.
Understand it in 90 seconds
-
When Is Complex Chunking Worth It? Reading a Multi-Objective Chunking Evaluation
Intermediate Retrieval, memory, and production RAGA close reading of arXiv 2608.16586 v1: how eight chunking strategies change retrieval quality, indexing throughput, query speed, and memory—and where this benchmark stops.
Understand it in 90 seconds
-
The Source Link Is Intact, but the Rule Is Stale: How Agents Miss Updates Under a Verification Budget
Advanced Agent runtime, safety, and evaluationA deep reading of When Stale Constraints Go Unchecked: immutable provenance, append-only supersession, and stale-consistent decisions under a two-record verification budget. Audits four runs, the held-out correction, denominators, and the Zenodo artifact while bounding the synthetic evidence.
Understand it in 90 seconds
-
Can Authorization Hold After an Agent Requests Payment? A Deep Reading of APort Vault
Advanced Agent runtime, safety, and evaluationA deep reading of APort Vault that separates payment requests, successful payments, policy decisions, recipient membership, and unpermitted transfers while examining what a pre-action authorization layer does and does not establish.
Understand it in 90 seconds
-
How to Shrink an Agent Regression Benchmark Without Distorting It
Advanced Agent runtime, safety, and evaluationA deep reading of Trajectory-Aware Benchmark Subset Selection: historical outcome strata, trajectory sanitization, geometric selection, 76 configurations, temporal validation, cost, and transfer limits.
Understand it in 90 seconds
-
How Can a Vulnerability Report Become Executable Evidence? A Deep Read of MobileCybench
Advanced Agent runtime, safety, and evaluationA deep read of MobileCybench: 13 Android apps and 495 executable security probes used to evaluate five coding agents, with careful distinctions among probe triggers, vulnerability attribution, maintainer confirmation, and the limits of the evidence.
Understand it in 90 seconds
-
XYEval: Why agents follow bad advice
Advanced Agent runtime, safety, and evaluationA critical reading of Wu et al.'s XYEval (arXiv 2609.23939 v1): controlled XY mutations across five models and six benchmark suites, with trace analyses of how misleading suggestions affect task completion, communication, and tool trajectories—and the limits of generators and judges.
Understand it in 90 seconds
-
Does a Correct RAG Answer Mean the System Behaved Correctly? Reading The RAT
Advanced Retrieval, memory, and production RAGA deep read of The RAT: how a joint Bayesian model separates retrieval, abstention, and answer correctness, and what its annotation-budget results, judge calibration, partial retrieval analysis, and controlled Wikipedia scope do and do not establish.
Understand it in 90 seconds
-
Prime Agent: How a Self-Improving RLM Harness Keeps Long-Run State Outside the Model
Advanced Agent runtime, safety, and evaluationA deep read of Prime Agent (arXiv 2608.23552 v1): its L0–L3 state hierarchy, persistent REPL, recursive subagents, and Continual Harness—alongside author-reported evaluations, a specification exploit retained through refinement, and the unsandboxed execution boundary.
Understand it in 90 seconds
-
SIFT: Turning Expensive Self-Improvement Evaluation into Cheap Ranking and Deferred Verification
Advanced Agent runtime, safety, and evaluationA deep reading of Self Improvement via Fast Tree-search (arXiv:2609.19526): pairwise LLM judging, regularized Bradley–Terry ranking, and asynchronous tree search decide which agent patches deserve expensive benchmark verification.
Understand it in 90 seconds
-
BioPhys-Bridge: Making Scientific RAG Walk from Evidence to Physics, Mechanism, and Decision
Advanced Retrieval, memory, and production RAGA deep reading of BioPhys-Bridge (arXiv:2609.19180 v1): a benchmark that makes evidence IDs, values and units, physical equations, assumptions, biological mechanisms, and next decisions explicit, while separating attribution from scientific correctness.
Understand it in 90 seconds
-
RAFT: Making Troubleshooting RAG Retrieve the State That Led to a Fix
Advanced Retrieval, memory, and production RAGA deep reading of RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents (arXiv:2609.20754): closed support cases become state-transition trajectories, intermediate entries are retrieved, and the parent case is returned with its matched anchor.
Understand it in 90 seconds
-
Learning When to Trust: Measuring and Training Selective Trust with Selective Context Preference Optimization
Advanced Retrieval, memory, and production RAGA deep read of Learning When to Trust via Selective Context Preference Optimization: MIST holds each reasoning item fixed while varying clean, misleading, correct-context, and irrelevant-context signals; SC2W isolates clean-correct flips, and SCOPE reduces them with balanced DPO preference quartets while retaining the paper's text-only, contamination, and deployment-prevalence boundaries.
Understand it in 90 seconds
-
Bounded Agents: When Agent Security Is an Authorization-Architecture Problem
Advanced Agent runtime, safety, and evaluationA deep read of Bounded Agents (arXiv:2608.15888 v1): the Agentic Principal Chain, six conjunctive authorization conditions, and composition closure for constraining delegated authority and cross-step side effects—together with the complete-restriction, serialized-admission, enforcement, and utility boundaries.
Understand it in 90 seconds
-
HyPE: Moving the RAG Question–Document Gap into Indexing
Intermediate Retrieval, memory, and production RAGA deep reading of Bridging the Question-Answer Gap in Retrieval-Augmented Generation: Hypothetical Prompt Embeddings. HyPE precomputes hypothetical questions for every chunk and turns query-to-document retrieval into question-to-question matching, with strong six-dataset results but real indexing, freshness, chunking, and single-generator boundaries.
Understand it in 90 seconds
-
After the Party: Governing What a Viral Agent-Skill Ecosystem Left Behind
Advanced Agent runtime, safety, and evaluationA deep reading of After the Party’s OpenClaw and ClawHub ecosystem study: 91 days of explosive growth, download concentration, the reviewability gap, privilege evidence, scanner disagreement, and the governance method that can transfer without pretending the rates do.
Understand it in 90 seconds
RESEARCH EXCHANGE