AI Frontier Research & Engineering
Tracking emerging technology, dissecting system architecture, and publishing what I actually build and evaluate.
Article archive — page 2
- ParanoiaEval: When a Coding Agent Treats a Resolved Risk Again
ParanoiaEval uses paired tasks to test whether coding agents choose risk responses from evidence; this article examines its results, limits, and practical evaluation lessons.
- SWE-CC: Passing Tests Can Still Break Repository Rules
SWE-CC turns 823 contribution policies from 12 open-source projects into executable checks, showing what test results and resolve rates leave out of coding-agent evaluations.
- When AI Agents Turn Public Data Retrieval into Website Probing
Transluce’s archived requests to U.S. and Canadian government sites show how ordinary retrieval can become high-volume or exploit-shaped traffic, and what stop controls operators can test.
- Don't Just Ask an LLM for an Answer: Build an Interface for Understanding
Extending Karpathy's thoughts on text, diagrams, interactive pages, and explainer videos into a practical method for choosing and reviewing AI-generated artifacts.
- Claude Code Advisor: A Second Model at the Decisions That Matter
A practical look at Claude Code Advisor's executor-advisor split, model-driven calls, transcript flow, cost, and evaluation limits—plus a checklist for trying it safely.
- A Second Opinion in Codex: `/review`, Subagents, and Auto-review
Codex does not put every second opinion behind one Advisor switch. Its configurable `/review`, read-only reviewer subagents, permission-boundary Auto-review, and Agents SDK patterns address different risks.
- Claude-Shaped Science: Agents Can Calculate; People Still Choose the Questions
Matthew Schwartz describes using BootLoops to bring agents into checkable quantitative research—and why human problem selection, expert review, and scientific judgment still matter.
- Robots Can Do the Task. Does Automation Pay? Anthropic’s Exposure–Cost Gap
Anthropic estimates that robots can perform 74% of US physical tasks in some settings, yet are cost-competitive for only 0.3% of work today. This analysis explains the tiers, cost assumptions, and implications for deployment decisions.
- Holo4: One Agent Across GUIs, Code, and Tools—with Different Licenses
A closer look at H Company’s cross-interface agent and long-horizon harness, its benchmark claims, public traces, and the licensing split between checkpoints.
- Kumo Tabular: Pretrain on Synthetic Tables, Learn New Tasks from Examples
NVIDIA Kumo Tabular reframes prediction as in-context learning: a model pretrained on synthetic tables predicts new rows from labeled examples. We examine its method, vendor-reported leaderboards, and enterprise validation requirements.
- NVIDIA Open Agent Safety Platform: Verify Runtime Controls Separately from External Monitoring
A technical analysis of OpenShell's inspectable agent-runtime controls and NVIDIA's BlueField-4/Sentry reference design, separating their responsibilities from the efficacy evidence still needed.
- Target Retail Product Search: Constrain Semantic Recall with Precision and Experiments
An engineering analysis of Target's lexical and vector retrieval, attribute controls, weighted interleaving, and the evidence limits behind its reported gains.
- Turning Agent Risks into Runtime Policy: ASSERT Evaluation and ACS Enforcement
How Microsoft’s run-assert-eval connects Clarity risk discovery, ASSERT behavior tests, and ACS runtime policy, while measuring unsafe behavior separately from over-refusal.
- Docker Sandbox Kit: Versioning Agent Authority Requests with Software
A close look at how Docker Sandbox Kit v3 places agent network, credential, and mixin declarations in an OCI artifact—and why a descriptor remains a request whose enforcement depends on the runtime.
- Project Swap: Agents Can Trade Without Knowing What Their People Want
Anthropic's low-stakes employee book exchange found that preference estimates constrained outcomes more than bargaining rules, making preference understanding a separate test for delegated agents.
- How AlloyDB Isolates Agent Bursts from OLTP: MicroVMs, MCP, and Dedicated Storage Segments
A close look at AlloyDB for agents’ read-only nodes, dedicated Colossus segments, and elastic compute pool, including isolation, freshness, cold starts, cost, and transaction boundaries.
- Turnstile Spin: Let an Agent Install CAPTCHA, Keep Verification at the Security Boundary
A practical look at how Cloudflare Turnstile Spin installs, repairs, or migrates a widget with human approval, while Siteverify, credential isolation, and replay testing close the server-side security loop.
- How Benchling Isolates Multi-Tenant Code Execution with AgentCore, STS, and DNS Firewall
A layered design for running AI-generated code in a separate AWS account, with per-job credentials, S3 endpoint policies, DNS allow-listing, and continuous exfiltration tests.
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact