Agent Security and Unlearning Evaluation
Series · 1 posts
-
K-Bench: Why Agentic Unlearning Cannot Be Certified from the Final Answer Alone
AdvancedA deep read of Yu et al.'s K-Bench (arXiv:2609.12808 v1): an end-to-end agent deployment benchmark that evaluates six observable channels across four memory substrates, using OR-of-channels leakage, collapse-aware K-Scores, and pre-registered statistical tests to separate forgetting from channel migration and agent collapse.
Understand it in 90 seconds
- Problem
- TOFU- and MUSE-like unlearning benchmarks treat a model mainly as a question-answering interface and read one direct answer. That is useful for a secret that lives in the weights and leaks only through that surface, but it does not cover context, RAG, database lookup, CoT scratchpads, tool calls, tool returns, or a later summary after deployment.
- Core insight
- control both where the secret lives and which surfaces the deployed agent exposes. K-Bench puts the same kind of PII into one substrate per cell, exposes a ReAct trace as six channels, and takes a logical OR over those channels for each query.
- Strongest evidence
- for the Llama-3.1-8B no-intervention baseline, aggregate OR(all) is 0.223 on C, 0.602 on R-text, and 0.855 on R-struct. TOFU/MUSE weight probes see no target memorization on those lanes. This is a coverage gap, not a case that weight unlearning merely needs to be stronger.
- Main boundary
- the result covers six observable text channels, four pure substrates, English PII, a fixed ReAct harness, and selected model/injection configurations. It is not a proof that production memories, logs, external databases, or model copies have been deleted.
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact