ADIAS: Turning Agent Self-Improvement into Traceable Issue Repair

The paper in 90 seconds

• Problem: Automated agent design is usually candidate-centric. Each round re-reads candidate code, scores, and trajectories, but it does not explicitly remember whether the same failure has already been repaired, which intervention worked, or which change caused a regression.
• Core insight: Make the issue being repaired, rather than the candidate agent, the persistent control state. Each issue has a stable identity, priority, supporting evidence, lifecycle status, and intervention-outcome history.
• Strongest evidence: ADIAS is evaluated on Tau-Bench, ALFWorld, TextCraft, WebShop, and ScienceWorld against five baselines. Table 1 reports an average score of 78.4 versus 62.6 for DGM-H, with shared wrappers, splits, action interfaces, scoring scripts, a ten-iteration optimization budget, and 15 training episodes per iteration (paper Section 4 and Table 1).
• Main boundary: The paper holds trajectory diagnosis and issue association fixed rather than measuring their accuracy separately, and evaluates only text-based interactive benchmarks. The GitHub repository still says Coming Soon as of 2026-08-12, so this article does not treat a paper-level code claim as a currently reproducible artifact.

The bounded verdict is: persistent repair state is a useful control-plane idea for agent optimization, but it pushes diagnosis and cost risk outside the evaluated method and does not yet establish safe automatic modification of production agents.

What to know first

An agent harness is the executable system around a backbone model: it decides how to process observations, plan, call tools, retain memory, recover from errors, and terminate a task. Automated agent design treats that harness as an editable artifact and repeats a loop of modification, evaluation, diagnosis, and revision.

A candidate-centric history $H_t$ may contain every generated agent, trajectory, score, and diagnostic report, yet remain indexed by candidates. ADIAS adds an issue state $E_t$ on top of that archive. It does not discard candidate history; it adds an issue-indexed repair ledger. This makes the paper complementary to Bloss0m’s OSReward agent-evaluation reading and Agent Trajectory Sentinel reading: those works focus on measuring or detecting behavior, while ADIAS asks how repair progress can persist across optimization rounds.

Why candidate-only history is insufficient

Suppose generation 2 fixes tool selection but breaks observation parsing, and generation 6 fixes parsing without preserving the useful intervention from generation 2. If the record is organized only by candidate, the next optimizer must infer the local changes worth preserving from whole-agent scores and long trajectories. Three problems follow:

1. Inefficient repair targeting: unresolved issues, the code surface to change, and attempted directions are scattered across candidate records.
2. Fragmented progress: one candidate can affect multiple issues, making a single intervention hard to attribute.
3. Regressive propagation: the optimizer may continue from the latest or highest-scoring candidate and repeat an ineffective direction.

This is not simply a lack-of-history problem. The repair objective has not become a first-class state. ADIAS claims that making it explicit lets the next round choose what to repair, where to resume, and which directions should not be repeated yet.

Core intuition

The candidate-centric mental model is: propose the next candidate with the highest aggregate quality from the archive. ADIAS changes it to: propose the next candidate most likely to advance unresolved issue repair.

An issue record can be represented as:

$e_i^t=(id_i,q_i^t,s_i^t,\mathcal{B}_i^t,\mathcal{U}_i^t)$

Here $id_i$ is the stable identity across candidates, $q_i^t$ is priority, $s_i^t$ is lifecycle status, $\mathcal{B}_i^t$ is trajectory and diagnostic evidence, and $\mathcal{U}_i^t$ is the intervention-outcome history. Operationally, this turns repair progress from a prose summary into state that the next decision can query.

Figure 2, the ADIAS overview in paper Section 3.3: external task priors and the initial trajectory construct issue state, followed by a propose → modify → evaluate → diagnose → update loop. Original figure; image from the arXiv HTML figure endpoint. The ADIAS source is marked with an arXiv.org perpetual non-exclusive license and does not separately grant an explicit reuse term; attribution is preserved here, but reuse permission should be confirmed before redistribution.

Walk one example through the method

The following is a simplified explanation of the representative TextCraft repair pattern in Figure 4, not an additional experiment:

1. Input: The agent receives a crafting task that requires obtaining a specified quantity of a resource, such as one unit of quartz.
2. Intermediate representation: The trajectory shows repeated selection of get quartz, while the environment requires a quantity-aware action such as get 1 quartz. The Issue Manager associates this failure with an existing command-selection issue instead of recording it only as a low score for one candidate.
3. Decision: The issue state prioritizes the active issue and passes prior attempts—prompt restructuring, command matching, and parent generations—to issue-guided planning.
4. Output: The code optimizer creates a focused full-code patch. The next evaluation and diagnosis write the intervention outcome back to the same issue.
5. Likely failure point: If diagnosis incorrectly labels a quantity error as a planning issue, stable identity can preserve the wrong abstraction across rounds. This is one of the risks the paper does not independently measure.

Technical mechanism

1. Initialization: a prior is not a fact

ADIAS performs a one-shot search over public task information to obtain possible task requirements and failure-mode priors $P_0$. It then executes the initial agent $A_0$, collecting trajectory $\mathcal{T}_0$ and performance $M_0$. A diagnostic agent produces $D_0$, and the Issue Manager reconciles $P_0$ with $D_0$. Only hypotheses supported by observed behavior should become confirmed issues.

This distinction matters: the external prior reduces cold-start uncertainty, but it is not permission to write an internet-derived failure directly into the control state.

2. Lifecycle: temporary absence is not a fix

New or currently observed issues are active. An issue that is not observed in the current evaluation may become tentatively-fixed; after at least $\alpha_{min}=2$ consecutive evaluations without reappearance, it becomes confirmed-fixed. If a fixed issue returns, it becomes regressed. The lifecycle separates “not seen once” from “reliably absent.”

3. The round-by-round control loop

1. Prioritize severe, repeated, active, or recently regressed issues from $E_{t-1}$.
2. Jointly choose a parent agent $A_{p_t}$ and revision plan $R_t$.
3. Apply a focused full-code modification. The editable space can still include prompts, memory, planning, tool policy, control flow, verification, or recovery.
4. Execute $A_t$ and collect trajectory and metrics.
5. Diagnose $D_t$, then update issue identity, lifecycle, and intervention outcome.

The abstraction is:

$A_{t+1}=\arg\max_{A\in\operatorname{Propose}(H_t,E_t)}J_{issue}(A;E_t)$

The difference from candidate-centric optimization is not whether an archive exists. It is whether the objective explicitly represents progress on unresolved issues.

How to read the evidence

Main comparison: gains across environments

Table 1 uses DeepSeek-V4-Flash as the default backbone. All automated methods share benchmark wrappers, task splits, action interfaces, scoring scripts, and a ten-iteration budget. ADIAS has the best native task score in each benchmark: 81.3 on Tau-Bench, 94.0 on ALFWorld, 91.0 on TextCraft, 69.4 on WebShop, and 56.3 on ScienceWorld, for an unweighted average of 78.4; DGM-H scores 62.6. This supports the narrower statement that issue-centric control and higher task scores co-occur in these controlled environments. It does not show that every self-improvement system will achieve the same gain.

Cross-model robustness: not only one backbone

Table 2 keeps the Tau-Bench protocol fixed while changing the backbone to DeepSeek-V4-Flash, GLM-5.2, Hy3-Preview, and GPT-5.4. ADIAS scores 81.3, 90.6, 84.4, and 87.5 respectively, and the paper reports the best task performance for all four. This is a useful cross-model slice, but it remains a Tau-Bench result rather than a full five-benchmark cross-model study.

Ablation: three coupled roles matter

Table 3 separates the persistent issue state’s evidence, representation, and optimization-control roles:

Setting	Tau-Bench	ALFWorld	TextCraft	Average
ADIAS full	81.3	94.0	91.0	88.8
w/o external prior	75.0	82.8	61.0	72.9
w/o round-level diagnosis	78.1	63.4	76.0	72.5
archive-wide synthesis	65.6	60.4	32.0	52.7
best-candidate revision	71.9	75.4	74.0	73.8
latest-candidate continuation	62.5	41.0	60.0	54.5

The authors interpret the pattern as follows: the external prior helps with cold start, round-level diagnosis supplies behavioral evidence, and persistent issue state turns both into a control signal. Keep the causal claim bounded: each ablation replaces a configuration of the whole method, so a single score cannot establish an independent production ROI for each module.

Figure 4, the TextCraft qualitative analysis in paper Section 5.3: archive-wide, best-candidate, latest-candidate, and ADIAS parent lineages lead to different repair trajectories. Original figure; image from the arXiv HTML figure endpoint. The ADIAS figure is covered by the arXiv.org perpetual non-exclusive license; reuse permission is not separately specified in the source, so this article preserves attribution and the restriction note.

Evidence map

• Paper directly supports: Table 1 scores across five interactive benchmarks, Table 2’s four-backbone Tau-Bench comparison, Table 3 ablations, and Figure 4’s qualitative comparison of parent lineage and issue-specific repair.
• Author interpretation: issue-centric state accumulates repair progress, avoids repeatedly proposing ineffective interventions, and improves the efficiency and stability of full-code optimization.
• Not established: independent diagnosis or issue-association accuracy, confidence intervals or multi-seed uncertainty, multimodal or online adversarial evaluation, production cost evidence, and safety against unseen-task regressions.
• Bloss0m engineering judgment: the most transferable idea is not letting an agent rewrite arbitrary production code. It is making issue lifecycle, intervention provenance, and rollback gates auditable, then constraining code generation inside a sandbox and validation pipeline.

Artifacts and reproducibility

As of 2026-08-12, the paper links to the scylj1/adias repository, but its README only says Coming Soon; Code, Documentation, and Usage examples are unchecked. There is no usable license, pinned release, benchmark wrapper, or configuration in the inspected endpoint. Classify the artifact as announced / not currently usable, not released or reproducible.

If the repository becomes complete, the smallest useful reproduction would fix one benchmark, backbone, wrapper, and ten-round budget; compare full ADIAS with archive-wide synthesis and latest-candidate continuation; and preserve issue identities, lifecycle transitions, patches, and held-out scores. At present, API cost, seed variance, benchmark licensing, and end-to-end reproduction time are unknown.

Engineering decision and when not to use it

Use it when: the harness has durable trajectories, a clear validation split, and a sandbox that can associate an intervention with an outcome. An issue-centric ledger can prevent teams from repeating the same failure repair and can make partial fixes and regressions queryable.

Do not use it directly when: there is no trustworthy evaluator, no rollback, no sampling audit of diagnostic labels, or the repair can touch production credentials, side effects, or safety policy. Start with an issue ledger, human review, shadow evaluation, and an independent safety gate before expanding the repair surface.

Three things to remember

1. Technical idea: persistent issue state lets the next candidate know what to repair, where to resume, and which interventions have already failed.
2. Evidence: the matched five-benchmark comparison and Table 3 suggest that external priors, round-level diagnosis, issue representation, and optimization control work as a coupled system.
3. Boundary: state quality depends on diagnosis and association. The official code is not currently usable, and the paper does not guarantee production-safe automated repair.

Primary sources

• ADIAS full arXiv HTML (v1, 2026-08-03)
• ADIAS arXiv abstract and version record
• ADIAS official repository
• ADIAS repository README and artifact status