SIFT: Turning Expensive Self-Improvement Evaluation into Cheap Ranking and Deferred Verification

This article reads Self Improvement via Fast Tree-search, arXiv v1 submitted on 2026-09-17. The authors are affiliated with MIT and Sakana AI. The reading covers the arXiv HTML/PDF Sections 1–5, Tables 1–7, Figures 1–9, Appendices A–B, and the safety discussion. The paper is marked CC BY 4.0; the figures below are local mirrors of the original paper assets, with source and location links kept in every caption.

The paper is not mainly asking whether an agent can modify its own code. It asks the more operational question: when self-improvement has produced dozens or hundreds of candidate patches, which ones deserve an expensive benchmark run first? SIFT answers by putting a cheap relative preference signal between candidate generation and full evaluation, allowing search to move ahead while verification is deferred to the most promising nodes.

The paper in 90 seconds

• Problem: recursive self-improvement repeatedly reruns downstream tasks for each candidate agent, so evaluation cost and wall-clock time quickly dominate patch generation.
• Method: a new patch is pairwise-judged against a small set of strong incumbents; all wins and losses are aggregated with a regularized Bradley–Terry model into a judge rank.
• Search policy: parent sampling combines judge rank, subset-accuracy rank, and a visit-count exploration term. The evaluation queue is also prioritized, so expansion and full evaluation can proceed asynchronously.
• Results: on Polyglot-225, SIFT reaches 31.1% with Qwen3-Coder-30B/Qwen3-480B and 35.1% with o3-mini/gpt-5.4. On TerminalBench 2.1, the judge-selected agent averages 36.7% over three full evaluations, compared with 29.2% for the starting agent.
• Boundary: the judge is a noisy ranking oracle, not a substitute for the benchmark. Without a public implementation, the reported CPU, API cost, and model-version details remain difficult for outside researchers to reproduce independently.

Core engineering takeaway: SIFT’s real contribution is reallocating resources between search and proof, not claiming that an LLM judge is more reliable than execution. It reframes self-improvement as speculative ranking followed by selective verification. As long as adoption still depends on full benchmarks, every judge rank remains a waypoint rather than a verdict.

Huahua’s engineering reminder

If an agent can rewrite its own harness, production systems should store candidate patches, judge rationale, benchmark traces, writable-file allow-lists, and the adopted commit as separate artifacts. SIFT can reduce exploration cost; it cannot by itself establish evaluation integrity.

Why prior approaches are not enough: every patch waits for a full benchmark

In self-improvement loops such as DGM, HGM, and SICA, a self-improver reads the current agent code and failure records, creates a child agent, and sends that child through a downstream benchmark. The result feeds the next round. The path is intuitive, but it has a scheduling bottleneck: the next parent often cannot be chosen until earlier candidates finish their full evaluations.

The cheap alternative is a benchmark subset, but subset scores are noisy, and one task execution can still take much longer than a pairwise judge call. SIFT keeps the signals separate and asks different questions:

Signal	What it answers	What it must not be read as
Subset accuracy	How did the candidate actually perform on a fixed small set?	A reliable full-benchmark ranking
Pairwise judge / BT rank	Which candidate looks more promising from its implementation?	Proof of downstream behavior
Full benchmark	How did the candidate perform on the complete task set?	A free guarantee on other models or tasks

This separation is the key to the paper: SIFT’s judge is a cheap search signal, not an automatic evaluator.

Core intuition: split self-improvement into ranking and verification

Figure 1 (original paper Figure 1, Section 3): the left side shows the pairwise win-loss matrix and BT strength, the center shows the agent tree, and the right shows a priority queue ranked by judge and accuracy signals. Original asset: arXiv Figure 1 · Section 3 anchor. The arXiv page marks the paper CC BY 4.0; the local mirror preserves attribution and must be reused under that license.

In plain language, SIFT follows this loop:

current agent harness
        ↓ self-improve
candidate patch
        ├─ pairwise judge → win/loss matrix → BT rank
        ├─ easy subset → early rejection or temporary accuracy
        └─ priority queue → full benchmark when resources are available

The cheap ranking signal and the actual downstream verification are deliberately separate. A candidate can generate a child while its full evaluation is still pending, but a benchmark remains necessary to decide whether the patch really improved the agent.

A worked example: follow one candidate node end to end

Assume the archive contains a root, node 3, node 4, and other harnesses. A self-improver creates node 10. Instead of immediately running all 225 Polyglot tasks, SIFT processes it as follows:

1. Generate a patch: the self-improvement model changes allowed runtime files and produces a child agent. Sandboxing and a writable-file allow-list prevent it from changing benchmark or evaluation-harness code.
2. Pass an easy gate: the candidate runs on a fixed small subset. A clearly broken patch is rejected; a candidate waiting for full evaluation temporarily inherits its parent’s accuracy for sampling.
3. Choose comparison targets: the new candidate is compared with strong archive nodes, typically the top 10, rather than every node.
4. Record preferences: the judge sees two candidate runtime implementations and returns which one is more likely to improve the agent. Each outcome becomes a W_ij win or loss.
5. Update BT ranks: sparse and potentially inconsistent pairwise outcomes are fit into latent strengths, and the search uses their ranks rather than treating the values as absolute quality scores.
6. Prioritize parents and evaluation: high judge rank, high subset accuracy, and a low visit count make a node more likely to be selected as a parent; promising nodes enter the full-evaluation priority queue.
7. Verify: the full benchmark, reruns, and cross-model transfer determine whether the patch deserves adoption.

This walkthrough exposes an easy mistake: a node can be expanded because its judge rank is high without being proven to be the best node.

The method: what Bradley–Terry is doing here

For every agent node i, the paper assumes an unobserved strength θ_i. A judge comparison between i and j is modeled as:

P(i preferred to j) = θ_i / (θ_i + θ_j)

The accumulated win-loss matrix W_ij is not used as the final rank directly. A regularized BT fit combines sparse comparisons into strengths for every node. The regularizer matters for new nodes: a node that has only been compared a few times should not look strong merely because it has a small raw win count.

SIFT then combines BT rank r_b(i), subset-accuracy rank r_a(i), and the number of times a node has been selected as a parent v_i:

P(i) ∝ exp(-α r_b(i) - β r_a(i) - η log(1 + v_i))

α and β control the influence of judge and measured subset signals; η discourages the search from repeatedly exploiting one lineage. The default is 1 for all three. With η=0, the o3-mini Polyglot result falls from 35.1% to 30.1%, suggesting that exploration is a real part of the search rather than cosmetic regularization.

The asynchronous pipeline: expansion does not wait for evaluation

Figure 2 (original paper Figure 2, Section 3): SIFT separates expansion, judging, and downstream evaluation into parallelizable work. A strong judge signal can expand a node before its full evaluation completes. Original asset: arXiv Figure 2 · Section 3 anchor. The chart is an original CC BY 4.0 paper asset mirrored locally.

In a blocking pipeline, the loop is roughly patch → benchmark → wait → next patch. SIFT’s orchestrator interleaves three workers:

• an expansion worker selects parents and produces children;
• a judge worker compares each child with frontier nodes and updates the BT rank;
• an evaluator worker consumes the priority queue for subset or full benchmark runs.

“Fully disaggregated” therefore does not mean “benchmark-free.” The gain comes from overlapping work and letting the judge guide exploration before downstream evaluation completes.

Cost model: the cheap signal is not free

With the default maximum of K=10 pairwise comparisons, the paper estimates up to ten judge calls for one candidate. Table 1 reports these average per-step costs:

Module	Model	API cost	CPU time
Self-improvement expansion	gpt-5-mini	$0.12	0.186 h
One pairwise judge call	gpt-5.4	$0.044	0.0042 h
Polyglot-50 full evaluation	o3-mini	$6.00	2.6 h

The intuition is clear: rank a candidate with a small number of judge calls, then reserve full evaluation for promising nodes. Total cost still depends on expansion count, judge model, full evaluations, and retries; one $0.044 comparison cannot be used to estimate an entire run.

Figure 3 (original paper Figure 3, Section 4): the Qwen3-Coder-30B configuration improves Polyglot-50 search performance and archive average over 30 evolution steps. The best descendant’s full Polyglot-225 result is 31.1%; not every intermediate point is a completed full evaluation. Original asset: arXiv Figure 3 · Section 4 anchor. The chart is CC BY 4.0 and its attribution is preserved.

Experiment 1: read Polyglot results together with cost

Polyglot contains 225 tasks across C++, Go, Rust, Java, JavaScript, and Python. Search first uses a four-task gate and then a fixed Polyglot-50 subset; the 225-task benchmark is held out for final validation. A fixed intermediate subset makes node comparisons more meaningful than random task sampling.

Method	Coding model	Judge	Polyglot-225
Base agent	Qwen3-Coder-30B	—	20.0%
SICA	Qwen3-Coder-30B	—	25.1%
DGM	Qwen3-Coder-30B	—	27.1%
HGM	Qwen3-Coder-30B	—	30.5%
SIFT	Qwen3-Coder-30B	Qwen3-Coder-480B	31.1%
SIFT	Qwen3-Coder-30B	gpt-5.4	32.0%
Base agent	o3-mini	—	14.2%
DGM	o3-mini	—	30.7%
SIFT	o3-mini	gpt-5.4	35.1%
SIFT	o3-mini	gpt-5-mini	31.6%

One Qwen SIFT run reaches 31.1% after 30 steps with $34.3 in API cost, 224 CPU-hours, and 6.7 hours of wall-clock time. The o3-mini/gpt-5.4 configuration reaches 35.1%; the same table reports $86.8, 59 CPU-hours, and 2.1 hours. These numbers support improved efficiency under the paper’s settings, not a universal cost/performance guarantee.

Transferability

Figure 4 (original paper Figure 4a, Section 4.1): the harness found with o3-mini is re-evaluated with other coding models, and the authors report improvement over the corresponding base agents. Original asset: arXiv Figure 4a · Section 4 anchor. This is an author-run transfer experiment, not an independent replication; the original page marks it CC BY 4.0.

The paper also transfers the Qwen3-Coder-30B configuration to other coding models. This matters because a patch that only helps the model that generated it could be overfitting to a judge or a backbone. Transfer is initial evidence of broader harness value, but the model count, task distribution, and full environmental details are not enough to establish generality.

Experiment 2: TerminalBench shows that judge rank is not accuracy rank

TerminalBench 2.1 contains long-horizon terminal tasks. Search uses a fixed random 50-task subset; selected agents receive three repeated full evaluations on 89 tasks:

Selection	Search evaluation	Repeated full mean
Starting agent	14/50	26.0/89 (29.2%)
SIFT judge rank 1	18/50	32.7/89 (36.7%)
SIFT accuracy rank 1	19/50	25.0/89 (28.1%)
No-judge accuracy rank 1	19/50	26.0/89 (29.2%)

The striking result is that the highest-scoring search candidate is not the best full-benchmark candidate. The judge-selected node scores only 18/50 during search but averages 36.7% in repeated evaluation. This supports judge rank as a way to find candidates hidden by subset noise, not as evidence that the judge can replace execution.

Figure 5 (original paper Figure 5, Section 4.2): nodes are grouped by BT judge rank and compared with realized full-benchmark accuracy. Original asset: arXiv Figure 5 · Section 4.2 anchor. It is a CC BY 4.0 paper result; correlation should not be read as a causal guarantee.

The stronger judge trade-off

The gpt-5.4-high judge reaches a 36.7% repeated mean, with ρ=0.72 between BT rank and full score and top-five recall of 4/5. The weaker gpt-5 judge still reaches 34.5% and ρ=0.71, but top-five pairwise agreement falls to 0.50. A cheaper judge can steer search toward the right region, yet make more mistakes when ordering the final frontier.

A practical design could therefore be tiered: a cheap judge handles most comparisons, a stronger judge reorders only the frontier, and real benchmark execution makes the final decision. This is a reasonable extension, not a completed production recipe from the paper.

What the judge sees changes ranking quality

The paper compares four candidate representations: diffs, diffs with swap ordering, full files, and full files with swap ordering. Across 50 non-root nodes:

Judge input	Spearman ρ	Cost per comparison
Diffs	0.40	$0.0076
Diffs + swap-order	0.43	$0.014
Full files	0.68	$0.011
Full files + swap-order	0.67	$0.021

This is an engineering-relevant counterintuitive result: full files provide more context than a diff, yet cost less than swap-ordered full files and produce the strongest ranking correlation. The authors therefore use full-file comparisons without swap ordering. A plausible explanation is that the judge can reason about the final runtime implementation directly instead of reconstructing behavior from a patch chain.

Figure 6 (original paper Figure 6, Section 4.3): averages across five runs decompose the contribution of disaggregation and speculative judge-guided expansion. Original asset: arXiv Figure 6 · Section 4.3 anchor. This is an author-reported result, not a general latency guarantee; the original page marks it CC BY 4.0.

SWE-60: reruns expose the instability of a single search score

In the SWE-60 appendix experiment, the starting agent scores 40.0% once. SIFT judge rank 1 (node 16) scores 53.3% during search and averages 50.4% over four full-60 evaluations; judge rank 2 (node 12) averages 53.8%. The no-judge accuracy rank 1 (node 11) scores 51.7% in search but only 44.6% across repeats.

Figure 7 (original paper Appendix A Figure 8, SWE-60 experiments): pooled repeated evaluations show why judge-guided selection can be more stable than selecting only by the search score. Original asset: arXiv figure asset · Appendix A anchor. The original page marks it CC BY 4.0; it is still a SWE-60 subset result, not a full SWE-bench claim.

The qualitative lineage makes the preference concrete. One favored lineage adds LLM-style wrappers to the git_patch tool, then adds guarded recovery for malformed patches and support for full-file/add-file wrapper blocks. The judge prefers the more conservative node because another patch could silently place an edit incorrectly. Static code review does not prove behavior, but it can expose a runtime risk that a single noisy benchmark score has not yet revealed.

Safety and evaluation integrity

The authors observed diagnosis agents occasionally proposing patches that relaxed timeouts or retry counts. If those patches were allowed, the agent could modify the evaluation environment and create a fake improvement. The paper uses three minimum controls:

1. Run candidates in sandboxed Docker containers.
2. Allow writes only to explicit runtime files, rejecting patches that touch benchmark or harness code.
3. Keep patch, judge decision, and downstream evaluation separate so judge rank cannot become the sole adoption condition.

This is why SIFT should be read as an engineering system rather than a single ranking trick: search efficiency is the benefit, but modification boundaries and evaluation provenance are the prerequisites.

Limitations: why this is 28/30, not 30/30

• No verified public implementation: the paper describes the method, prompts, costs, and experiments, but no official repository was verified for an independent rerun at the time of reading.
• The strongest judge is stronger than the coding backbone: this is practical, but it is not pure self-judged improvement.
• A single latent-strength assumption: BT compresses a candidate into one scalar rank. A patch can help long-horizon debugging while harming syntax repair; one rank hides that trade-off.
• Limited benchmark scope: core evidence is concentrated on Polyglot, TerminalBench, and SWE-60 coding-agent settings, not research agents, browser agents, or enterprise workflows.
• Limited independent verification: repeated runs and cross-model transfer are author-run evidence; model versions, task availability, and API pricing can all change.

Bloss0m engineering judgment and when not to use it

SIFT is a good fit when there are many candidates, full evaluation is expensive, and the search system can pin the subset, preserve runtime snapshots, and enforce a sandbox. Do not put judge rank into an automatic adoption path when the task is high-risk, the candidate set is small, or the platform cannot retain benchmark traces, allow-lists, and reproducible environments. In particular, never ship a self-modification directly to production from judge preference alone.

Evidence map: paper directly supports, author claims, and engineering inference

• Paper directly supports: the Polyglot, TerminalBench, and SWE-60 settings, comparison tables, cost reports, reruns, and judge-input ablation.
• Author claims: BT-guided asynchronous search can find stronger and transferable coding-agent harnesses with less resource use.
• Engineering inference in this article: production should treat judge rank as a speculative signal and combine versioned snapshots, sandboxing, full benchmarks, and human approval into an adoption gate. That is not a deployment guarantee established by the paper.

Engineering translation for agent platforms

I would turn SIFT into five explicit contracts:

Contract	Artifact to retain	Failure handling
Candidate patch	parent commit, complete changed files, dependencies, versions	reject if the parent cannot be reconstructed
Judge comparison	input snapshot, model, prompt, outcome, rationale	mark low confidence when the comparison graph is disconnected
Intermediate evaluation	fixed subset, task version, timeout, trace	use subset score for ranking only, never for publishing
Full verification	benchmark result, reruns, cost, environment hash	return to frontier review when variance is high
Adoption	allow-list diff, human approval, production canary	never write to production directly from a benchmark result

This mapping is an implementation checklist derived from the paper’s safety discussion and evidence boundary, not a new theorem. The important operational rule is to keep judge-only rank and full-benchmark result as separate fields so dashboards do not display a speculative preference as verified quality.

Three takeaways to remember

1. SIFT ranks before it verifies: pairwise judging and BT ranking decide what to explore, not whether the benchmark can be skipped.
2. Asynchrony is half of the cost story: the signal is useful because expansion, subset checks, judging, and full evaluation overlap.
3. Reliability comes from boundaries and reruns: strong judges, fixed subsets, sandboxing, allow-lists, and repeated full evaluation are all necessary; without a public implementation, the 28/30 reproducibility gap remains.

Primary sources

The primary sources for this article are the arXiv abstract, arXiv HTML v1, arXiv PDF v1, and the original figure assets. All numbers, figures, and limitations are fixed to v1; this article does not claim an independent rerun.

Sources and figure index

• arXiv abstract
• arXiv HTML v1
• arXiv PDF v1
• Original figure assets
• Bradley–Terry model background