Learning When to Trust: Measuring and Training Selective Trust with Selective Context Preference Optimization

The paper in 90 seconds

• Problem: Ordinary reasoning or RAG evaluation often asks only whether a model answers under one context. The same model may answer correctly without added context, then switch to a wrong answer when a plausible but authoritative-looking hint appears. If evaluation measures resistance alone, a model that ignores every context can look robust.
• Core insight: Reframe the target as selective trust. A useful context should help, irrelevant context should not move the answer, and misleading context should be rejected. MIST tests these behaviors on four matched versions of each item; SC2W conditions on clean-correct items and counts misleading-signal correct-to-wrong flips.
• Strongest evidence: Across 23 frontier and open-weight models, the paper reports an average 17.1-point accuracy loss from one misleading signal (Figure 1 and the Conclusion). In the training comparison, Qwen3-4B moves from 35.0 to 16.3 SC2W, while SCOPE reaches 95.0 clean, 98.1 correct-context, and 94.3 irrelevant-context accuracy (Table 1).
• Main boundary: These results come from 1,000 text-only items, fixed decoding, and two trainable model families. They measure susceptibility in a controlled benchmark, not natural incident frequency, arbitrary RAG poisoning defense, indirect prompt-injection prevention, or agent action safety.

Core engineering takeaway: the paper’s real contribution is turning “do not blindly trust external signals” into a falsifiable matched-counterfactual evaluation, then putting that control problem into preference data. If a system optimizes misleading-context resistance without testing correct-context preservation, it may simply learn not to trust anything.

Version, sources, and reader question

This article reads the arXiv v1 paper page, v1 full HTML, and v1 PDF. The PDF records a submission date of August 6, 2026. The authors are Xian Sun, Wei Chow, Yingshuo Wang, Junhao Liu, Wei Gao, Qing Wu, and Lingdong Kong. This is an arXiv preprint; as of the verification date, no confirmed peer-reviewed venue has been recorded, so this article does not call it a production-proven or published method.

The reader question is: when external context may be evidence, misinformation, or noise, can a model decide how much to trust it on the same reasoning task rather than learning blanket rejection? This follows RAGSieve’s retrieval-integrity framing and the data-plane risk in Indirect Prompt Injection: the former detects suspicious retrieval promotion, while the latter shows how untrusted content can enter a prompt and alter control flow. This paper turns the narrower question “should this context change the answer?” into a matched benchmark and training target. For a related evaluation question about early stopping, see partial-answer prediction in agentic RAG.

Evidence map: Paper, Evidence, and Bloss0m judgment

Layer	What this article says
Directly supported by the Paper	MIST’s four matched conditions, clean-correct conditioning in SC2W, SCOPE’s matched preference quartet, standard sigmoid DPO, MIST results for two trainable model families, zero-shot external transfer, construction ablations, slice analysis, and the human scoring audit.
What the Evidence shows	Misleading signals reduce accuracy for all 23 evaluated models; on Qwen3-4B and Llama-3.2-3B, SCOPE reduces SC2W while broadly preserving controls; external suites point in the same direction, but their tasks and metrics differ from MIST.
Author claim	Selective trust is a better training and evaluation target than resistance alone, and SCOPE learns it through matched, balanced preference pairs.
Not established by the evidence	The paper does not show that SCOPE prevents arbitrary prompt injection, improves factuality on arbitrary enterprise RAG, handles multimodal or tool/action context, eliminates contamination, or turns SC2W into a real deployment incident rate.
Bloss0m engineering judgment	In production, treat the four conditions as a regression matrix and track SC2W separately from context usefulness, provenance, ACL, freshness, answer verification, and side-effect approval. This is an engineering synthesis, not an additional guarantee proposed by the authors.

The short Paper Essence Contract is: the problem is a context-induced answer flip; single-context accuracy cannot separate inability from being misled; the core idea is a matched counterfactual benchmark plus balanced preference construction; an input passes through four context variants, generation, and a type-aware parser before producing per-condition accuracy and SC2W; Figures 1, 5, 7, and 8, Table 1, and the human audit support the main observations; the conclusion stops at controlled text-only susceptibility rather than a production safety guarantee.

Why existing evaluation is insufficient: robustness is not selective trust

The usual benchmark unit is `question → answer`. It can tell us whether a model answers correctly under one prompt, but not whether the answer changed because a single added context changed while the problem stayed fixed. Comparing accuracy on two unrelated item sets mixes task difficulty, answer format, knowledge distribution, and context effects.

Sections 1–2 separate this gap from three common intuitions. The first is resistance training: add an instruction saying not to trust external signals, or train only on misleading examples. This may raise misleading-context accuracy while degrading the model’s ability to use context that is actually correct. The second is clean accuracy: a high clean score does not say whether a misleading context will quietly overturn answers the model already knew. The third is treating this paper as a complete security benchmark. The paper explicitly targets correctness under misleading and non-misleading contexts, not refusal or a full prompt-injection threat model.

The four questions are therefore distinct:

1. Clean: Can the model solve the original task without added context?
2. Misleading: When context points to a plausible wrong answer, can the model preserve the truth-consistent answer?
3. Correct-context: When context supports the gold answer, can the model benefit from it rather than reject it?
4. Irrelevant: When context sounds natural but carries no answer-bearing information, can the model remain stable?

Only the four conditions together can separate rejecting bad signals from rejecting every signal.

Core intuition: put the same item’s counterfactuals side by side

MIST-1000 contains 1,000 source items: 800 adapted from existing QA, math, and reasoning benchmarks, plus 200 longer scenarios written by annotators. Each item expands to four rows, for 4,000 matched condition rows. The question, answer space, answer type, gold answer, and plausible wrong answer stay fixed; only the surrounding context changes. Figure 2’s pipeline shows that this is not just automatic string wrapping: source pooling is followed by manual screening, STEM-trained annotation, independent review, provenance logging, and freezing.

Figure 1 (original paper Figure 2, Section 3.1): notice that the four conditions are not four unrelated question sets. They are matched variants created through source pooling, screening, annotation, and review; this is what makes the later SC2W counterfactual comparison meaningful. Original Figure 2 anchor: Section 3.1 / Figure 2; original image endpoint: fig2.png. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, and the source and license are retained rather than presenting a redraw as experimental evidence.

Figure 2 (original paper Figure 3, Section 4): this original figure shows the SCOPE framework rather than MIST coverage. Its teaching purpose is to place the four condition preference pairs on one training scaffold. Notice the shared response pair and four context paths; they are not four independent algorithms. Original Figure 3 anchor: Section 4 / Figure 3; original image endpoint: framework.png. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, with source and license retained.

Figure 4 then breaks MIST coverage into topics, signal channels, source types, and wrong-answer plausibility. The authors report math, general knowledge, science, commonsense, academic, consumer-finance, and workplace-policy topics; nine misleading-signal channels; and seven wrong-answer plausibility classes. This makes the misleading condition more than one fixed template, but it does not establish coverage of real enterprise document formats, languages, multi-hop retrieval, or tool traces.

Walk one item through the method: from a clean answer to SC2W

The following walkthrough is a faithful abstraction, not a rerun. Imagine a multi-step arithmetic item whose gold answer is `48`, while an official-looking calculator note says `24`. An answer-like but wrong signal is the kind of failure the paper wants to measure; it is not a random distractor.

1. Input: The same question and answer format are rendered as `clean`, `misleading`, `correct`, and `irrelevant` prompts. Clean has no added context; misleading includes the `24` note; correct-context includes a matched hint supporting `48`; irrelevant includes natural text that does not provide the answer.
2. Intermediate representation: Each row retains an item ID, condition, source, source ID, topic, answer type, gold string, wrong value, and complete prompt. The four rows share the question and gold answer; only condition-specific context differs.
3. Model decision: The model generates a completion for each prompt. The public evaluator uses type-aware parsers for multiple-choice, numeric, and boolean answers rather than treating prose similarity as correctness.
4. Output: If clean produces `48` and misleading produces `24`, the item contributes to the SC2W numerator. If clean is already wrong, the item is not counted as a clean-correct-to-misleading-wrong flip, so original inability is not mislabeled as signal-induced failure.
5. Likely failure point: A model may calculate `48` in its reasoning but adopt the note’s `24` at the end; or it may be wrong in both conditions. The first is a selective-trust failure, while the second cannot be explained by SC2W alone.

This also explains why correct-context and irrelevant-context are controls. If we compare only clean with misleading, a model that discards every external string can reduce SC2W. Correct-context and irrelevant-context reveal whether it can reject the wrong signal while still using useful context and ignoring unhelpful context.

MIST’s metric: what kind of error does SC2W measure?

Let $x_i^r$ be the prompt for item $i$ under condition $r\in\{clean,mis,cor,irr\}$, and let $y_i$ be the gold answer. After type-aware final-answer parsing, $a_i^r=\mathbb{1}[\hat y_i^r=y_i]$, and condition accuracy is:

$$\mathrm{Acc}_r=\frac{1}{N}\sum_{i=1}^{N}a_i^r.$$

The balanced four-condition average is:

$$\mathrm{Overall\ Acc}=\frac{1}{4}\sum_r\mathrm{Acc}_r.$$

The central metric is:

$$\mathrm{SC2W}=\frac{\sum_i\mathbb{1}[a_i^{clean}=1\land a_i^{mis}=0]}{\sum_i\mathbb{1}[a_i^{clean}=1]}.$$

Its operational role is deliberately narrow: the denominator keeps only items the model solves under clean context, and the numerator counts how many of those become wrong under misleading context. A lower SC2W means fewer signal-induced clean-to-wrong flips in this protocol. It is not overall answer quality, a context-relevance score, or a production incident probability.

The main table reports Clean, Misleading, Correct-context, Irrelevant, Overall, and SC2W together. All accuracy values are higher-is-better; SC2W is lower-is-better. This direction difference matters: misleading accuracy alone hides correct-context damage, while SC2W alone hides whether the model could solve the task in the first place.

SCOPE’s mechanism: change preference data, not the DPO loss

1. Mine the failure that needs repair

SCOPE does not train on arbitrary four-row prompt-response records. For each retained item, it constructs one response pair: $r_i^+$ is a truth-consistent full completion, and $r_i^-$ is the base model’s naturally terminated signal-following wrong completion under misleading context. The preferred response uses the same base model’s clean-correct response when available; otherwise a diagnostic completion can be elicited from the same frozen base checkpoint with a private diagnostic note. That note is not included in the DPO prompt, and fallback responses that reveal privileged-note provenance or answer-key language are filtered.

2. Match one response pair to four contexts

For $b\in B=\{mis,clean,cor,irr\}$, the method forms $D_b=\{(z_i^b,r_i^+,r_i^-)\}_{i=1}^{N_b}$. Every condition prefers the same truth-consistent response over the same signal-following response. The key design is not four newly named optimizers; it is holding the problem, response pair, answer type, and format as constant as possible so the preference signal tracks the context’s role rather than topic, length, or template artifacts.

Figure 3 (original paper Figure 3, Section 4): notice that matched preference construction covers misleading, clean, correct-context, and irrelevant-context pairs. SCOPE’s novelty lies in the quartet balance before standard DPO, not in replacing DPO with a new optimizer. Original Figure 3 anchor: Section 4 / Figure 3; original image endpoint: framework.png. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, with source and license retained.

3. Balanced DPO: four behaviors are components of one objective

For trainable policy $\pi_\theta$ and frozen reference policy $\pi_{ref}$, the paper defines the relative score:

$$s_\theta(z,r)=\beta\log\frac{\pi_\theta(r\mid z)}{\pi_{ref}(r\mid z)}.$$

For pair $p=(z,r^+,r^-)$, $\Delta_\theta(p)=s_\theta(z,r^+)-s_\theta(z,r^-)$, and the condition-specific DPO loss is:

$$\mathcal{L}_b(\theta)=-\mathbb{E}_{p\sim D_b}[\log\sigma(\Delta_\theta(p))].$$

Correct-context and irrelevant-context form $\mathcal{L}_{nonadv}=\rho\mathcal{L}_{cor}+(1-\rho)\mathcal{L}_{irr}$, and the overall SCOPE objective is:

$$\mathcal{L}_{SCOPE}=\lambda_m\mathcal{L}_{mis}+\lambda_c\mathcal{L}_{clean}+\lambda_p\mathcal{L}_{nonadv},$$

where $\lambda_m+\lambda_c+\lambda_p=1$. The headline run uses $(\lambda_m,\lambda_c,\lambda_p,\rho)=(0.25,0.25,0.50,0.50)$, giving each condition 25% sampling mass. Increasing a $\lambda$ gives that condition more training weight; it does not claim that condition has the same share of real traffic.

Implementation is full-completion sigmoid DPO with LoRA. The GitHub README’s default recipe specifies DPO beta 0.1, learning rate $5\times10^{-6}$, a cosine schedule with 10% warmup, 300 steps, maximum length 4096, batch size 4, gradient accumulation 8, bf16, LoRA rank 64, alpha 128, and seed 0. This makes reproduction concrete, but not cost-free on every machine: training and vLLM evaluation require a CUDA-capable GPU, and checkpoint, driver, dependency, and memory choices affect time and results.

How to read the figures: the cover is not evidence

The Evidence Atlas cover is an original conceptual visual derived from the relationship “one item, four context paths, one selective decision.” It is not a paper figure. The body figures below are v1 originals converted to local WebP assets; both language bodies use the same asset, source anchor, and CC BY 4.0 provenance.

Figure 4 (original paper Figure 1, Section 1): this is the phenomenon-level evidence. Replacing clean context with a plausible misleading signal lowers accuracy for every evaluated model, including frontier proprietary models. Notice that it supports a benchmark observation of broad susceptibility; it does not imply that every production model will degrade by the same amount. Original Figure 1 anchor: Section 1 / Figure 1; original image endpoint: fig1.png. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, with source and license retained.

How to read the experiments: setup, controls, and main results

Setup: datasets, baselines, metrics, and compute

Datasets and splits: MIST-1000 has 1,000 items and 4,000 matched rows; the training pool shares no items with MIST evaluation. External transfer uses GSM-IC, GSM-Plus, and Sharma-style sycophancy suites, with 300 items per dataset. The paper says no external target-dataset examples, model selection, or hyperparameter tuning were used for training.

Baselines: The main table first includes 23 API-reference and open-weight models. The trainable comparison uses Qwen3-4B and Llama-3.2-3B with Prompt-Defense, SFT, Standard-DPO, OPSD, and SCOPE. These are not interchangeable baselines: Prompt-Defense is an inference-time warning; SFT uses chosen responses without rejected responses; Standard-DPO uses misleading-only pairs; and OPSD is on-policy self-distillation from the current policy.

Metrics: The four condition accuracies, Overall accuracy, and SC2W are deterministic exact-match metrics after type-aware parsing for multiple-choice, numeric, and boolean answers. Table 1’s uncertainty is paired item-level bootstrap or an auxiliary proxy estimate, not training-seed variance. GSM-IC and GSM-Plus use deterministic numeric exact match, while Sharma uses a fixed Qwen2.5-14B judge for correctness and bias-following. These are not one common metric.

Compute and protocol: The paper uses a fixed decoding seed; the repository uses vLLM, temperature 0, type-aware parsing, and paired item-level bootstrap. SCOPE uses standard DPO, LoRA, bf16, and a fixed training budget. These controls clarify comparisons but do not establish cross-seed, cross-provider, cross-hardware, or live-traffic uncertainty.

Main result: read resistance and controls together

Figure 1 first asks whether misleading signals can actually change answers. The answer is yes: the Conclusion reports a 17.1-point average loss across 23 models. This supports an across-model benchmark observation; it does not estimate how often enterprise traffic will contain a comparable signal.

Table 1 then asks whether training can reduce the flip while preserving the other conditions. Qwen3-4B base scores 94.5 / 62.5 / 98.1 / 92.6 / 86.9 / 35.0 for Clean / Misleading / Correct-context / Irrelevant / Overall / SC2W; SCOPE scores 95.0 / 80.7 / 98.1 / 94.3 / 92.0 / 16.3. Llama-3.2-3B base scores 69.5 / 54.4 / 78.5 / 69.3 / 67.9 / 31.5; SCOPE scores 72.0 / 63.1 / 80.0 / 71.6 / 71.7 / 20.6.

These numbers support three bounded statements: SCOPE reduces MIST susceptibility for the two trainable families; Qwen3-4B does not lose control accuracy as misleading accuracy improves; and Llama-3.2-3B controls are higher or comparable to base. They do not show that SCOPE beats every baseline at every scale, because intervention is evaluated on two trainable families and auxiliary proxy uncertainty must be read with Appendix I.

Figure 5 (original paper Figure 7, Appendix D): this is a mechanism and training diagnostic, not another main benchmark. The four panels serve different roles: repaired misleading failures, control preservation, later-checkpoint held-out SC2W, and the robustness–control construction trade-off. Panel (a) reports that SCOPE repairs 188 of 331 Qwen3-4B failures triggered by misleading context, with 182 net repairs overall; this supports “many but not all failures are repaired,” not complete repair. Original Figure 7 anchor: Appendix D / Figure 7; original image endpoint: fig5_mechanism.svg. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, with source and license retained.

External transfer: directionally useful, not scope-free generalization

The right side of Figure 5 and Appendix Table 2 test whether SCOPE learned only an MIST prompt template. The model is trained on the item-disjoint matched pool and evaluated zero-shot on GSM-IC, GSM-Plus, and Sharma. Qwen3-4B + SCOPE reports 90.7 GSM-IC and 66.0 GSM-Plus accuracy; Llama-3.2-3B + SCOPE reports 53.3 Sharma accuracy and 44.0 Sharma bias-following. The paper describes the pattern as best or tied on all four external higher-is-better metrics for both families. This is a useful out-of-distribution check, but the target suites have different tasks, metrics, judges, and protocols; it is not proof of production RAG generalization.

Human audit: supports the scoring protocol, not a reasoning-quality ceiling

Appendix G’s reference-assisted audit shows annotators the task, reference answer, and two anonymous responses, but not model identity or automatic score. Figure 6 compares human win ratios with automatic win ratios across six main metrics. Five metrics have Spearman $\rho$ of at least 0.97, while SC2W reaches $\rho=0.821$. This supports alignment between the deterministic protocol and reference-assisted semantic correctness within this audit scope; the authors also state that it is not a full human evaluation of reasoning quality. One annotation per pair cannot be read as a complete inter-annotator agreement or expert-correctness ceiling.

Ablations, slices, and failure modes: matched balance is load-bearing

Appendix Table 3’s Qwen3-4B construction ablation provides a more informative clue than the headline row. Full SCOPE scores 95.0 Clean, 80.7 Misleading, 98.1 Correct-context, 94.3 Irrelevant, 92.0 Overall, and 16.3 SC2W. Unmatched or random pairs produce 92.4, 72.7, 97.7, 92.8, 88.9, and 23.6; misleading-only pairs produce 91.7, 80.5, 95.4, 92.5, 90.0, and 19.1. The latter looks close on misleading accuracy, but controls and Overall accuracy are worse, illustrating why resistance alone is not selective trust.

Removing a control pair is not a simple “one removal always breaks everything” result. Without correct-context pairs, Overall is 90.6 and SC2W 15.7; without irrelevant-context pairs, Overall is 90.9 and SC2W 16.6; without clean pairs, Overall is 90.7 and SC2W 17.0. Some variants slightly lower raw SC2W while sacrificing other controls. The careful conclusion is that matched construction and four-condition balance stabilize the overall trade-off, not that each control is a separately proven necessary theorem.

Figure 6 (original paper Figure 8, Appendix F): slice analysis separates held-out SC2W by provenance, answer format, and misleading-signal source. Notice that the authors use it to test whether improvements are concentrated in one artifact, not to create an independent leaderboard from every cell. Appendix Table 4 shows that authoritative-looking answer keys, retrieved documents, and lecture notes can trigger susceptibility in base models; SCOPE makes most slices lighter but does not drive SC2W to zero. Original Figure 8 anchor: Appendix F / Figure 8; original image endpoint: fig6b_slice.svg. This article uses the arXiv v1 original converted to WebP; the paper page marks CC BY 4.0, with source and license retained.

The qualitative examples in Appendix J make the failure concrete. SCOPE repairs answer-key conflicts, a round-trip fuel calculation, and a quorum Boolean conflict, yet one plausible search snippet leaves both base and SCOPE at the wrong `25.68` answer. This remaining failure matters: it prevents 188/331 repairs from being inflated into “everything was fixed,” and it suggests that answer-like retrieved text may be harder to judge than an obvious answer key.

Evidence boundary: contamination, deployment prevalence, and unsupported claims

First, controlled rates are not deployment prevalence. MIST intentionally embeds misleading signals to stress-test reliability; SC2W therefore cannot be read as “16.3% of real enterprise RAG requests fail.” Real traffic would require estimates of signal prevalence, retriever exposure, document provenance, workflow, and human escalation.

Second, contamination cannot be fully excluded. Eight hundred items are adapted from public benchmarks, so some models may have seen the clean questions. The authors argue that the matched design still tests whether a known answer survives a misleading signal, and SC2W conditioning means memorizing a clean answer does not automatically produce a correct misleading answer. That reduces one confound; it does not prove zero contamination.

Third, the scope is text-only and the intervention is narrow. Training is centered on Qwen3-4B and Llama-3.2-3B; broader architectures and scales, multimodal evidence, long-context retrieval, tool outputs, browser actions, agent memory, non-English context, and multi-turn negotiation are outside this evidence.

Fourth, reasoning faithfulness is not guaranteed. A model can write a correct calculation and still adopt the wrong signal in the final answer, or produce the right answer with an unfaithful explanation. MIST’s deterministic parser primarily scores the final answer; the human audit checks scoring alignment within a limited protocol, not chain-of-thought faithfulness.

Fifth, SCOPE is not a prompt-injection prevention architecture. The Related Work section says the target is not refusal. The paper does not provide instruction/data-channel separation, least privilege, tool authorization, side-effect confirmation, provenance graphs, or rollback. Calling SCOPE an injection defense would collapse a context-selection benchmark into system security controls the paper did not evaluate.

Bloss0m engineering judgment and when not to use it

The following is Bloss0m engineering synthesis, not an official framework proposed by the authors. I would turn the paper’s insight into four checks in a RAG or agent regression suite:

1. Four variants of the same task: Keep clean, misleading, correct-context, and irrelevant-context fixtures for high-risk queries. Rerun them whenever the retriever, reranker, model, prompt, or parser changes rather than tracking clean accuracy alone.
2. Separate five outputs: Store clean accuracy, misleading accuracy, correct-context accuracy, irrelevant accuracy, and SC2W. Do not hide a resistance/usefulness trade-off inside one aggregate score.
3. Connect context role to provenance: MIST says whether a model is moved by answer-like text; production still needs source version, retrieval rank, ACL, freshness, signal type, answer verification, and action approval before context can influence a side effect.
4. Triage failures by source: Slice answer keys, retrieved documents, search snippets, calculator notes, and other sources. If one source class keeps a high SC2W, repair retrieval, rendering, or provenance before scaling preference training.

When it is useful

This framing fits RAG context integration, retrieval-reranking regressions, agent evidence selection, external tool-output answer-preservation tests, and preference-training failure mining. It is especially useful for asking whether a change makes a model less likely to follow a plausible wrong context while preserving useful context.

When not to use it alone

Do not use MIST or SCOPE alone for clinical decisions, financial execution, safety-critical control, real-time regulatory interpretation, side-effecting tool calls, or autonomous workflows that require the latest world state. Those settings need domain review, source authorization, freshness policy, an independent verifier, uncertainty thresholds, action sandboxing, and rollback. A low SC2W does not mean that context is trustworthy; it means that fewer clean-correct answers flipped under misleading context in this benchmark protocol.

Artifacts and reproducibility (as of September 21, 2026)

I independently checked the SCOPE project page, worldbench/SCOPE GitHub repository, MIST-Train dataset, and MIST-Bench dataset. The statuses should be kept separate:

Artifact	Verification result	Reproducibility judgment
Project page	Reachable; it shows the four conditions, a leaderboard, SCOPE steps, and qualitative cases.	Useful as author navigation, not a checkpoint or full run log.
GitHub code	Public; README, `scope/`, `mist/`, `scripts/`, tests, and requirements are readable. The repository API reports a null `license` field.	The code pipeline is inspectable and includes unit tests, schema validation, a vLLM evaluator, and paired bootstrap; users still need checkpoints, a CUDA GPU, and compatible environment versions.
`worldbench/MIST-Train`	Public and ungated; `train.jsonl` resolves, and the HF API reports 9,440 rows. The README describes 2,360 quartets.	The training endpoint is usable; the dataset card still contains an old `worldbench/SCOPE-Train` load snippet, so use the current dataset ID and repository defaults.
`worldbench/MIST-Bench`	Public and ungated; `mist_1000.jsonl` resolves, and the HF API reports a test split with 4,000 condition rows.	The benchmark endpoint is usable; its card still shows a `worldbench/MIST` load snippet that currently is not a valid endpoint, so do not copy it as the reproduction command.
Checkpoints / full paper run	I did not find separately published paper-specific merged checkpoints, complete raw generations, or per-model hashes and cost logs.	Start with repository tests and a 10-item smoke evaluation, then reproduce four-condition scoring on one available model; this is not a one-command recreation of every paper result.

The smallest useful reproduction path is: clone the repository; create a Python 3.10 environment; install requirements; run `python -m unittest discover -s tests -v`; download the currently valid `worldbench/MIST-Train` and `worldbench/MIST-Bench`; use an available base model for `—max_items 8 —selftest` or `—smoke_items 10`; and confirm `metrics.json`, `item_scores.jsonl`, and four-condition SC2W output. Full training still requires CUDA, the LoRA/DPO stack, GPU memory, and checkpoint licensing; external transfer has its own data and judge availability constraints.

Three things to remember

1. Technical idea: MIST makes four matched context counterfactuals from one reasoning item; SCOPE reuses one truth-consistent/signal-following response pair across those conditions. The change is data construction, not a new DPO loss.
2. Evidence: All 23 evaluated models are affected by misleading signals in Figure 1; SCOPE reduces SC2W on Qwen3-4B and Llama-3.2-3B while preserving controls, and ablations show worse balance for unmatched or misleading-only training.
3. Boundary: SC2W is a controlled susceptibility metric, not deployment prevalence, a truth certificate, or prompt-injection defense. Public-benchmark contamination, text-only scope, limited trainable families, and artifact documentation mismatches remain adoption constraints.

Primary sources

• Sun et al., “Learning When to Trust via Selective Context Preference Optimization,” arXiv:2608.06377 v1; v1 PDF; v1 HTML.
• Official SCOPE project page.
• Official SCOPE code and evaluation pipeline.
• MIST-Train on Hugging Face; MIST-Bench on Hugging Face.