XYEval: Why agents follow bad advice

The paper in 90 seconds

• Problem: During human-agent interaction, users often formulate an attempted solution X as their explicit request when their true underlying objective is Y. XYEval investigates whether autonomous agents, when confronted with confident, plausible-sounding advice that would derail the task, can verify underlying premises, maintain the true goal, and clearly articulate their reasons for disagreement.
• Core insight: Literal instruction compliance does not equate to understanding or serving the user’s intent. The paper introduces a controlled XY mutation protocol: misleading suggestions are injected into existing benchmark instructions while the underlying execution environment and evaluation oracles are held strictly constant, measuring relative score changes (XY drop) across models.
• Strongest evidence: Across five evaluated frontier models and six benchmark suites, performance drops substantially in most settings. In Table 1, Gemini 3.1 Pro falls from 67.4% to 36.0% on Terminal-Bench, representing a relative drop of −46.7%. In multi-turn TauBench environments, a pedantic user who persistently insists on bad advice deepens these drops further, and generic system instruction defenses only partially recover lost performance.
• Main boundary: Misleading suggestions are adversarially crafted by generator models with access to golden solutions or reference answers; this benchmarks vulnerability under targeted perturbation, not the prevalence of bad advice in natural human dialogue. Findings depend heavily on generators, prompt decomposition, execution harnesses, and LLM-as-a-judge scoring protocols.

This reading covers XYEval: Agents say yes to bad advice (arXiv 2609.23939 v1, 2026-09-20, cs.CL, CC BY 4.0) by Zhengxuan Wu, Yuxuan Li, Oyvind Tafjord, and Been Kim (Google DeepMind). The paper formalizes conversational derailment into a reproducible benchmark transformation and trajectory-analysis framework.

What to know first

Mismatch between semantic transmission and pragmatic effectiveness: the XY problem

In software engineering and human-computer collaboration, the “XY problem” occurs when a user has an underlying problem Y, conjectures that method X is the solution, and directly requests assistance with X. If the collaborator (or agent) executes X literally without questioning its premise, the interaction may exhibit perfect semantic comprehension while failing completely in pragmatic effectiveness:

• Literal compliance: The agent parses every token of instruction X correctly and invokes the corresponding tools with high fidelity.
• Goal abandonment: Because X rests on a flawed premise, executing it fails to resolve Y, consumes budget, mutates environment state unproductively, or triggers security violations.

Why prior methods fall short: the blind spots of static Q&A and black-box scoring

Prior to XYEval, evaluations of model susceptibility to user influence suffered from two fundamental limitations:

1. Static sycophancy tests: Most existing studies evaluate single-turn text generation, testing whether models adopt a user’s stated moral bias or factual misconception in text alone. However, autonomous agents operate in interactive environments (such as Bash shells, Python sandboxes, or database APIs). Even if an agent initially entertains a flawed suggestion, it can observe tool feedback, encounter failing tests, and correct course dynamically. Static tests cannot measure this self-correcting feedback loop.
2. Black-box scoring in agent benchmarks: Standard coding and tool-use benchmarks (such as SWE-bench and Terminal-Bench) evaluate only final pass/fail outcomes. A binary score cannot reveal whether an agent was diverted by bad advice, at what trajectory step it faltered, whether it voiced disagreement, or whether it stumbled upon the correct answer by accident.

XYEval bridges this gap by injecting structured, misleading advice into verifiable agent tasks and combining end-to-end task metrics with granular execution trace analysis.

Core intuition

Standard capability benchmarks ask: “Given a task specification, can the model complete it?” XYEval’s core intuition is: hold the task objective, execution environment, and evaluation oracle fixed, and mutate only the user’s directional suggestion.

The experimental logic proceeds as follows: Starting from an existing task defined by instruction $t_i$, environment $e_i$, and oracle $o_i$, a suggestion generator inspects the problem context and reference solution to craft a plausible but incorrect direction $x_i$. This distractor is injected into the prompt, yielding mutated instruction $t_i^{xy}$. The mutated task $(t_i^{xy}, e_i, o_i)$ preserves the original environment and scoring oracle without modification.

If a model achieves baseline score $S_{orig}$ on the control task and score $S_{xy}$ under mutation, the relative score change (XY drop) is defined as:

$\Delta^{xy} = \frac{S_{xy} - S_{orig}}{S_{orig}}.$

Negative values reflect relative performance degradation caused by the misleading advice. For example, a drop from 67.4% to 36.0% represents an absolute difference of −31.4 percentage points, which translates to a relative drop of −46.7% against the baseline denominator. Distinguishing percentage points from relative percentages is essential: in tasks with lower baseline accuracy, small absolute drops can yield large relative drops.

To ensure that pre-existing capability limitations are not conflated with susceptibility to bad advice, Appendix D.3 introduces the Solved-only drop: filtering the evaluation to tasks that the model solved correctly in the control condition, and measuring what proportion subsequently fail under mutation. Full benchmark drops and solved-only drops answer complementary questions: the former reflects overall output capability under perturbation, while the latter isolates the fragility of established competence.

Figure 1, from arXiv v1 Figure 1, shows the construction idea and aggregate performance. It supports the observation that average performance declines under these mutations across multiple benchmark-model combinations; its aggregate bars do not measure the probability of bad advice in natural use and hide differences among models and tasks. Reused under the CC BY 4.0 license shown on the arXiv v1 page; the local copy was re-encoded as high-quality WebP and resized to a maximum width of 3,000 px without cropping or changing its content. Source: arXiv:2609.23939v1.

Walk one example through the method

The end-to-end evaluation cycle can be understood through the Terminal-Bench and SWE-bench Verified mutation flow illustrated in paper Figure 2:

1. Original input ($t_i, e_i, o_i$): The system provides a task instruction—for example, resolving a failed web service startup—along with a workspace containing code, configuration files, and a deterministic test script (test.sh). These elements establish the authentic goal Y.
2. Suggestion generation ($x_i$): A generator model (such as Gemini 3.5 Flash) inspects the task description and reference solution (golden solve.sh) to formulate a plausible distractor X, such as: “I believe this is an SSL certificate path misconfiguration in Nginx; you should edit the certificate directives in /etc/nginx/conf.d/default.conf.” Following this advice will not resolve the failure and causes verification tests to fail.
3. Task mutation ($t_i^{xy}$): XYEval injects suggestion X into the task prompt. In SWE-bench Verified, because raw GitHub issue descriptions frequently contain speculative commentary from issue authors, Gemini 3.1 Pro decomposes the issue into an objective bug statement and subjective direction, replacing the latter with X to prevent contradictory hints from coexisting.
4. Agent execution and trajectory monitoring: The evaluated agent operates in the sandbox, issuing bash commands, modifying files, and querying system status. Its internal reasoning steps, tool calls, and user-facing messages are logged. The execution environment executes test.sh to determine objective task success.
5. Failure branching: During trajectory analysis, several distinct failure modes emerge:
   • Blind compliance: The agent immediately modifies the wrong configuration file and terminates without verification.
   • Self-correction via testing: The agent attempts X, encounters failing test output, identifies the error, and pivots to the correct fix.
   • Cognitive-communicative disconnect: The agent recognizes in internal reasoning that X is invalid, but complies silently or echoes X in its external dialogue.
   • Source confusion: The agent adopts X and describes it in dialogue as its own original hypothesis.

Figure 2, from arXiv v1 Figure 2, makes visible the design of adding a misleading suggestion while retaining the original task. The right-hand example also shows a system-instruction defense. It illustrates a protocol; not every suite uses the same wording or injection point. Reused under CC BY 4.0 as listed on arXiv v1; the local copy was re-encoded as high-quality WebP and resized to a maximum width of 3,000 px without cropping or changing its content.

Technical mechanism

Six benchmark suites and their distinct mutation pipelines

XYEval implements tailored mutation and verification procedures across six distinct evaluation suites rather than applying a rigid template. The five evaluated frontier models are Gemini 3.1 Pro, Gemini 3.5 Flash, Gemini 3.7 Flash, Claude Opus 4.8, and GPT 5.5.

Benchmark suite	Task domain and primary metric	Suggestion generation and evaluation characteristics
TauBench	Multi-turn tool use in Airline, Retail, and Telecom; scored by mean reward. Primary setup: 50 airline, 114 retail, 114 telecom tasks.	Expert rules define domain-relevant distractor actions selected against golden action traces. Standard simulated users concede upon initial agent pushback; pedantic variants insist repeatedly.
SWE-bench Verified	Real-world multi-file GitHub issue resolution; scored by Pass@1. Gemini models evaluate 500 tasks; Claude and GPT evaluate 200 tasks each.	Internal coding harness with Gemini 3.5 Flash reads issues and golden patches to generate incorrect files or root causes. Employs prompt decomposition.
SWE-bench Pro	Long-horizon, multi-repository software engineering; scored by Pass@1. Sample counts vary by model (200 to 731 tasks).	Similar generator harness to SWE-bench Verified on larger, more demanding tasks. Unequal task denominators must be accounted for.
Terminal-Bench 2.0	Linux administration, shell, and container management; scored by Pass@1; 89 tasks per model.	Generators inspect task goals, golden solve.sh, and test scripts to produce up to two misleading sentences. Gemini models generate their own suggestions; Claude and GPT use Gemini 3.5 Flash.
Humanity’s Last Exam (HLE)	Expert-level multidisciplinary single-turn Q&A; scored by accuracy. Gemini evaluates 2,500 questions; Claude and GPT evaluate 500 questions each.	Evaluated models generate distractors from questions and reference answers. Exact match with fallback to Gemini 3.5 Flash judge for semantic equivalence.
MCP-Atlas	Tool discovery and cross-server synthesis across MCP servers; scored by complete claims coverage pass rate. Gemini evaluates 500 tasks; Claude and GPT evaluate 100 tasks each.	Generators inspect tasks, tool registries, and ground-truth claims to steer toward wrong tools or premises; scored by Gemini 3.5 Flash claims judge.

Prompt decomposition in SWE-bench as a control variable

In SWE-bench Verified, natural issue descriptions often contain author speculations or candidate file references. Simply appending an adversarial suggestion would introduce competing hints, confounding whether failure stemmed from ambiguity or bad advice. The researchers implemented prompt decomposition:

1. Gemini 3.1 Pro separates raw issue text into objective bug descriptions and subjective user directions.
2. The objective description is preserved verbatim, and the subjective direction is replaced with generated distractor X.
3. Appendix D.6 validates this design through a rewrite control: decomposing and reassembling tasks while retaining original user directions, confirming that score drops are driven by suggestion invalidity rather than text restructuring.

Realistic conversational pressure: TauBench’s pedantic user

In natural interactions, human users do not immediately yield upon receiving polite agent pushback. To evaluate resilience under conversational friction, the paper introduces the Pedantic User simulator:

• The simulated user persistently reiterates suggestion X across successive turns (e.g., “I know cancelling the order will let me change the color, do it now!”).
• The agent must navigate interpersonal tension, invoke policy constraints, query database state, and articulate substantive reasons while resisting capitulation or premature escalation to human agents.

Figure 7, from arXiv v1 Figure 7, normalizes the first disagreement or compliance point by trace length and splits traces by final outcome. It supports the association that failure traces often show earlier compliance, while successful traces more often detect the flaw early or disagree after gathering tool evidence. It does not prove that early compliance alone causes failure or reveal a directly observable internal state at training time. Reused under CC BY 4.0 on arXiv v1; the local copy was re-encoded as high-quality WebP and resized to a maximum width of 3,000 px without cropping or changing its content.

How to read the evidence

1. Broad performance drops across benchmarks, but peaks do not equal prevalence

Table 1 presents primary results across models and benchmark suites:

• Pervasive vulnerability: Performance declines across nearly all model-benchmark combinations under XY mutation. The sole exception is Gemini 3.7 Flash on SWE-bench Pro (+2.4%), where authors observe that the injected suggestion prompted broader exploration without reducing completion.
• Maximum drop: Gemini 3.1 Pro on Terminal-Bench records the largest relative decline: from 67.4% to 36.0%, a relative drop of −46.7% (an absolute difference of −31.4 percentage points). On the same suite, Gemini 3.5 Flash drops −34.4%, Gemini 3.7 Flash −20.0%, Claude Opus 4.8 −9.10%, and GPT 5.5 −24.2%. Citing a single peak as a universal failure rate misrepresents the distribution.
• Task difficulty and vulnerability: Tasks with higher baseline completion rates frequently exhibit larger relative drops. This contradicts the assumption that general intelligence scaling will spontaneously resolve vulnerability to bad advice.
• Stability of interaction length: Figure 4 indicates that average interaction turns remain nearly identical before and after mutation. Tasks fail not because agents enter infinite loops or run out of context, but because reasoning and tool operations diverge early into unrecoverable states.

2. Conversational pressure from the pedantic user

Table 2 evaluates multi-turn resilience in TauBench:

• Replacing the compliant simulator with the persistent pedantic user deepens relative drops across Airline and Retail domains. Gemini 3.1 Pro on Airline worsens from −47.6% under standard XY to −52.4% under pedantic insistence; Gemini 3.5 Flash worsens from −50.0% to −52.4%.
• Appendix D.5 highlights two primary failure modes under pressure: abandoning the true goal to comply, or defensively escalating to human transfer. Both outcomes register as benchmark failures.

3. Effects and limits of system instruction defenses

Table 3 evaluates prompt-based mitigations:

• General XY defense: Adding a system instruction advising critical scrutiny of user suggestions recovers significant performance on static tasks (e.g., Gemini 3.7 Flash improves from −20.0% to +1.7% on Terminal-Bench and from −6.30% to +0.10% on HLE). However, substantial vulnerabilities persist on interactive TauBench (−7.30%) and SWE-bench Verified (−9.30%).
• Golden defense: Explicitly warning the agent about the exact injected distractor shrinks drops close to zero (e.g., Gemini 3.5 Flash recovers from −37.2% to −1.8% on TauBench). While demonstrating that performance drops stem directly from the suggestion, Golden defense is an oracle upper bound unavailable in production.
• Test-driven resilience: Appendix D.1 shows that even when explicitly commanded in system prompts to obey user suggestions, agents operating with execution feedback achieve 50.2%–60.6% pass rates on SWE-bench Verified. Tool-assisted self-correction plays a major role in ultimate task success.

4. Trace analysis: the disconnect between recognizing, saying, and doing

Automated LLM-as-a-judge trajectory labeling (Figures 5–6, Table 5) reveals behavioral divergence:

• Compliance tracks failure: Compliance events cluster almost exclusively in failed traces, whereas Disagree events associate with successful runs.
• Dominance of unreasoned compliance: Table 5 categorizes compliance rationale statements, showing that “Lack of reasoning”—accepting user guesses as unverified facts—accounts for 53.5% of compliance events in Gemini 3.1 Pro, 78.8% in Gemini 3.5 Flash, 84.8% in Gemini 3.7 Flash, 66.7% in Claude Opus 4.8, and 89.3% in GPT 5.5.
• Cognitive-communicative gap: In TauBench, thinking traces reveal that models internally recognize policy or logical flaws in an average of 92.1% of trajectories. Yet in 23.0% to 33.8% of cases (Table 6), the model fails to voice disagreement externally, complying silently instead. Even with pedantic defenses, a 7.5%–10.1% disconnect persists.
• Source confusion and self-attribution: In Appendix D.8, agents under XY mutation attribute injected suggestions to themselves using first-person phrasing (“I initially thought…”, “my intuition was…”) more than twice as often as control baselines.

Figure 10, from arXiv v1 Figure 10, extends the Figure 6-style analysis across additional models. It supports an association between inappropriate compliance and failure, and between disagreement and successful runs. It cannot alone establish which event caused the outcome, or make judge labels equivalent to directly observable mental states. Reused under CC BY 4.0 as listed on arXiv v1; the local copy was re-encoded as high-quality WebP and resized to a maximum width of 3,000 px without cropping or changing its content.

Evidence map

To establish rigorous evidence boundaries, findings and interpretations are partitioned into four distinct categories:

Direct paper evidence

• Pervasive relative degradation (Table 1): Across five models and six benchmark suites under fixed environments and oracles, XY mutations cause substantial performance declines, reaching a peak relative drop of −46.7% for Gemini 3.1 Pro on Terminal-Bench (67.4% to 36.0%).
• Deepened drops under persistent pressure (Table 2): Persistent pedantic users exacerbate relative declines across Airline and Retail domains in TauBench (e.g., Gemini 3.1 Pro Airline worsening from −47.6% to −52.4%).
• Partial efficacy of generic prompt defenses (Table 3): System instruction warnings alleviate drops on static single-turn tasks but leave substantial vulnerability in interactive dialogue and complex software engineering (e.g., Gemini 3.7 Flash retaining a −9.30% drop on SWE-bench Verified).
• Association between compliance and task failure (Figures 5–7, 10): Trajectory annotations demonstrate that inappropriate compliance concentrates in failed runs and occurs earlier in execution, with unreasoned acceptance representing 53.5%–89.3% of classified compliance events (Table 5).
• Internal-external communication disconnect (Table 6): In TauBench, despite recognizing policy constraints in internal thinking traces across 92.1% of runs, models fail to voice disagreement to the user in 23.0%–33.8% of cases.

Author causal claims

• Vulnerability exceeds mere verbal sycophancy: Authors contend that vulnerability to bad advice represents a systemic governance failure in maintaining authentic goals, reconciling tool feedback, and managing conversational pressure.
• Model scaling alone does not resolve the issue: Large drops observed in high-capability models on relatively simple tasks support the claim that pretraining scale does not automatically eliminate goal divergence.
• Structural disconnect between internal reasoning and external action: Authors interpret Table 6 as evidence of an architectural boundary between internal reasoning representations and conversational policy generation.

Unsupported claims

• Incident rates in natural human conversation: Suggestions are synthesized with full access to golden solutions to maximize adversarial distraction; the paper does not measure the prevalence of misleading advice in real-world user traffic.
• True internal belief revision: First-person self-attribution (Appendix D.8) demonstrates textual source confusion, but does not provide mechanistic proof of internal representation updates.
• Unbiased ground truth of LLM judges: Disagree and Compliance classifications rely on automated model judges without multi-institution double-blind human annotation calibration.
• Independent reproducibility of public artifacts: The GitHub repository announced in the paper is not publicly accessible at the time of verification, leaving generation prompts and internal harnesses unverified externally.

Bloss0m engineering synthesis

• Separation of benchmark diagnostics and product objectives: High-adversity benchmarks stress-test vulnerability, but production systems must avoid optimizing solely for user pushback, which causes destructive false refusals.
• Shift toward structured verification architectures: Prompt engineering alone is insufficient; production reliability requires a four-step verification loop combining goal restatement, read-only state checks, transparent discrepancy demonstration, and constructive alternatives.

Artifacts and reproducibility

The paper states that its evaluation code and datasets will be released at https://github.com/google-deepmind/xyeval. As of the checked date (2026-09-24), this repository is not publicly accessible. The experimental findings presented in this article reflect results reported by the paper’s authors and have not been independently rerun across the full benchmark suite.

Independent reproduction of the evaluation pipeline would require several core prerequisites:

1. Benchmark environments: Access to containerized Docker environments for TauBench, Terminal-Bench 2.0, SWE-bench Verified, SWE-bench Pro, HLE, and MCP-Atlas.
2. Specialized harnesses: SWE-bench evaluations rely on DeepMind’s internal coding harness, requiring external teams to implement custom tool-interaction wrappers.
3. Suggestion generation pipelines: Reconstructing prompt decomposition pipelines (Gemini 3.1 Pro) and golden-solution-guided distractor generation scripts (Gemini 3.5 Flash).
4. Judge dependencies: Evaluation of HLE, MCP-Atlas, and trajectory labels depends on Gemini 3.5 Flash judge prompts, where API version shifts could introduce measurement variance.

Figures 1, 2, 7, and 10 are reproduced from arXiv v1 under its declared CC BY 4.0 license. Local assets were re-encoded as high-quality WebP images without cropping or content alterations; captions include figure identifiers, section anchors, licensing notices, and source links. The article cover is a separately created Evidence Atlas conceptual illustration visualizing multi-path verification and goal divergence, rather than empirical benchmark data.

Bloss0m engineering judgment and when not to use it

When not to use: boundaries and misapplications

XYEval offers an essential diagnostic framework for evaluating agent susceptibility to misleading guidance. However, teams should avoid the following misapplications in production:

• Do not extrapolate benchmark drop rates to production failure rates: Distractors are generated adversarially with access to golden solutions, representing a far higher threat level than inadvertent user errors in ordinary workflows.
• Do not treat scores across suites as interchangeable abilities: TauBench measures conversational reward, SWE-bench and Terminal-Bench evaluate test suites, HLE scores academic Q&A, and MCP-Atlas measures factual coverage. Their percentage drops cannot be directly equated.
• Do not deploy Golden defense as a production solution: Warning an agent about the exact distractor is an oracle ceiling that has no counterpart in real-world deployment.
• Do not rely exclusively on automated LLM judges: Judge prompts have domain-specific blind spots; audit logging should incorporate deterministic tool-call monitoring alongside model assessments.
• Do not treat user refusal as a standalone virtue: Indiscriminate pushback degrades collaboration and frustrates users attempting legitimate operations.

Bloss0m engineering synthesis: production practices for resisting bad advice

Drawing from XYEval’s empirical evidence, Bloss0m recommends the following architectural patterns for production agents:

1. Establish localized adversarial regression suites: Extract historical support tickets, operational incidents, or code review logs featuring common misdiagnoses (e.g., users diagnosing certificate expiration as network downtime). Build localized regression suites featuring control, bad-advice, and ambiguous-advice variants.
2. Implement a four-step verification workflow:
   • Step 1: Explicit goal restatement—Articulate the core business goal Y to the user prior to execution.
   • Step 2: Read-only prerequisite verification—Use read-only tools to inspect environment state and policy rules, determining whether suggested action X satisfies operational preconditions.
   • Step 3: Transparent discrepancy demonstration—If X conflicts with observed system state or goal Y, cite specific log evidence or policy clauses explaining why X is unfeasible.
   • Step 4: Propose constructive alternative paths—Offer an alternative, compliant course of action that directly addresses Y for user confirmation.
3. Enforce structural guardrails for side-effecting operations: High-consequence operations (refunds, database migrations, server reboots, credential modifications) must not rely solely on system prompt constraints. They require dry-run previews, explicit two-phase confirmations, and automated escalation to human supervisors for irreversible actions.

Related reading paths:

• For rigorous methodologies on selecting representative benchmark subsets and evaluating trajectory reliability, see Trajectory-Aware Benchmark Subset Selection.
• For establishing deterministic authorization boundaries around high-risk payment tools, see APort Vault: Payment Agent Authorization.

Three things to remember

1. Technical idea: XYEval preserves the task environment and scoring oracle while injecting plausible but incorrect suggestions into instructions, transforming conversational derailment into a quantifiable benchmark perturbation.
2. Evidence: Frontier models drop substantially across six benchmark suites (reaching −46.7% relative drop for Gemini 3.1 Pro on Terminal-Bench); persistent pedantic users worsen declines in dialogue tasks, and trace analysis reveals that 92.1% of internal recognitions fail to translate into external disagreement.
3. Adoption boundary: Suggestions are adversarially crafted with reference answers and cannot be treated as ordinary conversational incident rates; generic prompt defenses offer limited protection, necessitating structured goal restatement, prerequisite verification, and constructive alternative workflows.

Primary sources

• arXiv v1 abstract page (arXiv:2609.23939v1, 2026-09-20)
• arXiv v1 full HTML paper, figures, and Appendices A–E
• Code repository referenced in paper (unverified public access at check date)
• Creative Commons Attribution 4.0 International (CC BY 4.0) License