Reflexion deep reading

Series · 1 posts

← Back to Paper Reading

  • Reflexion: Write Verbal Reflections into Memory — Do Not Mistake Retries for Weight Learning

    Intermediate Agent runtime, safety, and evaluation
    Reflexion deep reading: Part 1 , Note: Aug 27, 2026 , Paper: 2023 , AI Engineering

    A deep read of Shinn et al., NeurIPS 2023: frozen weights plus linguistic feedback stored in episodic memory for across-trial verbal credit assignment. HumanEval pass@1 91.0 vs GPT-4 80.1 is a programming setup with self-tests and retries; WebShop and MBPP mark the boundary.

    Understand it in 90 seconds
    Problem
    Language agents can already act in external environments, but learning from trial-and-error usually means expensive sample-heavy RL with weight updates, while few-shot in-context teaching barely stores interpretable experience across episodes.
    Core insight
    Keep weights frozen. Amplify binary or scalar feedback into a verbal reflection, append it to an episodic memory buffer, and condition the next trial on that text. The changed control point is across-trial verbal credit assignment, not a parameter gradient.
    Strongest evidence
    HumanEval (PY) Reflexion pass@1 91.0 versus GPT-4 single-sample 80.1 (Table 1); ALFWorld heuristic setting solves 130/134 (Section 4.1); HotPotQA reports about +20% over strong baselines (Section 4 lead-in). Rust ablation: full Reflexion 0.68; omitting reflection or tests falls to 0.60 / 0.52 (Table 3).
    Main boundary
    Needs a usable evaluation signal; reflections can be wrong; extra trials cost compute; memory is a sliding window (typically 1–3), not enterprise governance. WebShop barely improves (Figure 6); MBPP (PY) drops to 77.1. This is not weight learning and not a deployable runtime.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact