Multi-Turn

This section tracks papers on multi-turn behavior in LLMs and agents: when capability drops across turns, what mechanisms may explain it, and how to diagnose or recover reliability.

Papers

  • LLMs Get Lost In Multi-Turn Conversation Reports large-scale simulated comparisons of single-turn vs multi-turn performance across six generation tasks. Finds an average 39% drop in multi-turn settings, with degradation driven more by increased unreliability than by pure aptitude loss.
  • Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap Introduces Found in Conversation (FiC), a self-distillation framework that transfers strong single-turn behavior into weak multi-turn behavior via asymmetric task views. Across multiple model families and 3B-14B scales, it reports recovery of at least 92% of single-turn performance and up to 100% on two Llama backbones.
  • When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction Proposes a channel-transition account where attention access to goal-defining tokens collapses over turns, while some goal information can persist in residual representations. Introduces Goal Accessibility Ratio (GAR) and reports architecture-dependent failure modes, including strong recall collapse under forced attention-channel closure.
  • Lost in the Noise: How Reasoning Models Fail with Contextual Distractors Presents NoisyBench, evaluating robustness to random documents, irrelevant chat history, and hard negative distractors across RAG, reasoning, alignment, and tool-use tasks. Reports up to 80% performance drops under contextual noise, and shows rationale-aware reward training (RARE) improves resilience compared with standard prompting and tuning strategies.
  • Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures Argues that outcome-only evaluation hides where failures originate, and introduces an interaction-centric taxonomy that localizes failures by component interaction and repair side. Organizes 41 failure modes and reports reproducible labeling structure with independent judges, including agreement up to Cohen’s $\kappa = 0.76$ against human labels.