Link Repo
BibTeX Citation 
@misc{moore_delusioneval_2026,
  title = {DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots},
  author = {Moore, Jared and Mock, Andrea and Mai, Yifan and Anthis, Jacy Reese and Louie, Ryan and Agnew, William and Mehta, Ashish and Klyman, Kevin and Liang, Percy and Haber, Nick and Lin, Eric and Ong, Desmond C.},
  year = {2026},
  url = {https://arxiv.org/abs/2608.05004},
  note = {Preprint},
}

Executive Summary

Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs), including “delusional spirals” in which concerning human and LLM behaviors reinforce each other over time. With growing public use of LLM-powered chatbots, there is an urgent need to build evaluations grounded in real-world episodes of psychological harm experienced by users.

DelusionEval is an evaluation protocol to assess how LLMs respond to messages with prior context from real transcripts of people who report psychological harm from LLMs. We prompt each model with 589 unique conversation histories from 18 participants, comprising 12,591 messages.

We show that, while some delusion-linked behaviors have decreased with larger and more recent LLMs, rates remain high. Across model families, scaling effects are uneven and sometimes reverse sign, and we do not see monotonic improvements with larger models, newer models, or higher test-time reasoning across all categories.

Evaluation Protocol

Our evaluation replays real conversation histories from user transcripts. We split each transcript into overlapping windows of up to 20 messages. Within each window, we create one sample per user turn. For each sample, the evaluated model receives the exact transcript prefix up to that user message and then generates a reply.

The next sample in the same window is built from the original transcript, not from the evaluated model’s prior output. This gives a sequence of counterfactual single-turn evaluations over the same multi-turn context.

An example DelusionEval window replay with original and evaluated model responses.
An example of DelusionEval. We take an existing conversational window from a user's transcript and evaluate a model by successively prompting it with chains of the original context.

Main Results

Every evaluated LLM exhibited lower prevalence than the original-transcript baseline on delusional, sycophantic, relationship, and facilitates-harm categories. Discourages-harm behavior varied by model.

At the same time, all evaluated LLMs still exhibited some level of the behaviors we tested for. Only a small number of model-code pairs were exactly zero prevalence, mostly concentrated in facilitates-harm codes.

Comparison of model families across behavior categories in DelusionEval.
Model-family comparison across GPT, Claude, Gemini, and Qwen. Bars show prevalence by category with 95% bootstrap confidence intervals.

Across model families, scaling effects are uneven and sometimes reverse sign. We do not see monotonic improvements with larger models, newer models, or higher test-time reasoning across all categories.

A key comparability finding is that rerun gpt-4o shows substantially lower prevalence than the original-transcript baseline, even though most of the baseline conversations were produced with gpt-4o. This is expected because the evaluation set specifically selects harmful conversations. The remaining discrepancy may also reflect additional system-level factors in original deployments that are not captured by our replay protocol, such as system prompts, additional context, cross-conversation memory, or snapshot variants. As a result, this evaluation may underestimate the prevalence of delusion-linked behaviors in some real chatbot settings.

Context Length Effects

Requested context depth changes the prevalence of several behaviors. For gpt-5.4, deeper requested context is associated with higher prevalence of delusional behavior and lower prevalence of bot-discourages-violence.

Delusional category prevalence versus requested context length in DelusionEval.
Category-level context effect for delusional behavior in gpt-5.4. Points show prevalence versus context length with 95% bootstrap confidence intervals.

These shifts are not explained solely by accumulating prior-turn content, and depth effects persist over and above accumulated assistant content.

Discussion

We caution that strong performance on this evaluation should not be taken as evidence that a model is clinically safe. Our coverage is not comprehensive, as it focuses on sycophancy and delusion and does not cover other kinds of mental health or psychosocial harms.

Our evaluation was constructed from conversational transcripts from only 18 users, relies on static evaluation, and does not make use of memory systems that may be used in deployed chatbot systems (for example retrieval-augmented context, summarization, or cross-session state).

While some delusion-linked behaviors have decreased with larger and more recent LLMs, rates remain high, especially when considered across the millions of people globally who interact with LLMs. We encourage further empirical work that engages critically with the mental health impacts of LLMs and the development of strategies to mitigate harm.