"Act Like a 5th Grader" is Not Enough: Bounding Knowledge in LLM-Based User Simulators
Research paper by Krisztian Balog and collaborators. arXiv record dated 2026-08-30. Research area: Language technology.
Resource information
- Authors
- Krisztian Balog, Arild Michel Bakken
- Research area
- Language technology
- Application domain
- General AI research
- AI methods
- Large language models
- Research topics
- cs.CL, cs.AI
- arXiv record date
- 2026-08-30
- Publication
- arXiv
- Upstream last modified
- Sep 1, 2026
- License
- CC BY 4.0
- Last record update
- Nov 4, 2026
Abstract
Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of over 71,000 reading comprehension responses from 2,359 primary-school students (grades 4--6), we demonstrate that standard persona prompting yields near-perfect, deterministic performance, failing to capture the natural variance of developing readers. To address this, we introduce the Cognitively Bounded User Simulator (CBUS), an architectural framework that explicitly models the restricted working memory of young readers through an episodic bottleneck. Within this framework, we formalize two distinct test-taking strategies to emulate different reading behaviors. Our evaluation shows that explicitly modeling cognitive bounds significantly narrows the simulation gap across multiple LLM backbones, demonstrating that enforcing architectural constraints is more effective for high-fidelity simulation than simply scaling raw model capabilities.