Adaptive Transfer of Representational Signatures for LLM Reliability Detection

Resource information

Authors

Yongtong Li, Yixuan Florence Wu, Jiawei Zhang, Naichen Shi

Research area

Language technology

Application domain

General AI research

AI methods

Large language models

Research topics

cs.CL, cs.LG

arXiv record date

2026-10-24

Publication

arXiv

Upstream last modified

Oct 27, 2026

Last record update

Nov 4, 2026

Abstract

Large language models (LLMs) are reported to be susceptible to reliability issues including hallucinations and reasoning errors. Existing quality detectors can predict unreliable outputs based on internal representations with decent accuracies. However, training these detectors often requires substantial domain-specific labeled data, which limits their applicability to new domains. We propose to address this challenge by transferring representational signatures across related domains. More specifically, we build a statistical framework that first extracts low-dimensional signatures for each domain, then uses cross-fitting to learn composition weights of domain-specific predictors. The weights adapt to each source domain's predictive relevance to the target domain. Our asymptotic analysis of a least-squares analogue characterizes the limiting prediction risk and aggregation weights in terms of domain discrepancies and representation bias. Experiments across multiple hallucination datasets and open-weight LLMs show that, with only 25 labeled questions in the new domain, adaptive transfer improves hallucination-detection AUROC by 6.8%. For reasoning-step prediction, it improves AUROC by 8.4%.