How large language models encode theory-of-mind: a study on sparse parameter patterns by Yuheng Wu & Wentao Guo & Zirui Liu & Heng Ji & Zhaozhuo Xu & Denghui Zhang instant download

How large language models encode theory-of-mind: a study on sparse parameter patterns by Yuheng Wu & Wentao Guo & Zirui Liu & Heng Ji & Zhaozhuo Xu & Denghui Zhang instant download

This paper investigates the emergence of Theory-of-Mind (ToM) capabilities in large language models(LLMs) from a mechanistic perspective, focusing on the role of extremely sparse parameter patterns.1234567890():,;1234567890():,;We introduce a novel method to identify ToM-sensitive parameters and reveal that perturbing as littleas 0.001% of these parameters significantly degrades ToM performance while also impairingcontextual localization and language understanding. To understand this effect, we analyze theirinteractions with core architectural components of LLMs. Our findings demonstrate that thesesensitive parameters are closely linked to the positional encoding module, particularly in models usingRotary Position Embedding (RoPE), where perturbations disrupt dominant frequency activationscritical for contextual processing. Furthermore, we show that perturbing ToM-sensitive parametersaffects LLMs’ attention mechanism by modulating the angle between queries and keys underpositional encoding. These insights provide a deeper understanding of how LLMs acquire socialreasoning abilities, bridging AI interpretability with cognitive science.