Base models know how to reason, thinking models learn when

Conference item

Why do thinking language models like DeepSeek R1 outperform their base counterparts? Despite consistent performance gains, it remains unclear to what extent thinking models learn entirely new reasoning capabilities or repurpose pre-existing base model ones. In this work, we propose a hybrid model where we activate reasoning mechanisms in base models at the right time to elicit thinking-modellevel reasoning chains, implying that thinking models exploit already existing capabilities. To ground our analysis, we introduce an unsupervised, bottom-up approach for uncovering human-interpretable reasoning behaviors in thinking models. This approach provides an unbiased method to discover reasoning behaviors without imposing manual or LLM-derived assumptions. Across three base and four thinking models, using GSM8K and MATH500, our hybrid model recovers up to 91% of the performance gap to thinking models without any weight updates while steering only 12% of tokens. Concretely, our empirical setup provides a simple, causal way to test the effectiveness of existing reasoning mechanisms in base models by invoking them directly and measuring the resulting task performance. More broadly, these results reframe our understanding of how thinking models are trained: pre-training is when models acquire most of their reasoning mechanisms, and post-training teaches efficient deployment of these mechanisms at the right time, enabling efficient use of their inference-time compute.

Authors: Venhoff, C; Arcuschin, I; Torr, P; Conmy, A; Nanda, N

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: OpenReview
• Publication date: 2025-09-18
• Acceptance date: 2025-09-30
• Event title: 39th Conference on Neural Information Processing Systems (NeurIPS 2025)
• Event location: San Diego, California, USA and Mexico City, Mexico
• Event website: https://neurips.cc/Conferences/2025
• Event start date: 2025-11-30
• Event end date: 2025-12-07
• Language: English