The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.
69d9152b9e8a77977ac8a12a
Rishab Balasubramanian
69d9152b9e8a77977ac8a12b
Pin-Jie Lin
69d9152b9e8a77977ac8a12c
Rituraj Sharma
69d9152b9e8a77977ac8a12d
Anjie Fang
69d9152b9e8a77977ac8a12e
Fardin Abdi
69d9152b9e8a77977ac8a12f
Viktor Rozgic
69d9152b9e8a77977ac8a130
Zheng Du
69d9152b9e8a77977ac8a131
Mohit Bansal
69d9152b9e8a77977ac8a132
Tu Vu
The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.
Post-trained model capabilities can be transferred across different model scales through linear alignment of latent subspace directions without requiring retraining.
Master Key Hypothesis capability direction latent subspace linear alignment UNLOCK source variants target model Chain-of-Thought mathematical reasoning pre-training post-training inference time output distribution