ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but inherently sequential decoding leads to substantial latency, especially on complex tasks. Recent work on adaptive parallel reasoning aims to improve inference efficiency by decomposing the problem-solving process into concurrent reasoning threads when beneficial. However, existing methods on realistic tasks are either limited to supervised behavior cloning or exhibit significant accuracy drops compared to widely-used sequential long chain-of-thought (CoT) baselines. Moreover, many require customized inference engines, complicating deployment. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that achieves accuracy on par with popular sequential reasoning models of comparable size while significantly reducing inference latency. ThreadWeaver's performance stems from three key innovations: 1) a two-stage parallel trajectory generator that produces large-scale, high-quality CoT data with parallel annotations for supervised fine-tuning; 2) a trie-based training-inference co-design that enables parallel reasoning on any off-the-shelf autoregressive inference engine without modifying position embeddings or KV caches; and 3) a parallelization-aware reinforcement learning framework that teaches the model to balance accuracy with effective parallelization. Across six challenging mathematical reasoning benchmarks, ThreadWeaver trained atop Qwen3-8B achieves accuracy comparable to cutting-edge sequential reasoning models (71.9% on average and 79.9% on AIME24) while delivering up to 1.53x average speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.
6938e51ddfc35938ba129fb2
Long Lian
claimed_verified
2025-12-10T08:56:15.315Z
63797c273f575acc2f6893c0
/avatars/32d7a6a8881c8c4d80a097b732ed24b6.svg
Long(Tony) Lian
longlian
user
longlian
6938e51ddfc35938ba129fb3
Sida Wang
6938e51ddfc35938ba129fb4
Felix Juefei-Xu
claimed_verified
2025-12-10T13:07:56.324Z
6417cf37dce1e4c0229f17b1
https://cdn-avatars.huggingface.co/v1/production/uploads/6417cf37dce1e4c0229f17b1/7h-ZCB5f4wif7TsnF-B1M.jpeg
Felix Xu
katanaxu
user
katanaxu
6938e51ddfc35938ba129fb5
Tsu-Jui Fu
6938e51ddfc35938ba129fb6
Xiuyu Li
6938e51ddfc35938ba129fb7
Adam Yala
6938e51ddfc35938ba129fb8
Trevor Darrell
6938e51ddfc35938ba129fb9
Alane Suhr
6938e51ddfc35938ba129fba
Yuandong Tian
6938e51ddfc35938ba129fbb
Xi Victoria Lin
claimed_verified
2025-12-31T20:58:12.918Z
61df349e34b589fbaf2e93c4
/avatars/9dd7776d3b610d78e40c2e178ab2f112.svg
Victoria Lin
VictoriaLinML
user
VictoriaLinML
ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but inherently sequential decoding leads to substantial latency, especially on complex tasks. Recent work on adaptive parallel reasoning aims to improve inference efficiency by decomposing the problem-solving process into concurrent reasoning threads when beneficial. However, existing methods on realistic tasks are either limited to supervised behavior cloning or exhibit significant accuracy drops compared to widely-used sequential long chain-of-thought (CoT) baselines. Moreover, many require customized inference engines, complicating deployment. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that achieves accuracy on par with popular sequential reasoning models of comparable size while significantly reducing inference latency. ThreadWeaver's performance stems from three key innovations: 1) a two-stage parallel trajectory generator that produces large-scale, high-quality CoT data with parallel annotations for supervised fine-tuning; 2) a trie-based training-inference co-design that enables parallel reasoning on any off-the-shelf autoregressive inference engine without modifying position embeddings or KV caches; and 3) a parallelization-aware reinforcement learning framework that teaches the model to balance accuracy with effective parallelization. Across six challenging mathematical reasoning benchmarks, ThreadWeaver trained atop Qwen3-8B achieves accuracy comparable to cutting-edge sequential reasoning models (71.9% on average and 79.9% on AIME24) while delivering up to 1.53x average speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.
ThreadWeaver, a framework for adaptive parallel reasoning, achieves accuracy comparable to sequential models while reducing inference latency through parallel trajectory generation, trie-based training-inference co-design, and parallelization-aware reinforcement learning.
Large Language Models sequential decoding adaptive parallel reasoning long chain-of-thought CoT ThreadWeaver parallel trajectory generator trie-based training-inference co-design parallelization-aware reinforcement learning Qwen3-8B AIME24