Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
Recent advances in visual generation have increasingly explored the integration of reasoning capabilities. They incorporate textual reasoning, i.e., think, either before (as pre-planning) or after (as post-refinement) the generation process, yet they lack on-the-fly multimodal interaction during the generation itself. In this preliminary study, we introduce Thinking-while-Generating (TwiG), the first interleaved framework that enables co-evolving textual reasoning throughout the visual generation process. As visual content is progressively generating, textual reasoning is interleaved to both guide upcoming local regions and reflect on previously synthesized ones. This dynamic interplay produces more context-aware and semantically rich visual outputs. To unveil the potential of this framework, we investigate three candidate strategies, zero-shot prompting, supervised fine-tuning (SFT) on our curated TwiG-50K dataset, and reinforcement learning (RL) via a customized TwiG-GRPO strategy, each offering unique insights into the dynamics of interleaved reasoning. We hope this work inspires further research into interleaving textual reasoning for enhanced visual generation. Code will be released at: https://github.com/ZiyuGuo99/Thinking-while-Generating.
691fd939cce0eb9b6387afac
Ziyu Guo
691fd939cce0eb9b6387afad
Renrui Zhang
admin_assigned
2025-11-21T10:42:45.049Z
645b8b2687c79b6ec0bb3b7a
/avatars/00a9db32a42dc950112bf2593bb109cb.svg
Renrui
ZrrSkywalker
user
ZrrSkywalker
691fd939cce0eb9b6387afae
Hongyu Li
claimed_verified
2025-11-24T12:27:03.148Z
66fd43e80cde4879f9aeca01
/avatars/7bc9afa5e023e00820333e8d18dc4bc5.svg
Hongyu Li
appletea2333
user
appletea2333
691fd939cce0eb9b6387afaf
Manyuan Zhang
691fd939cce0eb9b6387afb0
Xinyan Chen
691fd939cce0eb9b6387afb1
Sifan Wang
691fd939cce0eb9b6387afb2
Yan Feng
691fd939cce0eb9b6387afb3
Peng Pei
691fd939cce0eb9b6387afb4
Pheng-Ann Heng
Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
Recent advances in visual generation have increasingly explored the integration of reasoning capabilities. They incorporate textual reasoning, i.e., think, either before (as pre-planning) or after (as post-refinement) the generation process, yet they lack on-the-fly multimodal interaction during the generation itself. In this preliminary study, we introduce Thinking-while-Generating (TwiG), the first interleaved framework that enables co-evolving textual reasoning throughout the visual generation process. As visual content is progressively generating, textual reasoning is interleaved to both guide upcoming local regions and reflect on previously synthesized ones. This dynamic interplay produces more context-aware and semantically rich visual outputs. To unveil the potential of this framework, we investigate three candidate strategies, zero-shot prompting, supervised fine-tuning (SFT) on our curated TwiG-50K dataset, and reinforcement learning (RL) via a customized TwiG-GRPO strategy, each offering unique insights into the dynamics of interleaved reasoning. We hope this work inspires further research into interleaving textual reasoning for enhanced visual generation. Code will be released at: https://github.com/ZiyuGuo99/Thinking-while-Generating.
A new framework, Thinking-while-Generating (TwiG), integrates interleaved textual reasoning during visual generation, enhancing context-awareness and semantic richness through zero-shot prompting, supervised fine-tuning, and reinforcement learning.
Thinking-while-Generating TwiG textual reasoning pre-planning post-refinement visual generation zero-shot prompting supervised fine-tuning reinforcement learning TwiG-50K dataset TwiG-GRPO