Omni-Weather: Unified Multimodal Foundation Model for Weather Generation and Understanding
Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.
695217d6746a34b55dd548ee
Zhiwang Zhou
695217d6746a34b55dd548ef
Yuandong Pu
claimed_verified
2025-12-29T14:22:54.030Z
625d5b9f0bec31f086e04cd9
https://cdn-avatars.huggingface.co/v1/production/uploads/1650285458447-noauth.jpeg
YuandongPu
Andrew613
user
Andrew613
695217d6746a34b55dd548f0
Xuming He
695217d6746a34b55dd548f1
Yidi Liu
695217d6746a34b55dd548f2
Yixin Chen
695217d6746a34b55dd548f3
Junchao Gong
695217d6746a34b55dd548f4
Xiang Zhuang
695217d6746a34b55dd548f5
Wanghan Xu
695217d6746a34b55dd548f6
Qinglong Cao
695217d6746a34b55dd548f7
Shixiang Tang
695217d6746a34b55dd548f8
Yihao Liu
695217d6746a34b55dd548f9
Wenlong Zhang
695217d6746a34b55dd548fa
Lei Bai
Omni-Weather: Unified Multimodal Foundation Model for Weather Generation and Understanding
Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.
Omni-Weather is a multimodal foundation model that integrates weather generation and understanding using a shared self-attention mechanism and a Chain-of-Thought dataset to enable interpretable, high-quality outputs.
multimodal foundation model radar encoder shared self-attention mechanism Chain-of-Thought dataset