OmniX: From Unified Panoramic Generation and Perception to
  Graphics-Ready 3D Scenes
There are two prevalent ways to constructing 3D scenes: procedural generation
and 2D lifting. Among them, panorama-based 2D lifting has emerged as a
promising technique, leveraging powerful 2D generative priors to produce
immersive, realistic, and diverse 3D environments. In this work, we advance
this technique to generate graphics-ready 3D scenes suitable for physically
based rendering (PBR), relighting, and simulation. Our key insight is to
repurpose 2D generative models for panoramic perception of geometry, textures,
and PBR materials. Unlike existing 2D lifting approaches that emphasize
appearance generation and ignore the perception of intrinsic properties, we
present OmniX, a versatile and unified framework. Based on a lightweight and
efficient cross-modal adapter structure, OmniX reuses 2D generative priors for
a broad range of panoramic vision tasks, including panoramic perception,
generation, and completion. Furthermore, we construct a large-scale synthetic
panorama dataset containing high-quality multimodal panoramas from diverse
indoor and outdoor scenes. Extensive experiments demonstrate the effectiveness
of our model in panoramic visual perception and graphics-ready 3D scene
generation, opening new possibilities for immersive and physically realistic
virtual world generation.
6904193e2c556835fa67f063
Yukun Huang
claimed_verified
2025-10-31T14:27:11.118Z
638ee900ee7e45e0474a5712
https://cdn-avatars.huggingface.co/v1/production/uploads/638ee900ee7e45e0474a5712/KLli_eCbWwffKR7oLDmV3.jpeg
Yukun Huang
KevinHuang
user
KevinHuang
6904193e2c556835fa67f064
Jiwen Yu
6904193e2c556835fa67f065
Yanning Zhou
6904193e2c556835fa67f066
Jianan Wang
6904193e2c556835fa67f067
Xintao Wang
6904193e2c556835fa67f068
Pengfei Wan
6904193e2c556835fa67f069
Xihui Liu
OmniX: From Unified Panoramic Generation and Perception to
  Graphics-Ready 3D Scenes
There are two prevalent ways to constructing 3D scenes: procedural generation
and 2D lifting. Among them, panorama-based 2D lifting has emerged as a
promising technique, leveraging powerful 2D generative priors to produce
immersive, realistic, and diverse 3D environments. In this work, we advance
this technique to generate graphics-ready 3D scenes suitable for physically
based rendering (PBR), relighting, and simulation. Our key insight is to
repurpose 2D generative models for panoramic perception of geometry, textures,
and PBR materials. Unlike existing 2D lifting approaches that emphasize
appearance generation and ignore the perception of intrinsic properties, we
present OmniX, a versatile and unified framework. Based on a lightweight and
efficient cross-modal adapter structure, OmniX reuses 2D generative priors for
a broad range of panoramic vision tasks, including panoramic perception,
generation, and completion. Furthermore, we construct a large-scale synthetic
panorama dataset containing high-quality multimodal panoramas from diverse
indoor and outdoor scenes. Extensive experiments demonstrate the effectiveness
of our model in panoramic visual perception and graphics-ready 3D scene
generation, opening new possibilities for immersive and physically realistic
virtual world generation.
OmniX repurposes 2D generative models for panoramic perception and generation, creating graphics-ready 3D scenes suitable for PBR, relighting, and simulation.
procedural generation 2D lifting panorama-based 2D lifting 2D generative priors immersive 3D environments physically based rendering PBR OmniX cross-modal adapter structure panoramic perception panoramic generation panoramic completion large-scale synthetic panorama dataset immersive virtual world generation