SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page here: https://nvlabs.github.io/sage.
698bf1aa6052d3bed9630a2d
Hongchi Xia
698bf1aa6052d3bed9630a2e
Xuan Li
698bf1aa6052d3bed9630a2f
Zhaoshuo Li
698bf1aa6052d3bed9630a30
Qianli Ma
698bf1aa6052d3bed9630a31
Jiashu Xu
698bf1aa6052d3bed9630a32
Ming-Yu Liu
698bf1aa6052d3bed9630a33
Yin Cui
698bf1aa6052d3bed9630a34
Tsung-Yi Lin
698bf1aa6052d3bed9630a35
Wei-Chiu Ma
698bf1aa6052d3bed9630a36
Shenlong Wang
698bf1aa6052d3bed9630a37
Shuran Song
698bf1aa6052d3bed9630a38
Fangyin Wei
SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page here: https://nvlabs.github.io/sage.
SAGE is an agentic framework that automatically generates simulation-ready 3D environments for embodied AI by combining layout and object composition generators with evaluative critics for semantic plausibility and physical stability.
embodied agents simulation-ready environments scene-generation systems agentic framework layout generation object composition semantic plausibility visual realism physical stability iterative reasoning adaptive tool selection policy training embodied AI