NitroGen: An Open Foundation Model for Generalist Gaming Agents
We introduce NitroGen, a vision-action foundation model for generalist gaming agents that is trained on 40,000 hours of gameplay videos across more than 1,000 games. We incorporate three key ingredients: 1) an internet-scale video-action dataset constructed by automatically extracting player actions from publicly available gameplay videos, 2) a multi-game benchmark environment that can measure cross-game generalization, and 3) a unified vision-action model trained with large-scale behavior cloning. NitroGen exhibits strong competence across diverse domains, including combat encounters in 3D action games, high-precision control in 2D platformers, and exploration in procedurally generated worlds. It transfers effectively to unseen games, achieving up to 52% relative improvement in task success rates over models trained from scratch. We release the dataset, evaluation suite, and model weights to advance research on generalist embodied agents.
695dce13c03d6d81e4399ed0
Loïc Magne
695dce13c03d6d81e4399ed1
Anas Awadalla
695dce13c03d6d81e4399ed2
Guanzhi Wang
695dce13c03d6d81e4399ed3
Yinzhen Xu
695dce13c03d6d81e4399ed4
Joshua Belofsky
695dce13c03d6d81e4399ed5
Fengyuan Hu
695dce13c03d6d81e4399ed6
Joohwan Kim
695dce13c03d6d81e4399ed7
Ludwig Schmidt
695dce13c03d6d81e4399ed8
Georgia Gkioxari
695dce13c03d6d81e4399ed9
Jan Kautz
695dce13c03d6d81e4399eda
Yisong Yue
695dce13c03d6d81e4399edb
Yejin Choi
695dce13c03d6d81e4399edc
Yuke Zhu
695dce13c03d6d81e4399edd
Linxi "Jim" Fan
NitroGen: An Open Foundation Model for Generalist Gaming Agents
We introduce NitroGen, a vision-action foundation model for generalist gaming agents that is trained on 40,000 hours of gameplay videos across more than 1,000 games. We incorporate three key ingredients: 1) an internet-scale video-action dataset constructed by automatically extracting player actions from publicly available gameplay videos, 2) a multi-game benchmark environment that can measure cross-game generalization, and 3) a unified vision-action model trained with large-scale behavior cloning. NitroGen exhibits strong competence across diverse domains, including combat encounters in 3D action games, high-precision control in 2D platformers, and exploration in procedurally generated worlds. It transfers effectively to unseen games, achieving up to 52% relative improvement in task success rates over models trained from scratch. We release the dataset, evaluation suite, and model weights to advance research on generalist embodied agents.
NitroGen is a vision-action foundation model trained on extensive gameplay data that demonstrates strong cross-game generalization and effective transfer learning capabilities.
vision-action foundation model large-scale behavior cloning cross-game generalization transfer learning embodied agents