A Generalist Dynamics Model for Control
We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.
6466fe5f14e059dde8bc6bb1
Ingmar Schubert
6466fe5f14e059dde8bc6bb2
Jingwei Zhang
admin_assigned
2023-05-26T09:06:32.879Z
63ff47cc12c51862e5d9e2ce
/avatars/fd08feaee58cebb3c4541eac8fe6bed6.svg
Jingwei Zhang
allanz
user
allanz
6466fe5f14e059dde8bc6bb3
Jake Bruce
6466fe5f14e059dde8bc6bb4
Sarah Bechtle
6466fe5f14e059dde8bc6bb5
Emilio Parisotto
6466fe5f14e059dde8bc6bb6
Martin Riedmiller
6466fe5f14e059dde8bc6bb7
Jost Tobias Springenberg
6466fe5f14e059dde8bc6bb8
Arunkumar Byravan
6466fe5f14e059dde8bc6bb9
Leonard Hasenclever
6466fe5f14e059dde8bc6bba
Nicolas Heess
A Generalist Dynamics Model for Control
We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.
Transformer sequence models as dynamics models (TDMs) excel in single-environment learning, generalize well to unseen environments, and outperform policy generalization in control tasks.
transformer sequence models dynamics models DeepMind control suite few-shot learning zero-shot learning generalist model policy generalization foundation model of control