Conference item
Goal-conditioned end-to-end visuomotor control for versatile skill primitives
- Abstract:
- Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick- and-place from raw images. Conditioning VMC on desired goal states is a promising way of achieving versatile skill primitives. However, common conditioning schemes either rely on task-specific fine tuning - e.g. using one-shot imitation learning (IL) - or on sampling approaches using a forward model of scene dynamics i.e. model-predictive control (MPC), leaving deployability and planning horizon severely limited. In this paper we propose a conditioning scheme which avoids these pitfalls by learning the controller and its conditioning in an end-to-end manner. Our model predicts complex action sequences based directly on a dynamic image representation of the robot motion and the distance to a given target observation. In contrast to related works, this enables our approach to efficiently perform complex manipulation tasks from raw image observations without predefined control primitives or test time demonstrations. We report significant improvements in task success over representative MPC and IL baselines. We also demonstrate our model's generalisation capabilities in challenging, unseen tasks featuring visual noise, cluttered scenes and unseen object geometries.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Accepted manuscript, pdf, 4.4MB, Terms of use)
-
- Publisher copy:
- 10.1109/ICRA48506.2021.9560752
Authors
- Publisher:
- IEEE
- Host title:
- 2021 IEEE International Conference on Robotics and Automation (ICRA)
- Pages:
- 1319-1325
- Publication date:
- 2021-10-18
- Acceptance date:
- 2021-02-28
- Event title:
- 2021 International Conference on Robotics and Automation (ICRA 2021)
- Event location:
- Xi'an, China
- Event website:
- https://www.ieee-ras.org/component/rseventspro/event/1920-icra-2021
- Event start date:
- 2021-05-30
- Event end date:
- 2021-06-05
- DOI:
- EISSN:
-
2577-087X
- ISSN:
-
1050-4729
- EISBN:
- 978-1-7281-9077-8
- ISBN:
- 978-1-7281-9078-5
- Language:
-
English
- Keywords:
- Pubs id:
-
1233315
- Local pid:
-
pubs:1233315
- Deposit date:
-
2022-01-26
Terms of use
- Copyright holder:
- IEEE
- Copyright date:
- 2021
- Rights statement:
- © 2021 IEEE
- Notes:
- This is the accepted manuscript version of the paper. The final version is available online from IEEE at https://doi.org/10.1109/ICRA48506.2021.9560752
If you are the owner of this record, you can report an update to it here: Report update to this record