Bayesian reinforcement learning for single-episode missions in partially unknown environments

Conference item

We consider planning for mobile robots conducting missions in realworld domains where a priori unknown dynamics affect the robot’s costs and transitions. We study single-episode missions where it is crucial that the robot appropriately trades off exploration and exploitation, such that the learning of the environment dynamics is just enough to effectively complete the mission. Thus, we propose modelling unknown dynamics using Gaussian processes, which provide a principled Bayesian framework for incorporating online observations made by the robot, and using them to predict the dynamics in unexplored areas. We then formulate the problem of mission planning in Markov decision processes under Gaussian process predictions as Bayesian model-based reinforcement learning. This allows us to employ solution techniques that plan more efficiently than previous Gaussian process planning methods are able to. We empirically evaluate the benefits of our formulation in an underwater autonomous vehicle navigation task and robot mission planning in a realistic simulation of a nuclear environment.

Authors: Budd, M; Duckworth, P; Hawes, N; Lacerda, B

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: OpenReview
• Host title: Proceedings of the 6th Conference on Robot Learning (CoRL 2022)
• Publication date: 2022-10-03
• Acceptance date: 2022-09-10
• Event title: 6th Conference on Robot Learning (CoRL 2022)
• Event location: Auckland, New Zealand
• Event website: https://corl2022.org/
• Event start date: 2022-12-14
• Event end date: 2022-12-18
• Language: English
• Keywords: FFR; Gaussian processes; single-episode Bayesian reinforcement learning; planning under uncertainty