A Bayesian exploration−exploitation approach for optimal online sensing and planning with a visually guided mobile robot

Journal article

We address the problem of online path planning for optimal sensing with a mobile robot. The objective of the robot is to learn the most about its pose and the environment given time constraints. We use a POMDP with a utility function that depends on the belief state to model the finite horizon planning problem. We replan as the robot progresses throughout the environment. The POMDP is high-dimensional, continuous, non-differentiable, nonlinear, non-Gaussian and must be solved in real-time. Most existing techniques for stochastic planning and reinforcement learning are therefore inapplicable. To solve this extremely complex problem, we propose a Bayesian optimization method that dynamically trades off exploration (minimizing uncertainty in unknown parts of the policy space) and exploitation (capitalizing on the current best solution). We demonstrate our approach with a visually-guide mobile robot. The solution proposed here is also applicable to other closely-related domains, including active vision, sequential experimental design, dynamic sensing and calibration with mobile sensors.

Authors: Martinez−Cantin, R; Freitas, N; Brochu, E; Castellanos, J; Doucet, A

• Journal: Autonomous Robots
• Volume: 27
• Issue: 2
• Pages: 93-103
• Publication date: 2009-01-01
• DOI: 10.1007/s10514-009-9130-2
• ISSN: 0929-5593