Situated decision-making in uncertain environments

Thesis

Mobile robots are highly suited to large-scale data-gathering, monitoring and exploration tasks in challenging and poorly understood environments such as oceans and hazardous unmapped areas. To operate reliably in these settings, they need decision-making methods that can function under a lack of knowledge about the environment. This lack of knowledge is known as epistemic uncertainty. A robot situated in such an environment faces several challenges: it has only local information about its environment, its actions may be affected by unknown environment dynamics such as water currents, and it may only have access to its own on-board computational resources and sensors.

In this thesis, we aim to answer the question of how an autonomous system such as a mobile robot can make the best possible decisions, given its bounded computational resources and limited knowledge of its deployment environment. We do this by contributing new decision-making methods for control of robot actions, and new metareasoning methods for control of reasoning processes.

In one strand of work, we design three new decision-making methods for mobile robots operating in epistemically uncertain environments. These range from an offline method that pre-plans robot policies, to online methods that better adapt to the environment but require more computational resources. We demonstrate our methods in domains including underwater and hazardous environment mobile robotics, and two real field deployments. In each case our algorithms achieve their goals while incurring less cost than previous approaches, or are able to tackle problem settings that previous approaches could not.

In an orthogonal research direction, we develop learning-based metareasoning methods that automatically adapt a robot's decision-making strategy to the deployment environment, the robot's reasoning capabilities, and the problem at hand. We do this by proposing two metareasoning frameworks, which apply to offline and online decision-making methods respectively. We evaluate our methods in environments specifically designed to test how metareasoning improves decision-making in situated systems, and demonstrate a significant performance increase over fixed strategies and prior metareasoning methods.

Authors: Budd, M

• DOI: 10.5287/ora-qmw4vq9d1
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: Markov decision processes; Bayesian reinforcement learning; epistemic uncertainty; metareasoning; decision-making under uncertainty
• Subjects: Robotics; Artificial intelligence; Machine learning; Decision making