% \begin{savequote}[8cm]
% \textlatin{Neque porro quisquam est qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit...}

% There is no one who loves pain itself, who seeks after it and wants to have it, simply because it is pain...
%   \qauthor{---- Cicero's \textit{de Finibus Bonorum et Malorum}}
% \end{savequote}

\chapter{Introduction}
\label{ch:intro}

\minitoc


\section{Overview}
% Paragraphs:
%   General hook
%   Where long-term is valuable
%   What the planning challenges are
%   What we do

The field of robotics has seen immense progress in recent years, producing robots that are more robust, capable, and widely accessible than ever before.
%
\correction{This has enabled successful deployments in increasingly complex and challenging domains, from unknown subterranean environments, to forests, and the depths of  oceans.}
%
Moreover, improved robustness across diverse domains has created new opportunities for \emph{long-term} deployments, in which robots operate for prolonged periods---weeks, months, or years---without the need for human intervention or supervision. % without the need for human supervision or input.
%
However, long-term robot missions present a unique combination of challenges which make it difficult to come up with effective plans.
%
Key challenges include practical constraints on resources such as time and battery life, dynamic and partially observable environments, and the high computational demand of long-horizon planning.
%
In order to unlock the full potential of long-term robot deployments, we require  planning algorithms that explicitly tackle these challenges.

\begin{figure}[]
    \centering
    \begin{subfigure}[b]{0.48\linewidth}
        \centering
        \includegraphics[width=1.0\textwidth]{chapters/chIntro/figures/robot_wytham.jpg}
        \caption{A Clearpath Husky robot navigating the site of an ecological research project.}
        \label{sfig:robot_wytham}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.48\linewidth}
        \centering
        \includegraphics[width=1.0\textwidth]{chapters/chIntro/figures/robot_corsham.JPG}
        \caption{A Boston Dynamics Spot robot exploring a disused military bunker.}
        \label{sfig:robot_corsham}
    \end{subfigure}
    \caption[Examples of autonomous robots in unstructured environments]{Examples of mobile robots operating autonomously in unstructured environments}
    \label{fig:robot_images}
\end{figure}

Long-term monitoring for robots holds significant promise across a range of scientific and industrial applications.
%
In precision agriculture, robots have been used to monitor pasturelands, providing long-term models that help farmers to optimise crop health and manage resources more effectively~\cite{liu2020monitoring,botta2022review}.
%
Robots have been deployed to conduct long-term visual surveying~\cite{griffith2017survey} and mapping~\cite{dong20174d} of natural environments, where they can provide data of much higher spatial and temporal resolution than human surveyors or satellite imagery, at a significantly lower cost.
%
Autonomous unmanned air vehicles~(UAVs) have become increasingly widespread tools for detection, prevention and monitoring of wildfires~\cite{bailon2022real,couceiro2019semfire}, providing unparalleled situational awareness which has proved particularly valuable in remote areas~\cite{afghah2019wildfire}.
%
Scientists in many areas of earth and life sciences have also realised the potential of robots for large-scale and long-term data gathering.
%
Autonomous robots have been used extensively by oceanographers to study marine environments, as well as by geologists for monitoring of volcanic activity~\cite{muscato2012volcanic}, and by meteorologists studying the atmosphere to learn about microclimates~\cite{dunbabin2012robots}.
%
As these technologies mature, we can expect that the scope of viable applications will only continue to grow.


In this thesis, we address \textbf{planning under uncertainty} for \textbf{long-term robot monitoring} of \textbf{continuous spatiotemporal processes} in \textbf{unstructured environments}.
%
Such processes might include bioactivity in coral reefs, or the distribution of wildfire hotspots---in each case, features that can be observed locally by robots using on-board sensors.
%
Planning involves generating \emph{policies} that specify actions for the robot to take in order to best serve its mission objectives, given the available information to it at each decision point.
%
The planning problem for our long-term monitoring setting comes with several distinct yet interrelated challenges.
%
One such challenge is that of modelling the  processes of interest using the available data.
%
The observational data produced by a robot in this setting will be sparse and often noisy.
%
The environment processes of interest, on the other hand, are often complex and dynamic, making them difficult to capture accurately with limited data.
%
% Throughout this thesis, we use Gaussian processes~(GPs) as our weapon of choice for modelling these processes, due to their flexibility and effectiveness across a wide range of natural domains.
%
Even the topology of the environment is often dynamic over the long term from a robot planning perspective.
%
Obstacles might shift over time, or certain routes might be unsafe to traverse in adverse weather conditions.
%
In some cases, the layout of the environment might not be known \emph{a priori} at all, requiring the robot to build a map of its surroundings online---a task which has been the subject of considerable research.


Mobile robots are also inherently resource-constrained, primarily by battery life, which limits the duration for which they can remain continuously active.
%
% Achieving long-term operation without human intervention requires the physical capability for the robot to replenish its own resources (e.g. self-docking to recharge its batteries), as well as the \emph{planning} capability...
%
Long-term operation without any human intervention requires robots to manage their own resources in order to avoid resource depletion, whilst continuing to service their mission objectives.
%
This presents technological challenges, in terms of developing reliable and cost-effective infrastructure for autonomous recharging, as well as algorithmic challenges in terms of incorporating these resource constraints into long-term planning.
%
However, where independent resource management \emph{is} possible, it can unlock the ability for a single robot to monitor much larger environments---provided, of course, that the required infrastructure (e.g. battery charging stations) is present.


Planning over long horizons is challenging in general, and particularly so in the dynamic and highly uncertain environments on which we focus throughout this thesis.
%
These environments tend to produce complex interactions with the robots that operate within them, resulting in further uncertainty over the outcomes of robot actions.
%
Varied terrain might make it difficult to predict the time required to navigate between locations, for example, or the robot's battery might deplete at a rate that varies significantly with the ambient temperature.
%
These phenomena are often too complex to capture explicitly in deterministic models, so must instead be represented through uncertain, probabilistic models.
%
Model uncertainties accumulate over the planning horizon, making it difficult to achieve reliable decision making.
%
However, in missions set in large environments or over long timescales, long-horizon planning is \emph{essential} to ensure good performance, since short-horizon planners may be unable to consider the full scale of the environment or the environmental process of interest.


In this thesis, we develop planning algorithms for long-term robot monitoring missions in natural environments.
%
Existing works have addressed the challenges of this setting in different ways, but often in isolation, thereby avoiding the compounding effect of the interactions between them.
%
We therefore aim to address the challenges of long-term robot monitoring in a more unified manner, developing algorithmic solutions that are modular and applicable to a wide variety of practical problems.

% Venn diagram - long-horizon planning, resource constraints, environment modelling, stochasticity


\section{Challenges}
\label{sec:challenges}


This thesis aims to develop planning solutions that address the practical challenges of long-term robot monitoring missions.
%
The difficulty of planning over long horizons is compounded by the contextual challenges faced by robots in the field.
%
Each of these challenges has been addressed on some level by previous works, each with their own particular formulations and assumptions.
%
Our goal is to demonstrate planning solutions that can tackle these challenges in combination, making them applicable to a wider range of practical problems.
%
The challenges we focus on in this thesis are:

\begin{challenge}[Environment Modelling]
    \label{challenge:modelling}
    The features of interest in natural environments come with rich and complex behaviours, varying both spatially and temporally.
    %
    Effective monitoring requires flexible modelling solutions which can capture this complexity from the sparse, noisy, local observations produced by on-board sensors.
\end{challenge}


% \begin{challenge}[Scale]
%     \label{challenge:scale}
%     Large scale, long horizon. 
% \end{challenge}

\begin{challenge}[Resource Constraints]
    \label{challenge:resources}
    Robots operating for extended periods are inevitably limited by their finite resources.
    %
    Integrating resource management into planning adds complexity to the planning problem, while also unlocking a new layer of intelligent behaviour.
\end{challenge}

\begin{challenge}[Stochastic Interactions]
    \label{challenge:stochastic}
    Robots interact with natural environments in complex ways, and the outcomes of these interactions are often impossible to model deterministically.
    %
    Instead, the uncertainty itself must be incorporated into planning, in such a way that produces robust and effective policies.
\end{challenge}

\begin{challenge}[Limited Priors]
    \label{challenge:priors}
    Prior knowledge can be an invaluable asset for planning, but in practice it is often expensive or impractical to acquire in unstructured environments.
    %
    % Minimising assumptions about prior knowledge leads to systems that can operate across more diverse application domains.
    Maximising the value of the limited prior knowledge that is available leads to systems that can operate effectively across more diverse application domains.
\end{challenge}

\section{Contributions}
\label{sec:contribution}

The core contributions of this thesis to addressing the challenges outlined in Section~\ref{sec:challenges} are as follows:

\begin{itemize}
    \item
          A planning algorithm for long-term monitoring of a continuous spatiotemporal process across \emph{repeated} missions.
          %
          This addresses \challengeref{challenge:modelling} by using a GP model with hyperparameters that are learned online, demonstrating that this model functions sufficiently well with sparse data to enable effective planning in this setting.
          To a lesser degree, it also addresses \challengeref{challenge:priors} by incorporating online learning of GP hyperparameters to reduce reliance on prior domain knowledge of the environment process.
    \item
          A novel batch evaluation method for GP-based rewards during MCTS rollouts, which improves computational performance by reducing the number of GP evaluations required.
          %
          This method is used in Chapters~\ref{ch:LTM1}--\ref{ch:LTM3} to make planning over a GP model (\challengeref{challenge:modelling}) computationally tractable.
    \item
          A hierarchical planning framework for long-term monitoring in resource-constrained settings (\challengeref{challenge:resources}).
          %
          With this new framework, we demonstrate the benefits of integrating resource management into long-term planning, showing that the flexibility of doing so can lead to better outcomes.
    \item
          An extension of the above hierarchical planner to consider uncertainty over action outcomes, thereby additionally addressing \challengeref{challenge:stochastic}.
          %
          We adopt a chance-constrained approach to resource management, a novel method in the context of long-term robot monitoring.
          %
    \item
          A planning algorithm for safe exploration and mapping of unknown environments with no prior map (\challengeref{challenge:priors}).
          %
          Although this contribution does not directly address long-term monitoring problems, its algorithmic framework has many similar elements, and offers a pathway to extend the above contributions to environments in which a prior map is not available.
\end{itemize}



\section{Thesis structure}
\label{sec:structure}

In this thesis, we develop planning algorithms that address the long-term robot monitoring challenges outlined in the previous section.
%
Chapter~\ref{ch:background} summarises the relevant background knowledge for this thesis, introducing models and techniques that are used throughout.
%
Chapter~\ref{ch:related_work} provides an overview of the relevant literature.
%
Additional background material and literature are also presented within the later chapters as needed.
%
In Chapter~\ref{ch:LTM1}, we formulate the long-term robot monitoring problem as a series of repeated deployments, and present a planning approach to solve it.
%
We use GPs to model continuous spatiotemporal environment processes, with a kernel function geared specifically towards the monitoring problems that motivate the work in this thesis.
%
We also incorporate online learning of GP hyperparameters to reduce reliance on domain knowledge, and demonstrate that our GP model can be used effectively for planning in this setting.
%
Finally, we introduce a batch evaluation method for GP-based rewards in MCTS, enhancing the computational efficiency and overall performance of our planner.
%
The work in this chapter is published in the proceedings of IROS 2024, the \nth{37} IEEE/RSJ International Conference on Intelligent Robots and Systems~\cite{stephens2024planning}.


In Chapter~\ref{ch:LTM2}, we consider long-term monitoring in a resource-constrained setting, developing a hierarchical planning framework that integrates resource management directly into the planning process.
%
We frame the problem using depots, pre-determined locations within the environment at which the robot can replenish its resources.
%
We demonstrate the benefits of our planning approach, which allows the robot to decide when and where to replenish its resources in order to best serve its mission objective.
%
The hierarchical approach also enables more effective long-term planning, by decoupling responsibilities for short-term action selection and longer-term, high-level mission planning between two levels of the hierarchy.
%
The work in this chapter is published in the proceedings of ECAI 2024, the \nth{27} European Conference on Artificial Intelligence~\cite{stephens2024hierarchical}.

Chapter~\ref{ch:LTM3} extends the work from the previous chapter to consider uncertainty over action outcomes.
%
The durations of navigation actions, and the outcomes of resource replenishment actions, are modelled as random variables with known distributions.
%
We then adopt a chance-constrained approach to resource management, enforcing constraints on the probability of the resource level dropping below a specified minimum level.
%
This adds significant complexity to the planning problem, but we demonstrate that we are able to obtain comparable performance (measured by cumulative reward) to a deterministic planning baseline, while limiting the number of resource constraint violations.

Chapter~\ref{ch:safe_exp} considers the problem of safe exploration and mapping of unknown environments.
%
We extend existing work on safe exploration of an unknown hazard in a pre-mapped environment, developing an algorithm that instead incrementally constructs the navigation graph during exploration.
%
This work bridges the gap between the long-term monitoring and environments in which no prior map is available, enabling a robot to safely and independently construct a topological map which can subsequently form the basis of a monitoring mission.
%
The problem formulated in this chapter shares many commonalities with the long-term monitoring work in the previous chapters, considering GP models of continuous features and planning over a graph-based environment representation.
%
The work in this chapter is published in the journal Autonomous Robots~\cite{stephens2024auro}.

Chapter~\ref{ch:conclusion} concludes the thesis and discusses potential avenues for future work.


\section{Publications}

The research presented in this thesis is published as follows:
%
\begin{itemize}
    \item \fullcite{stephens2024planning}.%~\cite{stephens2024planning}.
    \item \fullcite{stephens2024hierarchical}.%~\cite{stephens2024hierarchical}.
    \item \fullcite{stephens2024auro}.%~\cite{stephens2024auro}.
\end{itemize}

