Intelligent decision making on-board satellites

Thesis

In this thesis we explore opportunities that are given to us with Remote Sensing data collected by satellites and their processing using Machine Learning. Satellites provide unprecedented quantity of data, repeated and uniform observations of Earth and other planetary surfaces and they allow for detection of many events of interest. Some of these events are closely tied with one of the key challenges of this generation, Climate Change.

However, the current systems on most devices in communication constrained environments (such as satellites) do not rely on on-board processing of the observed data and operate only in passive modes. They need to be manually scheduled, and the data needs to be manually processed by experts on the ground. Furthermore, the quantity of the observed data is often larger than the communication capabilities and the downlinked data is limited. As a result, events of potential interest are often lost or detected late. The uplink constrains are even more severe, making this an interesting asymmetrical scenario that could be addressed with model adaptation on-board.

The central topic of this thesis is the question of how to enable smart on-board processing of data to allow for autonomous decision making. We present three works where we explore this challenge.

In our first project, we successfully deploy a change detection machine learning model on-board of a real satellite and we test it in space. Our encoder-decoder model enables data compression into latent vectors, multi-temporal comparisons between observed images and allows training on-board of the satellite as an experimental proxy for adaptation.

In our second project, we focus on processing data from a variety of sensors for the task of automated methane leak detection. We build benchmark datasets of hyperspectral data and explore classically used methods which we improve with trained models.

Finally, in our last project, we design novel machine learning architectures aiming at deployment of automated general detectors on-board of hyperspectral satellites. We show that imaging spectroscopy data requires specific adaptations of machine learning architectures to prevent informational and computational bottlenecks. We demonstrate the efficacy of our models on two example tasks, further improving methane detection and exploring mineral identification from space. We measure the inference speeds of our models on low-compute hardware which was previously tested in harsh space environments.

Authors: Růžička, V

• DOI: 10.5287/ora-y0bvvqrzb
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: remote sensing; methane detection; imaging spectroscopy; hyperspectral satellites; climate change mitigation; change detection; AI on-board satellites; machine learning; unsupervised learning
• Subjects: Remote sensing; Hyperspectral imaging; Machine learning; Deep learning (Machine learning); Neural networks (Computer science); Edge computing; Methane