Learning (to learn) from few examples

Bertinetto, L

Alternative title:: Visual tracking and classification with limited data

Abstract:: The availability of large labelled datasets has played a crucial role in the recent success of deep neural networks. However, there are many situations in which training data is scarce. For instance, in the case of object tracking, such a limitation arises directly from the definition of the problem, which requires an estimate of the position of an object of interest in every frame of a video with the sole supervision of a bounding box in the first frame. Another scenario which can be doomed by data scarcity is classification. For example, one might want to identify the species of a rare insect with a single reference image, or train a tumour classification system from a few labelled MRI scans. For both tracking and classification, we propose techniques that sidestep per-task data scarcity by leveraging a large number of small episodes, each characterised by a limited training set. This strategy is particularly novel for tracking, for which for many years the standard approach has been to train a discriminative model exclusively online, while the video is streaming. Moreover, we examine how this general approach can be framed as "learning to learn", in the sense that the knowledge distilled within a training task is accrued and used across tasks. We show how such a framework allows one to devise systems that can be trained with a small amount of per-task data while also being dynamically tailored to the problem at hand. From a practical point of view, the proposed methods have a common focus on simplicity, efficiency and speed, achieved by exploiting the shape or redundancies of the data.

All the code produced in this thesis is open source and available at http://www.robots.ox.ac.uk/~luca

Actions

Email

Email this record

Send the bibliographic details of this record to your email address.

Your Email
Please enter the email address that the record information will be sent to.

-
Your message (optional)
Please add any additional information to be included within the email.
Cite

Cite this record

APA Style

Bertinetto, L. (2019). Learning (to learn) from few examples [PhD thesis]. University of Oxford.

MLA Style

Bertinetto, L. Learning (to Learn) from Few Examples. University of Oxford, 2019.

Chicago Style

Bertinetto, L. 2019. “Learning (to Learn) from Few Examples.” PhD thesis, University of Oxford.
Share
Print

Access Document

Files:: bertinetto_dphil_thesis_2019.pdf

(Preview, pdf, 11.7MB, Terms of use)

Authors

+ Bertinetto, L More by this author

Division:: MPLS
Department:: Engineering Science
Department:: University of Oxford
Role:: Author

Contributors

+ Torr, P

Role:: Supervisor

+ Apical Ltd (now part of ARM) More from this funder

DOI:: 10.5287/ora-rvakvmnp8
Type of award:: DPhil
Level of award:: Doctoral
Awarding institution:: University of Oxford

Language:: English
Keywords:: few-shot learning

Siamese network

meta-learning

visual object tracking

machine learning

ridge regression

Correlation filter

artificial intelligence

similarity learning

deep learning

metric learning

computer vision
Subjects:: Machine learning

Computer vision

Artificial intelligence
UUID:: uuid:1f93b8a0-9a51-428b-9214-ce563161085a
Deposit date:: 2019-09-15

Related item:: Fully-convolutional siamese networks for object tracking
Description:: The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object's appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.
Related item:: Meta-learning with differentiable closed-form solvers
Description:: Adapting deep networks to new concepts from a few examples is challenging, due to the high computational requirements of standard fine-tuning procedures. Most work on few-shot learning has thus focused on simple learning techniques for adaptation, such as nearest neighbours or gradient descent. Nonetheless, the machine learning literature contains a wealth of methods that learn non-deep models very efficiently. In this paper, we propose to use these fast convergent methods as the main adaptation mechanism for few-shot learning. The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data. This requires back-propagating errors through the solver steps. While normally the cost of the matrix operations involved in such a process would be significant, by using the Woodbury identity we can make the small number of examples work to our advantage. We propose both closed-form and iterative solvers, based on ridge regression and logistic regression components. Our methods constitute a simple and novel approach to the problem of few-shot learning and achieve performance competitive with or superior to the state of the art on three benchmarks.
Related item:: Staple: Complementary learners for real-time tracking
Description:: Correlation Filter-based trackers have recently achieved excellent performance, showing great robustness to challenging situations exhibiting motion blur and illumination changes. However, since the model that they learn depends strongly on the spatial layout of the tracked object, they are notoriously sensitive to deformation. Models based on colour statistics have complementary traits: they cope well with variation in shape, but suffer when illumination is not consistent throughout a sequence. Moreover, colour distributions alone can be insufficiently discriminative. In this paper, we show that a simple tracker combining complementary cues in a ridge regression framework can operate faster than 80 FPS and outperform not only all entries in the popular VOT14 competition, but also recent and far more sophisticated trackers according to multiple benchmarks.
Related item:: Fast online object tracking and segmentation: A unifying approach
Description:: n this paper we illustrate how to perform both visual objecttracking and semi-supervised video object segmentation,in real-time, with a single simple approach. Our method,dubbed SiamMask, improves the offline training procedureof popular fully-convolutional Siamese approaches for ob-ject tracking by augmenting their loss with a binary seg-mentation task. Once trained, SiamMask solely relies on asingle bounding box initialisation and operates online, pro-ducing class-agnostic object segmentation masks and ro-tated bounding boxes at 55 frames per second. Despite itssimplicity, versatility and fast speed, our strategy allows usto establish a new state of the art among real-time track-ers on VOT-2018, while at the same time demonstratingcompetitive performance and the best speed for the semi-supervised video object segmentation task on DAVIS-2016and DAVIS-2017. The project website ishttp://www.robots.ox.ac.uk/~qwang/SiamMask
Related item:: End-to-end representation learning for correlation filter based tracking
Description:: The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations. It is well suited to object tracking because its formulation in the Fourier domain provides a fast solution, enabling the detector to be re-trained once per frame. Previous works that use the Correlation Filter, however, have adopted features that were either manually designed or trained for a different task. This work is the first to overcome this limitation by interpreting the Correlation Filter learner, which has a closed-form solution, as a differentiable layer in a deep neural network. This enables learning deep features that are tightly coupled to the Correlation Filter. Experiments illustrate that our method has the important practical benefit of allowing lightweight architectures to achieve state-of-the-art performance at high framerates.
Related item:: Long-term tracking in the wild: A benchmark
Description:: We introduce the OxUvA dataset and benchmark for evalu-ating single-object tracking algorithms. Benchmarks have enabled greatstrides in the field of object tracking by defining standardized evalua-tions on large sets of diverse videos. However, these works have focusedexclusively on sequences that are just tens of seconds in length and inwhich the target is always visible. Consequently, most researchers havedesigned methods tailored to this “short-term” scenario, which is poorlyrepresentative of practitioners’ needs. Aiming to address this disparity,we compile a long-term, large-scale tracking dataset of sequences withaverage length greater than two minutes and with frequent target ob-ject disappearance. The OxUvA dataset is much larger than the objecttracking datasets of recent years: it comprises 366 sequences spanning 14hours of video. We assess the performance of several algorithms, consid-ering both the ability to locate the target and to determine whether itis present or absent. Our goal is to offer the community a large and di-verse benchmark to enable the design and evaluation of tracking methodsready to be used “in the wild”. The project website is http://oxuva.net
Related item:: Learning feed-forward one-shot learners
Description:: One-shot learning is usually tackled by using generative models or discriminative embeddings. Discriminative methods based on deep learning, which are very effective in other learning scenarios, are ill-suited for one-shot learning as they need large amounts of training data. In this paper, we propose a method to learn the parameters of a deep model in one shot. We construct the learner as a second deep network, called a learnet, which predicts the parameters of a pupil network from a single exemplar. In this manner we obtain an efficient feed-forward one-shot learner, trained end-to-end by minimizing a one-shot classification objective in a learning to learn formulation. In order to make the construction feasible, we propose a number of factorizations of the parameters of the pupil network. We demonstrate encouraging results by learning characters from single exemplars in Omniglot, and by tracking visual objects from a single initial exemplar in the Visual Object Tracking benchmark.

Terms of use

Copyright holder:: Bertinetto, L

Licence:: Terms and Conditions of Use for Oxford University Research Archive

Views and Downloads

About views and downloads

If you are the owner of this record, you can report an update to it here: Report update to this record

Thesis

Learning (to learn) from few examples

Actions

Access Document

Authors

Contributors

Terms of use

Views and Downloads

Altmetrics

Dimensions

Thesis

Learning (to learn) from few examples

Actions

Access Document

Authors

Contributors

Funding

Bibliographic Details

Item Description

Related Items

Terms of use

Metrics

Views and Downloads

Altmetrics

Dimensions