Deep learning with synthetic, temporal, and adversarial supervision

Gupta, A

Abstract:: In this thesis we explore alternatives to manually annotated training examples for supervising the training of deep learning models. Specifically, we develop methods for learning under three different supervision paradigms, namely — (1) synthetic data, (2) temporal data, and (3) adversarial supervision for learning from unaligned examples. The dominant application domain of our work is text spotting, i.e. detection and recognition of text instances in images. We learn text localisation net...
Expand abstract
Collapse abstract

Actions

Email

Email this record

Send the bibliographic details of this record to your email address.

Your Email
Please enter the email address that the record information will be sent to.

-
Your message (optional)
Please add any additional information to be included within the email.
Cite

Cite this record

APA Style

Gupta, A. (2018). Deep learning with synthetic, temporal, and adversarial supervision [PhD thesis]. University of Oxford.

MLA Style

Gupta, A. Deep Learning with Synthetic, Temporal, and Adversarial Supervision. University of Oxford, 2018.

Chicago Style

Gupta, A. 2018. “Deep Learning with Synthetic, Temporal, and Adversarial Supervision.” PhD thesis, University of Oxford.
Share
Print

Access Document

Files:: AnkushGupta18_compress.pdf

(Preview, pdf, 11.0MB, Terms of use)

Authors

+ Gupta, A More by this author

Division:: MPLS
Department:: Engineering Science
Department:: Engineering Science
Role:: Author

Contributors

+ Vedaldi, A

Institution:: University of Oxford
Division:: MPLS
Department:: Engineering Science
Role:: Supervisor

+ Zisserman, A

Institution:: University of Oxford
Division:: MPLS
Department:: Engineering Science
Role:: Supervisor

+ Engineering and Physical Sciences Research Council More from this funder

Grant:: EP/L015987/2

+ UK EPSRC CDT in Autonomous Intelligent Machines and Systems More from this funder

Grant:: EP/L015987/2

DOI:: 10.5287/ora-2nqvgy7gz
Type of award:: DPhil
Level of award:: Doctoral
Awarding institution:: University of Oxford

UUID:: uuid:0a9d2c4a-0e8d-4722-9e0b-3f247efa89e8
Deposit date:: 2019-04-14

Related item:: Unsupervised Learning of Object Landmarks through Conditional Image Generation
Description:: We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision. We cast this as the problem of generating images that combine the appearance of the object as seen in a first example image with the geometry of the object as seen in a second example image, where the two examples differ by a viewpoint change and/or an object deformation. In order to factorize appearance and geometry, we introduce a tight bottleneck in the geometry-extraction process that selects and distils geometry-related features. Compared to standard image generation problems, which often use generative adversarial networks, our generation task is conditioned on both appearance and geometry and thus is significantly less ambiguous, to the point that adopting a simple perceptual loss formulation is sufficient. We demonstrate that our approach can learn object landmarks from synthetic image deformations or videos, all without manual supervision, while outperforming state-of-the-art unsupervised landmark detectors. We further show that our method is applicable to a large variety of datasets - faces, people, 3D objects, and digits - without any modifications.
Related item:: Learning to Read by Spelling: Towards Unsupervised Text Recognition
Description:: This work presents a method for visual text recognition without using any paired supervisory data. We formulate the text recognition task as one of aligning the conditional distribution of strings predicted from given text images, with lexically valid strings sampled from target corpora. This enables fully automated, and unsupervised learning from just line-level text-images, and unpaired text-string samples, obviating the need for large aligned datasets. We present detailed analysis for various aspects of the proposed method, namely - (1) impact of the length of training sequences on convergence, (2) relation between character frequencies and the order in which they are learnt, (3) generalisation ability of our recognition network to inputs of arbitrary lengths, and (4) impact of varying the text corpus on recognition accuracy. Finally, we demonstrate excellent text recognition accuracy on both synthetically generated text images, and scanned images of real printed books, using no labelled training examples.
Related item:: Synthetic Data for Text Localisation in Natural Images
Description:: In this paper we introduce a new method for text detection in natural images. The method comprises two contributions: First, a fast and scalable engine to generate synthetic images of text in clutter. This engine overlays synthetic text to existing background images in a natural way, accounting for the local 3D scene geometry. Second, we use the synthetic images to train a Fully-Convolutional Regression Network (FCRN) which efficiently performs text detection and bounding-box regression at all locations and multiple scales in an image. We discuss the relation of FCRN to the recently-introduced YOLO detector, as well as other end-to-end object detection systems based on deep learning. The resulting detection network significantly out performs current methods for text detection in natural images, achieving an F-measure of 84.2% on the standard ICDAR 2013 benchmark. Furthermore, it can process 15 images per second on a GPU.
Related item:: Inductive Visual Localisation: Factorised Training for Superior Generalisation
Description:: End-to-end trained Recurrent Neural Networks (RNNs) have been successfully applied to numerous problems that require processing sequences, such as image captioning, machine translation, and text recognition. However, RNNs often struggle to generalise to sequences longer than the ones encountered during training. In this work, we propose to optimise neural networks explicitly for induction. The idea is to first decompose the problem in a sequence of inductive steps and then to explicitly train the RNN to reproduce such steps. Generalisation is achieved as the RNN is not allowed to learn an arbitrary internal state; instead, it is tasked with mimicking the evolution of a valid state. In particular, the state is restricted to a spatial memory map that tracks parts of the input image which have been accounted for in previous steps. The RNN is trained for single inductive steps, where it produces updates to the memory in addition to the desired output. We evaluate our method on two different visual recognition problems involving visual sequences: (1) text spotting, i.e. joint localisation and reading of text in images containing multiple lines (or a block) of text, and (2) sequential counting of objects in aerial images. We show that inductive training of recurrent models enhances their generalisation ability on challenging image datasets.

Terms of use

Copyright holder:: Gupta, A

Licence:: Terms and Conditions of Use for Oxford University Research Archive

Views and Downloads

About views and downloads

If you are the owner of this record, you can report an update to it here: Report update to this record

Thesis

Deep learning with synthetic, temporal, and adversarial supervision

Actions

Access Document

Authors

Contributors

Terms of use

Views and Downloads

Altmetrics

Dimensions

Thesis

Deep learning with synthetic, temporal, and adversarial supervision

Actions

Access Document

Authors

Contributors

Funding

Bibliographic Details

Item Description

Related Items

Terms of use

Metrics

Views and Downloads

Altmetrics

Dimensions