Learning multiple visual domains with residual adapters

Conference item

There is a growing interest in learning data representations that work well for many different types of problems and data. In this paper, we look in particular at the task of learning a single visual representation that can be successfully utilized in the analysis of very different types of images, from dog breeds to stop signs and digits. Inspired by recent work on learning networks that predict the parameters of another, we develop a tunable deep network architecture that, by means of adapter residual modules, can be steered on the fly to diverse visual domains. Our method achieves a high degree of parameter sharing while maintaining or even improving the accuracy of domain-specific representations. We also introduce the Visual Decathlon Challenge, a benchmark that evaluates the ability of representations to capture simultaneously ten very different visual domains and measures their ability to perform well uniformly.

Authors: Rebuffi, S; Bilen, H; Vedaldi, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Neural Information Processing Systems
• Host title: Thirty-first Annual Conference on Neural Information Processing Systems (NIPS 2017)
• Journal: Advances in Neural Information Processing Systems
• Publication date: 2018-06-01
• Acceptance date: 2017-09-04
• ISSN: 1049-5258