Our Research

Deep Image Retrieval: End-to-end Learning of Deep Visual Representations for Image Retrieval

deep image retrieval

 

 

 

Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus     

Abstract:

While deep learning has become a key ingredient in the top performing methods for many computer vision tasks, it has failed so far to bring similar improvements to instance-level image retrieval. In this article, we argue that reasons for the underwhelming results of deep methods on image retrieval are threefold: i) noisy training data, ii) inappropriate deep architecture, and iii) suboptimal training procedure. We address all three issues. First, we leverage a large-scale but noisy landmark dataset and develop an automatic cleaning method that produces a suitable training set for deep retrieval. Second, we build on the recent R-MAC descriptor, show that it can be interpreted as a deep and differentiable architecture, and present improvements to enhance it. Last, we train this network with a Siamese architecture that combines three streams with a triplet loss. At the end of the training process, the proposed architecture produces a global image representation in a single forward pass that is well suited for image retrieval. Extensive experiments show that our approach significantly outperforms previous retrieval approaches, including state-of-the-art methods based on costly local descriptor indexing and spatial verification. On Oxford 5k, Paris 6k and Holidays, we respectively report 94.7, 96.6, and 94.8 mean average precision. Our representations can also be heavily compressed using product quantization with little loss in accuracy. For additional material, please see www.xrce.xerox.com/Deep-Image-Retrieval. 

 Model: 

Deep learning architecture used to train our model

 Deep learning architecture used to train our model

Papers:

Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus: End-to-end Learning of Deep Visual Representations for Image Retrieval
arXiv preprint [pdf]

@article{gordo2016end,
author = {Albert Gordo and Jon Almaz{\'{a}}n and J{\'{e}}r{\^{o}}me Revaud and Diane Larlus},
title = {End-to-end Learning of Deep Visual Representations for Image Retrieval}
journal = {CoRR},
volume = {abs/1610.07940},
year = {2016}}

Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus: Deep Image Retrieval: Learning global representations for image search
ECCV 2016 [pdf] [Publication entry]

@inproceedings{gordo2016deep,
author = {Albert Gordo and Jon Almaz{\'{a}}n and J{\'{e}}rome Revaud and Diane Larlus},
title = {Deep Image Retrieval: Learning global representations for image search},
booktitle={ECCV},
year={2016}}

Model and evaluation script:

Dataset

Result