Counting 3,384 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Last Commit
Dec. 9, 2018
Nov. 26, 2018

MIRorR: Matchable Image Retrieval by Learning from Surface Reconstruction

Mirror is the matchable image retrieval pipeline for 3D reconstruction and related applications. Different from typical object retrieval, matchable image retrieval aims to find similar images with large overlaps. Typical CNN-based methods do not generalize well to this problem because models are trained to find objects of the same category. This project proposes a new method to tackle this problem, by utilizing regional feature aggregation and the accurate auto-annotated 3D geometric data. For more details, you can refer to the paper:

Matchable Image Retrieval by Learning from Surface Reconstruction

Tianwei Shen*, Zixin Luo*, Lei Zhou, Runze Zhang, Siyu Zhu, Tian Fang, Long Quan (* denotes equal contributions)

In ACCV 2018.

Feel free to submit issues if you have any questions.


The code base has been tested under TensorFlow 1.5 (CUDA 8.0) to TensorFlow (CUDA 9.0), using Python 2.7.12.

GL3D dataset

GL3D overview

The GL3D contains 90,590 high-resolution images in 378 different scenes. Each scene contains 50 to 1,000 images with large geometric overlaps, covering urban, rural area, or scenic spots captured by drones from multiple scales and perspectives. Several small objects are also included to enrich the data diversity. Each scene data is reconstructed to generate a triangular mesh model by the state-of-the-art 3D reconstruction pipeline.

Currently we release the 9368 test images for evaluation (see #Test). The full dataset can be found here.


Training code will be released soon.


First download the model files using the following command, suppose you are in the root directory of the project $MIRROR_ROOT

sh ./data/model/

Please refer to for using the image retrieval pipeline. We release two trained models to demonstrate the use. The googlenet model can be used to reproduce the results in the paper, which achieves 0.758 mAP@200 on GL3D, 0.768 mAP on Oxford5K and 0.820 on Paris6K using the default settings in

We have additionally trained a ResNet-50 model not documented in the original paper, which achieves better performance than GoogleNet on object retrieval datasets (Oxford5K and Paris6K).

Model GL3D (mAP@200) Oxford5K Paris6K Holidays
GoogleNet 0.758 0.768 0.820 0.861
ResNet-50 0.745 0.833 0.805 0.892
ResNet-50 + QE - 0.894 0.858 0.881

Note on GL3D

Download the GL3D test images of size 896 x 896 (around 3.7GB) and unzip them to your GL3D image folder (suppose it is $MIRROR_ROOT/data/gl3d/), and then run the commands for testing. The detailed instruction is given below.

# Download evaluation dataset and unzip
mv ./data/gl3d/
cd data/gl3d/
cd -
# Set up testing configs
net_type=resnet-50 #googlenet
step=720000 #400000 
# Extract features
python retrieval/ --img_list $test_image_list --ckpt_step $step --img_size $img_size \
    --net $net_type --pool $pooling --rmac_step $rmac_step --ckpt_path $ckpt_save_dir --gl3d_root $gl3d_root_dir
# PR-MAC + Top-k
python retrieval/ --feature_list $test_feature_list --out_dir $output_dir --top 200 --out_dim $feature_dim --dataset_root $gl3d_root_dir
# Compute statistics
python retrieval/ $test_image_list $output_dir/match_pairs data/gl3d/gl3d_gt_pairs

Note on Oxford5K or Paris6K

To run the model on Oxford5K or Paris6K, there is one additional step. Suppose you are in the root directory of this repo, you need to first compile the C++ program compute_ap for computing the average precision (AP).

cd cpp
g++ -o compute_ap compute_ap.cpp

Note on Holidays

Downloads INRIA Holidays dataset at official website, prepare the required input lists and use the commands in to reproduce the results.

Related Projects

Also checkout the following related geometric learning repositories:

GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints

MVSNet: Depth Inference for Unstructured Multi-view Stereo