To aggregate or not to aggregate: Selective match kernels for image search

Giorgos Tolias, Yannis Avrithis, Herve Jegou

Introduction

This paper is interested in improving visual recognition of objects, locations and scenes. To do this, we require some methods to compute the similarity of two different objects. 'Kernel' is the important thing during similarity computation. The author first introduced some basic kernels, then proposed their advanced methods, which added some new features called selectivity and aggregation.

Basic match kernels

(1) Bag-of-words (BOW)

(2) Hamming Embedding (HE)

h: Hamming distance

(3) Vector or Locally Aggregated Descriptor (VLAD)

Common kernel model

Non-aggregated and aggregated kernel have their individual form as following:

Non-aggregated kernels

:an arbitrary vector representation function

:a scalar selectivity function

Aggregated kernels

:another vector representation function

:aggregated vector representation of a set Xc of descriptors in a cell

Advanced match kernel

Following the forms above, the author chose their different representation function and proposed their own kernels

Selective match kernel (SMK)

Aggregated selective match kernel (ASMK)

Binarization

To save memory, the author used binary vector b instead of residual r(x).

SMK*

ASMK*

Experiments

Plot the mAP to different alpha

Plot the mAP to different tau

Plot the mAP to different k

The result that using aggregation to save memory usage

Compare with other methods

The result that using different feature