Fine-Grained Crowdsourcing for Fine-Grained Recognition

Jia Deng, Jonathan Krause, Li Fei-Fei Computer Science Department, Stanford University 

Description

Find-grained recognition is not just finding some ordinate object in the picture. It concerns more detail, more specific feature. For instance, we want to distinguish two different breeds of birds, and they could be only different in their tails, or spot at their chest.
Find-grained recognition is challenging, not only because the feature points might be in a small area and it's difficult to detect, but there are only small amount labeled data can be used in training process. 

Method

For the sake of getting labeled data efficiently, the author designed a game called 'bubbles'. First, we want to know whether this subject can be trusted. It means that whether this subject can distinguish two target objects are the same or different. Thus, in the beginning, this game will give some question with exact answer (or say ground truth). If the subject get the high score, we can apply some question without ground truth, and believe that the subject will give the correct answer.

The game UI.

As we can see, two classes of object are showed on the left and right. What we need to recognize is in the middle. It is a blurred image, and the subject can let some area be clear. These areas could hold some important feature points for the training process later.

After getting the feature area, the author used SIFT as feature descriptor, and trained it with SVM. For reducing the complexity, the author picked only the most confusing category pairs to apply their classifier.

Result

We can see that this method got about 10% improvement.