Story-Driven Summarization for Egocentric Video

Z. Lu and K. Grauman

Introduction

Egocentric video is the video taken from the camera that placed at the human position, not necessary in your hand, but on your head or arm. Such devices are more and more popular in recent years, and those videos are usually taken from amateur. One of the problem of those videos is too long. There is little possibility to spend several hours on watching some nameless videos, not to mention those videos are too many in the number. Thus, video summarization became valuable. This paper preposed a novel method that took not only low level feature, but also "Story" into account, resulted in great outperformance.

Method

This method can be separated into several parts :

(1) Segment the original video into a series of n sub-shot

(2) Define the components of objective function

(3) Optimizing the objective function

(4) Final summarization

(1) The author used optical flow and blur as features, and trained the classifier using SVM. 

(2) For every order-preserving chain of K selected nodes, selecting the optimal K-node chain S* :

S(S) is the story term, which can be computed by :

and

To account for coherency as well as influence, the objective function was modified to :

I(S) is the importance of individual sub-shots, which is computed using another work :

See 

14. Y. J. Lee, J. Ghosh, and K. Grauman. Discovering important people and objects for egocentric video summarization. In CVPR, 2012. 
    

for the detail of this term.

D(S) is diversity among transitions :

(3) For optimization, the author referred to 

14. D. Shahaf and C. Guestrin. Connecting the dots between news arti- cles. In KDD, 2010.
    
and do some modification.

(4) Sometime, the measurement of influence across the boundaries of major distinct event may be incorrect, the author posed the final summarization task in two layers. 

Using the formulation above, plus some extra steps, we can decompose the video into some events. Then we vary the argument K in the previous step to find the best chains, and concatenate it together.

Performance

We can see that most people regard this work performs better.

This work has the higher average true positive rate.

Result