Foundations and Trends® in Information Retrieval > Vol 2 > Issue 4

Concept-Based Video Retrieval

By Cees G. M. Snoek, University of Amsterdam, The Netherlands, cgmsnoek@uva.nl | Marcel Worring, University of Amsterdam, The Netherlands, worring@uva.nl

 
Suggested Citation
Cees G. M. Snoek and Marcel Worring (2009), "Concept-Based Video Retrieval", Foundations and Trends® in Information Retrieval: Vol. 2: No. 4, pp 215-322. http://dx.doi.org/10.1561/1500000014

Publication Date: 27 May 2009
© 2009 C. G. M. Snoek and M. Worring
 
Subjects
Video analysis and event recognition
 

Free Preview:

Download extract

Share

Download article
In this article:
1 Introduction 
2 Detecting Semantic Concepts in Video 
3 Using Concept Detectors for Video Search 
4 Evaluation 
5 Conclusions 
Acknowledgments 
References 

Abstract

In this paper, we review 300 references on video retrieval, indicating when text-only solutions are unsatisfactory and showing the promising alternatives which are in majority concept-based. Therefore, central to our discussion is the notion of a semantic concept: an objective linguistic description of an observable entity. Specifically, we present our view on how its automated detection, selection under uncertainty, and interactive usage might solve the major scientific problem for video retrieval: the semantic gap. To bridge the gap, we lay down the anatomy of a concept-based video search engine. We present a component-wise decomposition of such an interdisciplinary multimedia system, covering influences from information retrieval, computer vision, machine learning, and human–computer interaction. For each of the components we review state-of-the-art solutions in the literature, each having different characteristics and merits. Because of these differences, we cannot understand the progress in video retrieval without serious evaluation efforts such as carried out in the NIST TRECVID benchmark. We discuss its data, tasks, results, and the many derived community initiatives in creating annotations and baselines for repeatable experiments. We conclude with our perspective on future challenges and opportunities.

DOI:10.1561/1500000014
ISBN: 978-1-60198-234-6
120 pp. $85.00
Buy book (pb)
 
ISBN: 978-1-60198-235-3
120 pp. $100.00
Buy E-book (.pdf)
Table of contents:
1: Introduction
2: Detecting Semantic Concepts in Video
3: Using Concept Detectors for Video Search
4: Evaluation
Conclusions
Acknowledgements
References

Concept-Based Video Retrieval

Concept-Based Video Retrieval reviews 300 references on video retrieval, indicating when the text-only solutions of present-day video search engines are unsatisfactory and showing the promising alternatives which are primarily concept-based. Central to the discussion, therefore, is the fundamental notion of a semantic concept: an objective linguistic description of an observable entity. The book aims to motivate and explain how automated detection, selection under uncertainty, and interactive usage might solve the major scientific problems for video retrieval: the semantic gap. In striving to bridge this gap, the authors structured their review by laying down the anatomy of a concept-based video search engine. They present a component-wise decomposition and evaluation of such an interdisciplinary multimedia system, covering influences from information retrieval, computer vision, machine learning, and human-computer interaction. Concept-Based Video Retrieval is aimed primarily at researchers and developers in the broad area of information retrieval. It will also be an invaluable reference for students in computer and information science at the (post)graduate level, as well as industrial practitioners

 
INR-014