Foundations and Trends® in Signal Processing > Vol 5 > Issue 4

Minimum Probability of Error Image Retrieval: From Visual Features to Image Semantics

Nuno Vasconcelos, University of California, USA, nuno@ece.ucsd.edu Manuela Vasconcelos, University of California, USA, maspcv@gmail.com
 
Suggested Citation
Nuno Vasconcelos and Manuela Vasconcelos (2012), "Minimum Probability of Error Image Retrieval: From Visual Features to Image Semantics", Foundations and TrendsĀ® in Signal Processing: Vol. 5: No. 4, pp 265-389. http://dx.doi.org/10.1561/2000000015

Published: 10 Dec 2012
© 2012 N. Vasconcelos and M. Vasconcelos
 
Subjects
Image and Video Retrieval,  Learning and statistical methods,  Object and scene recognition,  Pattern recognition and learning,  Quantization,  Storage, Access Methods, and Indexing,  Probabilistic Data Management,  Architectures for IR,  Classification and prediction,  Graphical models,  Robustness,  Texture analysis and synthesis
 

Free Preview:

Article Help

Share

Download article
In this article:
1 From Pixels to Semantic Spaces: Advances in Content-Based Image Search
2 Theoretical Foundations of MPE Retrieval
3 A Unified View of Image Similarity
4 An MPE Architecture for Image Retrieval
5 MPE Image Annotation and Semantic Retrieval
6 Weakly Supervised Estimation of Probability Densities
7 Query By Semantic Example
8 Conclusions
A Proofs
References

Abstract

The recent availability of massive amounts of imagery, both at home and on the Internet, has generated substantial interest in systems for automated image search and retrieval. In this work, we review a principle for the design of such systems, which formulates the retrieval problem as one of decision-theory. Under this principle, a retrieval system searches the images that are likely to satisfy the query with minimum probability of error (MPE). It is shown how the MPE principle can be used to design optimal solutions for practical retrieval problems. This involves a characterization of the fundamental performance bounds of the MPE retrieval architecture, and the use of these bounds to derive optimal components for retrieval systems. These components include a feature space where images are represented, density estimation methods to produce this representation, and the similarity function to be used for image matching. It is also shown that many alternative formulations of the retrieval problem are closely related to the MPE principle, typically resulting from simplifications or approximations to the MPE architecture. The MPE principle is then applied to the design of retrieval systems that work at different levels of abstraction. Query-by-visual-example (QBVE) systems are strictly visual, matching images by similarity of low-level features, such as texture or color. This is usually insufficient to produce perceptually satisfying results, since human users tend to make similarity judgments on the basis of image semantics, not visual attributes. This problem is addressed by the introduction of MPE labeling techniques, which associate descriptive keywords with images, enabling their search with text queries. This involves computing the probabilities with which different concepts explain each image. The query by example paradigm is then combined with these probabilities, by performing MPE image matching in the associated probability simplex. This is denoted query-by-semantic-example (QBSE), and enables example-based retrieval by similarity of semantics.

DOI:10.1561/2000000015
ISBN: 978-1-60198-608-5
128 pp. $85.00
Buy book
 
ISBN: 978-1-60198-609-2
128 pp. $110.00
Buy E-book
Table of contents:
1: From Pixels to Semantic Spaces: Advances in Content-Based Image Search
2: Theoretical Foundations of MPE Retrieval
3: A Unified View of Image Similarity
4: An MPE Architecture for Image Retrieval
5: MPE Image Annotation and Semantic Retrieval
6: Weakly Supervised Estimation of Probability Densities
7: Query By Semantic Example
8: Conclusions
A: Proofs

Minimum Probability of Error Image Retrieval

The recent availability of massive amounts of imagery, both at home and on the Internet, has generated substantial interest in systems for automated image search and retrieval. Minimum Probability of Error Image Retrieval reviews a principle for the design of such systems, which formulates the retrieval problem as one of decision-theory. Under this principle, a retrieval system searches the images that are likely to satisfy the query with minimum probability of error (MPE). It is shown how the MPE principle can be used to design optimal solutions for practical retrieval problems. This involves a characterization of the fundamental performance bounds of the MPE retrieval architecture, and the use of these bounds to derive optimal components for retrieval systems. Minimum Probability of Error Image Retrieval also shows that many alternative formulations of the retrieval problem are closely related to the MPE principle, typically resulting from simplifications or approximations of the MPE architecture. The MPE principle can be applied to the design of retrieval systems that work at different levels of abstraction. For instance, Query-by-visual-example (QBVE) systems are strictly visual, matching images by similarity of low-level features, such as texture or color. This is usually insufficient to produce perceptually satisfying results, since human users tend to make similarity judgments on the basis of image semantics, not visual attributes. This problem is addressed by the introduction of MPE labeling techniques, which associate descriptive keywords with images, enabling their search with text queries. This involves computing the probabilities with which different concepts explain each image. This book also shows how the query by example paradigm is then combined with these probabilities, by performing MPE image matching in the associated probability simplex. This is denoted query-by-semantic-example, (QBSE) and enables example-based retrieval by similarity of semantics. Minimum Probability of Error Image Retrieval is an ideal reference for anyone with an interest in image retrieval generally and the MPE retrieval framework in particular.

 
SIG-015