Abstract
A solution to the problem of image retrieval based on query-by-semantic-example (QBSE) is presented. QBSE extends the idea of query-by-example to the domain of semantic image representations. A semantic vocabulary is first defined, and a semantic retrieval system is trained to label each image with the posterior probability of appearance of each concept in the vocabulary. The resulting vector is interpreted as the projection of the image onto a semantic probability simplex, where a suitable similarity function is defined. Queries are specified by example images, which are projected onto the probability simplex. The database images whose projections on the simplex are closer to that of the query are declared its closest neighbors. Experimental evaluation indicates that 1) QBSE significantly outperforms the traditional query-by-visual-example paradigm when the concepts in the query image are known to the retrieval system, and 2) has equivalent performance even in the worst case scenario of queries composed by unknown concepts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Barnard, K., Duygulu, P., de Freitas, N., Forsyth, D., Blei, D., Jordan, M.I.: Matching words and pictures. JMLR 3, 1107–1135 (2003)
Blei, D., Jordan, M.I.: Modeling annotated data. In: Proceedings of the 26th Intl. ACM SIGIR Conf., pp. 127–134 (2003)
Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: NIPS (2003)
Picard, R.: Digital Libraries: Meeting Place for High-Level and Low-Level Vision. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035. Springer, Heidelberg (1996)
Feng, S.L., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. In: CVPR (2004)
Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval: the end of the early years. PAMI 22(12), 1349–1380 (2000)
Vasconcelos, N.: Image Indexing with Mixture Hierarchies. In: CVPR, Kawai, Hawaii (2001)
Sclaroff, S., Cascia, M.L., Sethi, S., Taycher, L.: Unifying textual and visual cues for content-based image retrieval on the world wide web. Computer Vision and Image Understanding 75(1-2), 86–98 (1999)
Carneiro, G., Vasconcelos, N.: Formulating Semantics Image Annotation as a Supervised Learning Problem. In: CVPR, San Diego (2005)
Vasconcelos, N.: Minimum Probability of Error Image Retrieval. IEEE Transactions on Signal Processing 52(8) (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rasiwasia, N., Vasconcelos, N., Moreno, P.J. (2006). Query by Semantic Example. In: Sundaram, H., Naphade, M., Smith, J.R., Rui, Y. (eds) Image and Video Retrieval. CIVR 2006. Lecture Notes in Computer Science, vol 4071. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11788034_6
Download citation
DOI: https://doi.org/10.1007/11788034_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-36018-6
Online ISBN: 978-3-540-36019-3
eBook Packages: Computer ScienceComputer Science (R0)