Article

A mutual semantic endorsement approach to image retrieval and context provision

Author:

Jia LiAuthors Info & Claims

MIR '05: Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval

Pages 173 - 182

https://doi.org/10.1145/1101826.1101856

Published: 10 November 2005 Publication History

Abstract

Learning semantics from annotated images to enhance content-based retrieval is an important research direction. In this paper,annotation data are assumed available for only a subset of images inside the database. An on the fly learning method is developed to capture the semantics of query images. Specifically, the semantics of annotated images in a visual proximity of a query are compared with each other to determine the amount of mutual endorsement. An image is considered endorsed by another if they possess similar semantics. Annotations with high mutual endorsement are used to narrow down a candidate pool of images. The new retrieval method is inherently dynamic and treats seamlessly different forms of annotation data. Experiments show that semantic endorsement can increase precision by as much as 70%in average for a wide range of parameter settings. We also develop a context provision mechanism to reveal the relationship between a query and semantic clusters extracted from the database. Context helps users explore the content of a database and provides a platform for them to tailor searches by stressing different perspectives in the interpretation of a query.

References

[1]

K. Barnard, P. Duygulu, N. deFreitas, D.A. Forsyth, D.M. Blei, and M.I. Jordan, "Matching words and pictures," Journal of Machine Learning Research, 3:1107--1135, 2003.

Digital Library

[2]

N. Boujemaa, J. Fauqueur, and V. Gouet "What's beyond query by example?", Trends and Advances in Content-Based Image and Video Retrieval, L. Shapiro, et al., eds., LNCS, Springer Verlag, 2004.

[3]

T.F. Cox and M.A. Cox, Multidimensional Scaling, Chapman & Hall, 1994.

[4]

I. Daubechies, Ten Lectures on Wavelets, Capital City Press, 1992.

Digital Library

[5]

P. Duygulu, K. Barnard, N. deFreitas, and D. Forsyth, "Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary," Proc. ECCV, 4:97--112, 2002.

Digital Library

[6]

A.K. Jain andR.C. Dubes, Algorithms for Clustering Data, Prentice-Hall, Englewood Cliffs, NJ, 1988.

Digital Library

[7]

J.J. Jiang and D.W. Conrath, "Semantic similarity based on corpus statistics and lexical taxonomy," Proc. Intl. Conf. on Research in Computational Linguistics, 1997.

[8]

D. Joshi, J.Z. Wang, and J. Li, "The story picturing engine: finding elite images to illustrate a story using mutual reinforcement," Proc. Int. Workshop on Multimedia Information Retrieval, New York, 2004.

Digital Library

[9]

B. Li, K.-S. Goh, E.Y. Chang, "Confidence-based dynamic ensemble for image annotation and semantics discovery," Proc. ACM Multimedia, Berkeley, CA, November 2003.

Digital Library

[10]

J. Li, R.M. Gray, and R.A. Olshen, "Multiresolution image classification by hierarchical modeling with two dimensional hidden Markov models," IEEE Trans. on Information Theory, 46(5):1826--41, 2000.

Digital Library

[11]

J. Li and J.Z. Wang, "Automatic linguistic indexing of pictures by a statistical modeling approach," IEEE Trans. PAMI, 25(9):1075--1088, 2003.

Digital Library

[12]

J. Li, J.Z. Wang, G. Wiederhold, "IRM:Integrated region matching for image retrieval," Proc.ACM Multimedia, pp.147--156, Los Angeles, Oct. 2000.

Digital Library

[13]

A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Trans. PAMI, 22(12):1349--1380, 2000.

Digital Library

[14]

A. Vailaya, M.A.T. Figueiredo, A.K. Jain, and H.-J. Zhang, "Image classification for content-based indexing," IEEE Trans.Image Processing, 10(1):117--130, 2001.

Digital Library

[15]

J.Z. Wang, J. Li, and G. Wiederhold, "SIMPLIcity: Semantics-sensitive Integrated Matching for Picture LIbraries," IEEE Trans.PAMI, 23(9):947--963, 2001.

Digital Library

[16]

R. Zhang, Z. Zhang, "Hidden semantic concept discovery in region based image retrieval," CVPR, Washington D.C., June 2004.

Digital Library

Cited By

Verma YGupta AMannem PJawahar C(2013)Generating Image Descriptions Using Semantic Similarities in the Output SpaceProceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops10.1109/CVPRW.2013.50(288-293)Online publication date: 23-Jun-2013
https://dl.acm.org/doi/10.1109/CVPRW.2013.50
Fan WBouguila N(2013)Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selectionPattern Recognition10.1016/j.patcog.2013.03.02646:10(2754-2769)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1016/j.patcog.2013.03.026
Zhu S(2008)Improving image retrieval effectiveness via random walk with restartOptical Engineering10.1117/1.302748147:11(117003)Online publication date: 1-Nov-2008
https://doi.org/10.1117/1.3027481
Show More Cited By

Index Terms

A mutual semantic endorsement approach to image retrieval and context provision
1. Computing methodologies
2. Information systems
  1. Information retrieval

Recommendations

A Context Centric Approach for Semantic Image Annotation and Retrieval
COMPUTATIONWORLD '09: Proceedings of the 2009 Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns

In this preliminary research, we discuss techniques to improve the quality of image retrieval and image management with the help of context information over the web. Our hypothesis is that leveraging the semantic annotated contextual metadata of the ...
Semantic text-based image retrieval with multi-modality ontology and DBpedia

PurposeThe purpose of this study is to reduce the semantic distance by proposing a model for integrating indexes of textual and visual features via a multi-modality ontology and the use of DBpedia to improve the comprehensiveness of the ontology to ...
A new approach for interactive semantic image retrieval using the high level semantics
SAC '08: Proceedings of the 2008 ACM symposium on Applied computing

Content-based image retrieval and text-based image retrieval are two Fundamental approaches in the field of image retrieval. Recently, the researchers use the combining approaches and semiautomatic image retrieval, using the user interaction in the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MIR '05: Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval

November 2005

274 pages

ISBN:1595932445

DOI:10.1145/1101826

General Chairs:
Hongjiang Zhang
Microsoft Research Advanced Technology Center, China
,
John Smith
IBM T. J. Watson Research Center, Hawthorne, NY, USA
,
Qi Tian
University of Texas at San Antonio, San Antonio, TX, USA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM&Sec '05

Sponsor:

MM&Sec '05: Multimedia and Security Workshop 2005

November 10 - 11, 2005

Hilton, Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
278
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Verma YGupta AMannem PJawahar C(2013)Generating Image Descriptions Using Semantic Similarities in the Output SpaceProceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops10.1109/CVPRW.2013.50(288-293)Online publication date: 23-Jun-2013
https://dl.acm.org/doi/10.1109/CVPRW.2013.50
Fan WBouguila N(2013)Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selectionPattern Recognition10.1016/j.patcog.2013.03.02646:10(2754-2769)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1016/j.patcog.2013.03.026
Zhu S(2008)Improving image retrieval effectiveness via random walk with restartOptical Engineering10.1117/1.302748147:11(117003)Online publication date: 1-Nov-2008
https://doi.org/10.1117/1.3027481
Tao Mei Yong Wang Xian-Sheng Hua Shaogang Gong Shipeng Li (2008)Coherent image annotation by learning semantic distance2008 IEEE Conference on Computer Vision and Pattern Recognition10.1109/CVPR.2008.4587386(1-8)Online publication date: Jun-2008
https://doi.org/10.1109/CVPR.2008.4587386
Wu XZhao WNgo CSebe NWorring M(2007)Near-duplicate keyframe retrieval with visual keywords and semantic contextProceedings of the 6th ACM international conference on Image and video retrieval10.1145/1282280.1282309(162-169)Online publication date: 9-Jul-2007
https://dl.acm.org/doi/10.1145/1282280.1282309
Datta RGe WLi JWang J(2007)Toward Bridging the Annotation-Retrieval Gap in Image SearchIEEE MultiMedia10.1109/MMUL.2007.6714:3(24-35)Online publication date: 1-Jul-2007
https://dl.acm.org/doi/10.1109/MMUL.2007.67
Datta RGe WLi JWang JNahrstedt KTurk MRui YKlas WMayer-Patel K(2006)Toward bridging the annotation-retrieval gap in image search by a generative modeling approachProceedings of the 14th ACM international conference on Multimedia10.1145/1180639.1180856(977-986)Online publication date: 23-Oct-2006
https://dl.acm.org/doi/10.1145/1180639.1180856

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents