research-article

Constructing visual phrases for effective and efficient object-based image retrieval

Authors:

Qing-Fang Zheng,

Wen GaoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 5, Issue 1

Article No.: 7, Pages 1 - 19

https://doi.org/10.1145/1404880.1404887

Published: 30 October 2008 Publication History

Abstract

The explosion of multimedia data necessitates effective and efficient ways for us to get access to our desired ones. In this article, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects (object-based image retrieval). The visual phrase is defined as a pair of frequently co-occurred adjacent local image patches and is constructed using data mining. We design methods on how to construct visual phrase and how to index/search images based on visual phrase. We demonstrate experiments to show our visual phrase-based approach can be very efficient and more effective than current visual word-based approach.

References

[1]

Agrawal, R. and Srikant, R. 1994. Fast algorithms for mining association rules. In Proceeding of the 20th International Conference on VLDB, J. B. Bocca, M. Jarke, C., and Zaniolo, Eds. Morgan Kaufmann, Los Altos, CA. 487--499.

Digital Library

[2]

Agarwal, S., Awan, A., and roth, D. 2004. Learning to detect objects in images via a sparse, part-based representation. IEEE Trans. Patt. Anal. Machine Intell. 26, 11, 1475--1490.

Digital Library

[3]

Carson, C., Thomas, M., Belongie, S., and Malik, J. 2002. Blobworld: Image segmentation using expectation maximization and its application to image querying. IEEE Trans. Patt. Anal. Machine Intell. 24, 8, 1026--1038.

Digital Library

[4]

Ceglar, A. and Roddick, J. F. 2006. Association mining. ACM Comput. Surv. 38, 2.

Digital Library

[5]

Cox, I. J., Miller, M. L., Minka, T. P., Papathomas, T. V., and Yianilos, P. N. 2000. The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. 9, 1, 20--37.

Digital Library

[6]

Datta, R., Joshi, D., Li, J., and Wang, J. 2006. Image retrieval: ideas, influences, and trends of the new age. Penn Sate University Tech. rep. CSE 06--009.

[7]

Enser, P. G. B. and Sandom, C. J. 2003. Towards a comprehensive survey of the semantic gap in visual image retrieval. In Proceedings of the 2nd International Conference on Image and video retrieval. E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe, and X. Zhou, Eds. Springer-Verlag, London, UK. 291--299

Digital Library

[8]

Fergus, R., Li, F. --F., Perona, P., and Zisserman, A. 2005. Learning object categories from Google's image search. In Proceedings of the 10th IEEE International Conference on Computer Vision. IEEE Computer Society, Vol. 2, 1816--1823.

Digital Library

[9]

FlickneR, M., Sawheny, H., Niblack, W., Ashley, J., Qian Huang Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Comput. 28, 9, 23--32.

Digital Library

[10]

Garcya-Perez, D., Mosquera, A., Berretti, S., and Del Bimbo, A. 2006. Object-based image retrieval using active nets. In Proceedings of the 18th International Conference on Pattern Recognition, Vol. 4, 750--753.

Digital Library

[11]

Goethals, B. 2003. Survey on frequent pattern mining. Helsinki Institute for Information Technology tech. rep.

[12]

Grabner, M., Grabner, H., and Bischof, H. 2006. Fast Approximated SIFT. In Proceedings of 7th Asian Conference on Computer Vision, P. J. Narayanan, S. K. Nayar, H.--Y. Shum, Eds. Springer-Verlag, London, UK. 918--927.

Digital Library

[13]

Hammouda, K. M. and Kamel, M. S. 2004. Efficient phrase-based document indexing for web document clustering. IEEE Trans. Knowl. Data Engin. 16, 10, 1279--1296.

Digital Library

[14]

Hoiem, R., Sukthankar, H., Schneiderman, H., and Huston, L. 2004. Object-based image retrieval using the statistical structure of images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 490--497.

Digital Library

[15]

Jansen, M. B. J., Spink, A., and Saracevic, T. 2000. Real life, real users, and real needs: A study and analysis of user queries on the web. Inform. Process. Manag. Int. J. 36, 2, 207--227.

Digital Library

[16]

Jin, R, Chai, J. Y., and Si, L. 2004. Effective automatic image annotation via a coherent language model and active learning. In Proceedings of 12th ACM International Conference on Multimedia. ACM Press, New York, 892--899.

Digital Library

[17]

Jing, F., Li, M., Zhang, H.-J., and Zhang, B. 2004, An efficient and effective region-based image retrieval framework. IEEE Trans. Image Process. 13, 5, 699--709.

Digital Library

[18]

Jurie, F. and Triggs, B. 2005. Creating efficient codebooks for visual recognition. In Proceedings of International Conference on Computer Vision. IEEE Computer Society, Washington, DC, Vol. 1, 604--610.

Digital Library

[19]

Ke, Y., Sukthankar, R., and Huston, L. 2004. Efficient near-duplicate detection and sub-image retrieval. In Proceedings of the 12th ACM International Conference on Multimedia. ACM Press, New York, NY. 869--876.

Digital Library

[20]

Kherfi, M. L., Ziou, D., and Bernardi, A. 2004. Image retrieval from the World Wide Web: Issues, techniques, and systems. ACM Comput. Surv. 36, 1, 35--67.

Digital Library

[21]

Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Comm. Appl. 2, 1, 1--19.

Digital Library

[22]

Li, J. and Wang, J. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Patt. Anal. Mach. Intell. 25, 9, 1075--1088.

Digital Library

[23]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110.

Digital Library

[24]

Luo, J. and Nascimento, M. 2003. Content based subimage retrieval via hierarchical tree matching. In Proceedings of the 1st ACM International Workshop on Multimedia Databases. ACM Press, New York. 63--69.

Digital Library

[25]

Marszalek, M. and Schmid, C. 2006. Spatial weighting for bag-of-features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 2118--2125.

Digital Library

[26]

Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Trans. Patt. Anal. Mach. Intell. 27, 10, 1615--1630.

Digital Library

[27]

Pantofaru, C., Dorko, G., Schmid, C., and Hebert, M. 2006. Combining regions and patches for object class localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop. IEEE Computer Society Press, Washington, DC. 23--30

Digital Library

[28]

Papageorgious, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In Proceedings of the 6th International Conference on Computer Vision. IEEE Computer Society, Washington, DC. 555--562.

Digital Library

[29]

Pentland, A., Picard, R. W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. Int. J. Comput. Vision 18, 3, 233--254.

Digital Library

[30]

Rui, Y., Huang, T. S., and Chang, S.--F. 1999. Image retrieval: Current techniques, promising directions and open Issues. J. Visual Comm. Image Represent. 10, 1, 39--62.

Digital Library

[31]

Sebe, N., Lew, M., and Huijsmans, D. 1999. Multi-scale subimage search. In Proceedings of the 7th ACM International Conference on Multimedia. ACM Press, New York. 79--82.

Digital Library

[32]

Sivic, J. and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the 9th IEEE International Conference on Computer Vision. IEEE Computer Society, Washington, DC. Vol. 2, 1470--1477.

Digital Library

[33]

Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 12, 1349--1380.

Digital Library

[34]

Smith, J. R. and Chang, S.--F. 1996. Querying by color regions using the VisualSEEK content-based visual query system. In Intell. Multimed. Inform. Retri. Mark T. Maybury, Ed. AAAI Press, 23--42.

Digital Library

[35]

Smyth, W. F., Lam, C. P., Chen, X., and Maxville, V. 2003. Heuristics for image retrieval using spatial configurations. In Proceedings of the 7th Digital Image Computing: Techniques and Applications. Sydney. 909--918.

[36]

Squire, D., Muller, W., Muller, H., and Pun, T. 2000. Content-based visual query of image databases: Inspirations from text retrieval. Patt. Recogn. Lett. 21, 13--14, 1193--1198.

Digital Library

[37]

Sun, Z, Bebis, G., and Miller, R. 2006. On-road vehicle detection: A review. IEEE Trans. Patt. Anal. Mach. Intell. 28, 5, 694--711.

Digital Library

[38]

Sung, K.--K. and Poggio, T. 1998, Example-based learning for view-based face detection. IEEE Trans. Patt. Anal. Mach. Intell. 20, 1, 39--51.

Digital Library

[39]

Veltkamp, R. C. and Tanase, M. 2000. Content-based image retrieval systems: A survey. Utrecht University technical rep. UU-CS-2000-34.

[40]

Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. 511--518.

[41]

Wang, G., Zhang, Y., and Li, F.-F. 2006. Using dependent regions for object categorization in generative framework. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 1597--1604.

Digital Library

[42]

Williams, H. E., Zobel, J., and BAHLE, D. 2004. Fast phrase querying with combined indexes. ACM Trans. Inform. Syst. 22, 4, 573--594.

Digital Library

[43]

Winn, J., Criminisi, A., and Minka, T. 2005. Object categorization by learned universal visual dictionary. In Proceedings of 10th IEEE Conference on Computer Vision. IEEE Computer Society, Washington DC, Vol. 2, 1800--1807.

Digital Library

[44]

Witter, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, Los Altos, CA.

Digital Library

[45]

Yan, X., Yu, P. S., and Han, J. 2005. Graph indexing based on discriminative frequent structure analysis. ACM Trans. Datab. Syst. 30, 4, 960--993.

Digital Library

[46]

Yang, M.--H., Kriegman, D. J., and Ahuja, N. 2002. Detecting faces in images: A survey. IEEE Trans. Patt. Anal. Mach. Intell. 24, 1, 34--58.

Digital Library

[47]

Yeh, T., Tollmar, K., and Darrell, T. 2004. Searching the Web with mobile images for location recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 76--81.

Digital Library

[48]

Zheng, Q.-F, Wang, W.-Q, and Gao, W. 2006. Effective and efficient object-based images retrieval using visual phrases. In Proceedings of the 14th ACM International Conference on Multimedia. ACM Press, New York. 77--80.

Digital Library

Cited By

Karthika NJanet B(2019)Feature Pair Index Graph for ClusteringJournal of Intelligent Systems10.1515/jisys-2018-033829:1(1179-1187)Online publication date: 15-Jan-2019
https://doi.org/10.1515/jisys-2018-0338
Asim MWasim MGhani Khan MMahmood NMahmood W(2019)The Use of Ontology in Retrieval: A Study on Textual, Multilingual, and Multimedia RetrievalIEEE Access10.1109/ACCESS.2019.28978497(21662-21686)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2897849
Zhang YWang JLi J(2015)Parallel Massive Clustering of Discrete DistributionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/270029311:4(1-24)Online publication date: 2-Jun-2015
https://dl.acm.org/doi/10.1145/2700293
Show More Cited By

Index Terms

Constructing visual phrases for effective and efficient object-based image retrieval
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Search engine architectures and scalability
      1. Search engine indexing
  2. Information systems applications
    1. Multimedia information systems
      1. Multimedia databases

Recommendations

Effective and efficient object-based image retrieval using visual phrases
MM '06: Proceedings of the 14th ACM international conference on Multimedia

In this paper, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects. The visual phrase is defined as a pair of adjacent local image patches and is ...
Toward a higher-level visual representation for object-based image retrieval

We propose a higher-level visual representation, visual synset, for object-based image retrieval beyond visual appearances. The proposed visual representation improves the traditional part-based bag-of-words image representation, in two aspects. First, ...
Spatial Similarity Measure of Visual Phrases for Image Retrieval
MMM 2014: Proceedings of the 20th Anniversary International Conference on MultiMedia Modeling - Volume 8326

Spatial information plays an essential role in accurate matching of local features in applications, e.g., image retrieval. Despite of previous work, it remains a challenging problem to extract appropriate spatial information. We propose an image ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 5, Issue 1

October 2008

201 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/1404880

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2008

Accepted: 01 December 2007

Revised: 01 May 2007

Received: 01 December 2006

Published in TOMM Volume 5, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
982
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karthika NJanet B(2019)Feature Pair Index Graph for ClusteringJournal of Intelligent Systems10.1515/jisys-2018-033829:1(1179-1187)Online publication date: 15-Jan-2019
https://doi.org/10.1515/jisys-2018-0338
Asim MWasim MGhani Khan MMahmood NMahmood W(2019)The Use of Ontology in Retrieval: A Study on Textual, Multilingual, and Multimedia RetrievalIEEE Access10.1109/ACCESS.2019.28978497(21662-21686)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2897849
Zhang YWang JLi J(2015)Parallel Massive Clustering of Discrete DistributionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/270029311:4(1-24)Online publication date: 2-Jun-2015
https://dl.acm.org/doi/10.1145/2700293
El Sayad IMartinet JZhang ZEisert P(2014)Multilayer Semantic Analysis in Image DatabasesReal World Data Mining Applications10.1007/978-3-319-07812-0_19(387-414)Online publication date: 14-Nov-2014
https://doi.org/10.1007/978-3-319-07812-0_19
Chen JFeng BXu B(2014)Spatial Similarity Measure of Visual Phrases for Image RetrievalProceedings of the 20th Anniversary International Conference on MultiMedia Modeling - Volume 832610.1007/978-3-319-04117-9_25(275-282)Online publication date: 6-Jan-2014
https://dl.acm.org/doi/10.1007/978-3-319-04117-9_25
Wang LSong DElyan EChen XLebanon GWang HZaki M(2012)Improving bag-of-visual-words model with spatial-temporal correlation for video retrievalProceedings of the 21st ACM international conference on Information and knowledge management10.1145/2396761.2398433(1303-1312)Online publication date: 29-Oct-2012
https://dl.acm.org/doi/10.1145/2396761.2398433
Qi HLi KShen YQu W(2012)Object-based image retrieval with kernel on adjacency matrix and local combined featuresACM Transactions on Multimedia Computing, Communications, and Applications10.1145/2379790.23797968:4(1-18)Online publication date: 30-Nov-2012
https://dl.acm.org/doi/10.1145/2379790.2379796
van Zwol RGarcia Pueyo LAdar ETeevan JAgichtein EMaarek Y(2012)Spatially-aware indexing for image object retrievalProceedings of the fifth ACM international conference on Web search and data mining10.1145/2124295.2124299(3-12)Online publication date: 8-Feb-2012
https://dl.acm.org/doi/10.1145/2124295.2124299
Chen JFeng BZhu LDing PXu B(2012)Effective near-duplicate image retrieval with image-specific visual phrase selection2012 19th IEEE International Conference on Image Processing10.1109/ICIP.2012.6467258(1909-1912)Online publication date: Sep-2012
https://doi.org/10.1109/ICIP.2012.6467258
El Sayad IMartinet JUrruty TDjeraba C(2012)Toward a higher-level visual representation for content-based image retrievalMultimedia Tools and Applications10.1007/s11042-010-0596-x60:2(455-482)Online publication date: 1-Sep-2012
https://dl.acm.org/doi/10.1007/s11042-010-0596-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents