research-article

Multimedia search reranking: A literature survey

Authors:

Qi TianAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 46, Issue 3

Article No.: 38, Pages 1 - 38

https://doi.org/10.1145/2536798

Published: 01 January 2014 Publication History

Abstract

The explosive growth and widespread accessibility of community-contributed media content on the Internet have led to a surge of research activity in multimedia search. Approaches that apply text search techniques for multimedia search have achieved limited success as they entirely ignore visual content as a ranking signal. Multimedia search reranking, which reorders visual documents based on multimodal cues to improve initial text-only searches, has received increasing attention in recent years. Such a problem is challenging because the initial search results often have a great deal of noise. Discovering knowledge or visual patterns from such a noisy ranked list to guide the reranking process is difficult. Numerous techniques have been developed for visual search re-ranking. The purpose of this paper is to categorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics, and benchmarking. We conclude with several promising directions for future research.

References

[1]

A. Amir, Janne Argillander, M. Campbell, A. Haubold, G. Iyengar, S. Ebadollahi, and F. Kang. 2005. IBM Research TRECVID-2005 video retrieval system. In Proceedings of the TRECVID Workshop.

[2]

R. Baeza-Yates and B. Ribeiro-Neto. 1999. Modern Information Retrieval. Addison Wesley.

Digital Library

[3]

N. Ben-Haim, B. Babenko, and S. Belongie. 2006. Improving web-based image search via content based clustering. In Proceedings of Computer Vision and Pattern Recognition Workshop on SLAM.

Digital Library

[4]

M. Bendersky and O. Kurland. 2008. Re-ranking search results using document-passage graphs. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[5]

A. Benitez, M. Beigi, and S.-F. Chang. 1998. Using relevance feedback in content-based image metasearch. IEEE Internet Computing 2, 4 (1998), 59--69.

Digital Library

[6]

T. L. Berg and D. A. Forsyth. 2006. Animals on the web. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1463--1470.

Digital Library

[7]

Bing. 2013. Homepage. Retrieved from http://www.bing.com/.

[8]

T. Bogers and A. Bosch. 2009. Authoritative ReRanking in fusing authorship-based subcollection search results. In Proceedings of Dutch-Belgian Information Retrieval Workshop. 49--55.

[9]

S. Boll. 2007. MultiTube—where multimedia and Web 2.0 could meet. IEEE Multimedia 14, 1 (Jan.-March 2007), 9--13.

Digital Library

[10]

S. Brin and L. Page. 1998a. The anatomy of a large-scale hyper textual Web search engine. Computer Networks and ISDN Systems 30, 1--7 (1998), 107--117.

Digital Library

[11]

S. Brin and L. Page. 1998b. The anatomy of a large-scale hypertextual Web search engine. In Proceedings of the International Conference on World Wide Web.

Digital Library

[12]

M. Brinkmeier. 2006. The anatomy of a large-scale hypertextual Web search engine. In Proceedings of the International Conference on World Wide Web. 282--301.

[13]

D. Cai, X. He, Z. Li, W.-Y. Ma, and J.-R. Wen. 2004. Hierarchical clustering of WWW image search results using visual, textual and link information. In Proceedings of ACM Multimedia. 952--959.

Digital Library

[14]

J. Cao, Y. D. Zhang, Y. C. Song, Z. N. Chen, X. Zhang, and J. T. Li. 2009. MCG-WEBV: A benchmark dataset for web video analysis. Technical Report ICT-MCG-09-001. Beijing, China.

[15]

Yang Cao, Changhu Wang, Liqing Zhang, and Lei Zhang. 2011. Edgel index for large-scale sketch-based image search. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. 761--768.

Digital Library

[16]

Yang Cao, Hai Wang, Changhu Wang, Zhiwei Li, Liqing Zhang, and Lei Zhang. 2010. MindFinder: Interactive sketch-based image search on millions of images. In Proceedings of ACM Multimedia.

Digital Library

[17]

Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li. 2007. Learning to rank: From pair-wise approach to list-wise approach. In Proceedings of International Conference on Machine Learning. 129--136.

Digital Library

[18]

J. Carbonell. 1997. Translingual information retrieval: A comparative evaluation. In Proceedings of International Joint Conference on Artificial Intelligence.

[19]

J. Carbonell and J. Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[20]

S.-F. Chang, W. Hsu, W. Jiang, L. Kennedy, D. Xu, A. Yanagawa, and E. Zavesky. 2006. Columbia University TRECVID-2006 video search and high-level feature extraction. In TREC Video Retrieval Evaluation Online Proceedings.

[21]

S.-F. Chang, W.-Y. Ma, and A. Smeulders. 2007. Recent advances and challenges of semantic image/video search. In Proceedings of the IEEE International Conferenc on Acoustics Speech and Signal Processing (ICASSP’07).

[22]

S. Chen, F. Wang, Y. Song, and C. Zhang. 2011. Semi-supervised ranking aggregation. Information Processing & Management 47, 3 (May 2011), 415--425.

Digital Library

[23]

T.-S. Chua, S.-Y. Neo, K.-Y. Li, G. Wang, R. Shi, M. Zhao, and H. Xu. 2004. TRECVID 2004 search and feature extraction task by NUS PRIS. In TREC Video Retrieval Evaluation Online Proceedings.

[24]

Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. 2009. NUS-WIDE: A real-world Web image database from National University of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval.

Digital Library

[25]

Ondrej Chum, Andrej Mikulik, Michal Perdoch, and Jiri Matas. 2011. Total Recall II: Query expansion revisited. In Proceedings of CVPR.

Digital Library

[26]

Ondrej Chum, James Philbin, Josef Sivic, Michael Isard, and Andrew Zisserman. 2007. Total Recall: Automatic query expansion with a generative feature model for object retrieval. In Proceedings of ICCV.

[27]

W. B. Croft. 2000. Combining approaches to information retrieval. In Advanced in Information Retrieval. 1--36.

[28]

J. Cui, F. Wen, and X. Tang. 2008a. IntentSearch: Interactive on-line image search re-ranking. In Proceedings of ACM Multimedia. 997--998.

Digital Library

[29]

J. Cui, F. Wen, and X. Tang. 2008b. Real time Google and live image search re-ranking. In Proceedings of ACM Multimedia. 729--732.

Digital Library

[30]

R. Datta, D. Joshi, J. Li, and J. Z. Wang. 2008. Image retrieval: Ideas, influences, and trends of the new age. Comput. Surveys 40, 65 (2008).

Digital Library

[31]

H. Deng, M. R. Lyu, and I. King. 2009b. Effective latent space graph-based re-ranking model with global consistency. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining.

Digital Library

[32]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009a. A large-scale hierarchical image database. In Proceedings of IEEE CVPR.

[33]

H. Ding, J. Liu, and H. Lu. 2008. Hierarchical clustering-based navigation of image search results. In Proceedings of ACM Multimedia. 741--744.

Digital Library

[34]

K. M. Donald and A. F. Smeaton. 2005. A comparison of score, rank and probability-based fusion methods for video shot retrieval. In Proceedings of ACM International Conference on Image and Video Retrieval.

Digital Library

[35]

C. Dwork, S. R. Kumar, M. Naor, and D. Sivakumar. 2001. Rank aggregation methods for the Web. In Proceedings of the International World Wide Web Conference. 613--622.

Digital Library

[36]

S. Ebadollahi, D. Joshi, M. Naphade, A. Natsev, J. Seidl, J. R. Smith, K. Scheinberg, J. Tesic, and L. Xie. 2006. IBM Research TRECVID-2006 video retrieval system. In TREC Video Retrieval Evaluation Online Proceedings.

[37]

M. Eitz, K. Hildebrand, T. Boubekeur, and M. Alexa. 2011. Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors. IEEE Trans. on Visualization and Computer Graphics 17, 11 (Nov. 2011), 1624--1636.

Digital Library

[38]

M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. The PASCAL visual object classes (VOC) challenge. International Journal of Computer Vision 88, 2 (2010), 303--338.

Digital Library

[39]

Facebook. 2013. Homepage. Retreived from http://www.facebook.com/.

[40]

R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman. 2005. Learning object categories from Google’s image search. In Proceedings of International Conference on Computer Vision. 1816--1823.

Digital Library

[41]

R. Fergus, P. Perona, and A. Zisserman. 2004. A visual category filter for Google images. In Proceedings of the European Conference on Computer Vision.

[42]

Flickr. 2013. Homepage. Retrieved from http://www.flickr.com/.

[43]

J. Fogarty, A. Kapoor, D. Tan, and S. Winder. 2007. CueFlik: Interactive concept learning in image search. In Proceeding of the SIGCHI Conference on Human Factors in Computing Systems. 29--38.

Digital Library

[44]

Bo Geng, Linjun Yang, Chao Xu, Xian-Sheng Hua, and Shipeng Li. 2011. The role of attractiveness in web image search. In Proceedings of ACM Multimedia. 63--72.

Digital Library

[45]

Google. 2013. Homepage. Retrieved from http://www.google.com/.

[46]

A. Haubold, A. Natsev, and M. R. Naphade. 2006. Semantic multimedia retrieval using lexical query expansion and model-based reranking. In Proceedings of the IEEE International Conference on Multimedia & Expo.

[47]

A. Hauptmann, M.-Y. Chen, M. Christel, C. Huang, W.-H. Lin, T. Ng, N. Papernick, A. Velivelli, J. Yang, R. Yan, H. Yang, and H. D. Wactlar. 2004. Confounded expectations: Informedia at TRECVID 2004. In TREC Video Retrieval Evaluation Online Proceedings.

[48]

A. G. Hauptmann, M. Christel, and R. Yan. 2008a. Video retrieval based on semantic concepts. In Proceedings of the IEEE. 602--622.

[49]

A. G. Hauptmann and M. G. Christel. 2004. Successful approaches in the TREC video retrieval evaluations. In Proceedings of ACM Multimedia. 668--675.

Digital Library

[50]

A. G. Hauptmann, M. G. Christel, and R. Yan. 2008b. Video retrieval based on semantic concepts. Proc. IEEE 96, 4 (April 2008), 602--622.

[51]

A. G. Hauptmann, W. H. Lin, R. Yan, J. Yang, and M. Y. Chen. 2006. Extreme video retrieval: Joint maximization of human and computer performance. In Proceedings of the ACM International Conference on Multimedia.

Digital Library

[52]

A. G. Hauptmann, R. Yan, W.-H. Lin, M. Christel, and H. Wactlar. 2007. Can high-level concepts fill the semantic gap in video retrieval&quest; A case study with broadcast news. IEEE Transaction on Multimedia 9, 5 (2007), 958--966.

Digital Library

[53]

T. H. Haveliwala. 2002. Topic-sensitive PageRank. In Proceedings of the International Conference on World Wide Web.

Digital Library

[54]

Jingrui He, Changshui Zhang, Nanyuan Zhao, and Hanghang Tong. 2005. Boosting web image search by co-ranking. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing. 409--412.

[55]

R. Herbrich, T. Graepel, and K. Obermayer. 2000. Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers (2000), 115--132.

[56]

S. C. Hoi and M. R. Lyu. 2007. A multimodal and multilevel ranking framework for content-based video retrieval. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing. 1520--6149.

[57]

R. Hong, G. Li, L. Nie, J. Tang, and T.-S. Chua. 2010. Exploring large scale data for multimedia QA: An initial study. In Proceedings of ACM International Conference on Image and Video Retrieval. 74--81.

Digital Library

[58]

W. Hsu and S.-F. Chang. 2007. Video search reranking through random walk over document-level context graph. In Proceedings of ACM Multimedia. 971--980.

Digital Library

[59]

W. Hsu, L. Kennedy, and S.-F. Chang. 2007. Reranking methods for visual search. IEEE Multimedia 14, 3 (July-Sept. 2007), 14--22.

Digital Library

[60]

W. H. Hsu, L. S. Kennedy, and S.-F. Chang. 2006. Video search reranking via information bottleneck principle. In Proceedings of the ACM International Conference on Multimedia.

Digital Library

[61]

Y. Hu, N. Yu, Z. Li, and M. Li. 2007. Image search result clustering and re-ranking via partial grouping. In Proceedings of the IEEE International Conference on Multimedia & Expo. 603--606.

[62]

Gang Hua and Qi Tian. 2009. What can visual content analysis do for text based image search&quest; In Proceedings of the IEEE International Conference on Multimedia & Expo. 1480--1483.

Digital Library

[63]

M. J. Huiskes and M. S. Lew. 2008. The MIR Flickr retrieval evaluation. In Proceedings of the ACM International Conference on Multimedia Information Retrieval.

Digital Library

[64]

M. J. Huiskes, B. Thomee, and M. S. Lew. 2010. New trends and ideas in visual concept detection: The MIR Flickr retrieval evaluation initiative. In Proceedings of ACM International Conference on Multimedia Information Retrieval.

Digital Library

[65]

ImageCLEF. 2013. ImageCLEF—The CLEack. Retrieved from http://www.imageclef.org/.

[66]

ImageNet. 2013. ImageNet. Retrieved from http://www.image-net.org/.

[67]

V. Jain and M. Varma. 2011. Learning to re-rank: Query-dependent image re-ranking using click data. In Proceedings of International World Wide Web Conference.

Digital Library

[68]

K. Jarvelin and J. Kekalainen. 2000. IR evaluation methods for retrieving highly relevant documents. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[69]

Kalervo Jarvelin and Jaana Kekalainen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4 (2002), 422--446.

Digital Library

[70]

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Improving bag-of-features for large scale image search. International Journal of Computer Vision 87, 3 (May 2010), 316--336.

Digital Library

[71]

F. Jing, C. Wang, Y. Yao, K. Deng, L. Zhang, and W.-Y. Ma. 2006. IGroup: Web image search results clustering. In Proceedings of ACM Multimedia. 377--384.

Digital Library

[72]

Y. Jing and S. Baluja. 2008a. PageRank for product image search. In Proceedings of the International World Wide Web Conference. 307--315.

Digital Library

[73]

Yushi Jing and Shumeet Baluja. 2008b. VisualRank: Applying PageRank to large-scale image search. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 11 (Nov. 2008), 1877--1890.

Digital Library

[74]

L. Kennedy and S.-F Chang. 2007. A reranking approach for context-based concept fusion in video indexing and retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval.

Digital Library

[75]

L. Kennedy, S.-F. Chang, and A. Natsev. 2008a. Query-adaptive fusion for multimodal search. Proceedings of IEEE 96, 4 (2008), 567--588.

[76]

L. Kennedy, S.-F. Chang, and A. Natsev. 2008b. Query-adaptive fusion for multimodal search. Proc. IEEE 96, 4 (April 2008), 567--588.

[77]

L. Kennedy and M. Naaman. 2008. Generating diverse and representative image search results for landmarks. In Proceedings of the International World Wide Web Conference. 297--306.

Digital Library

[78]

L. Kennedy, M. Naaman, S. Ahern, R. Nair, and T. Rattenbury. 2007. How Flickr helps us make sense of the world: Context and content in community-contributed media collections. In Proceedings of ACM Multimedia.

Digital Library

[79]

J. Koren, Y. Zhang, and X. Liu. 2008. Personalized interactive faceted search. In Proceedings of WWW.

Digital Library

[80]

Josip Krapac, Moray Allan, Jakob Verbeek, and Frdric Jurie. 2010. Improving web-image search results using query-relative classifiers. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 1094--1101.

[81]

Oren Kurland and Lillian Lee. 2005. PageRank without hyperlinks: Structural re-ranking using links induced by language models. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 306--313.

Digital Library

[82]

K.-S. Lee, Y.-C. Park, and K.-S. Choi. 2001. Re-ranking model based on document clusters. Information Processing & Management 37, 1 (Jan. 2001), 1--14.

Digital Library

[83]

M. S. Lew. 2000. Next-generation web searches for visual content. IEEE Computer Society 3, 11 (2000), 46--53.

Digital Library

[84]

M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Transactions on Multimedia Computing, Communications and Applications 2, 1 (February 2006), 1--19.

Digital Library

[85]

H. Li, M. Wang, and X.-S. Hua. 2009. MSRA-MM 2.0: A large-scale web multimedia dataset. In Proceedings of the ICDM Workshop on Internet Multimedia Mining.

Digital Library

[86]

Jia Li, Shih-Fu Chang, Michael Lesk, Rainer Lienhart, Jiebo Luo, and Arnold W. M. Smeulders. 2007. New challenges in multimedia research for the increasingly connected and fast growing digital society. In Proceedings of the ACM SIGMM International Workshop on Multimedia Information Retrieval. 3--10.

Digital Library

[87]

L. J. Li, H. Su, Y. Lim, and L. Fei-Fei. 2010b. Objects as attributes for scene classification. In Proceedings of the European Conference of Computer Vision, International Workshop on Parts and Attributes.

Digital Library

[88]

L.-J. Li, H. Su, E. P. Xing, and L. Fei-Fei. 2010c. Object bank: A high-level image representation for scene classification and semantic feature sparsification. In Proceedings of the Neural Information Processing Systems.

[89]

X. Li, C. G. M. Snoek, and M. Worring. 2010a. Unsupervised multi-feature tag relevance learning for social image retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval. 10--17.

Digital Library

[90]

X. Li, D. Wang, J. Li, and B. Zhang. 2007. Video search in concept subspace: A text-like paradigm. In Proceedings of ACM International Conference on Image and Video Retrieval. 603--610.

Digital Library

[91]

J. Lin. 2008. PageRank without Hyperlinks: Reranking with related document networks. In Technical Report LAMP-TR-146.

[92]

Jingjing Liu, Wei Lai, Xian-Sheng Hua, Yalou Huang, and Shipeng Li. 2007a. Video search re-ranking via multi-graph propagation. In Proceedings of ACM Multimedia. 208--217.

Digital Library

[93]

Tie-Yan Liu. 2009. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval 3, 3 (2009), 225--331.

Digital Library

[94]

X. Liu, Z. Li, Z. Shi, and Z. Shi. 2009a. Filter object categories: Employing visual consistency and semi-supervised approach. In IEEE International Conference on Multimedia & Expo. 678--681.

Digital Library

[95]

Y. Liu and T. Mei. 2011. Optimizing visual search reranking via pairwise learning. IEEE Trans. on Multimedia 13, 2 (April 2011), 280--291.

Digital Library

[96]

Yuan Liu, Tao Mei, and Xian-Sheng Hua. 2009b. CrowdReranking: Exploring multiple search engines for visual search reranking. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 500--507.

Digital Library

[97]

Y. Liu, T. Mei, X.-S. Hua, J. Tang, X. Wu, and S. Li. 2008a. Learning to video search rerank via pseudo preference feedback. In Proceedings of the IEEE International Conference on Multimedia & Expo.

[98]

Y. Liu, T. Mei, M. Wang, X. Wu, and X.-S. Hua. 2010. Typicality-based image search reranking. IEEE Trans. on Circuits System and Video Technology 20, 5 (May 2010), 749--755.

Digital Library

[99]

Y. Liu, T. Mei, X. Wu, and X.-S. Hua. 2008b. An optimization-based framework for video search re-ranking. In Proceedings of the ACM SIGMM Workshop on Multimedia Information Retrieval.

[100]

Y.-T. Liu, T.-Y. Liu, T. Qin, Z.-M. Ma, and H. Li. 2007b. Supervised Rank Aggregation. In Proceedings of the International World Wide Web Conference. 481--489.

Digital Library

[101]

D. G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (2004), 91--110.

Digital Library

[102]

Wenhao Lu, Jingdong Wang, Xian-Sheng Hua, Shengjin Wang, and Shipeng Li. 2011. Robust visual reranking via sparsity and ranking constraints. In Proceedings of ACM Multimedia. 513--522.

Digital Library

[103]

T. Mei, X.-S. Hua, W. Lai, L. Yang, and et al. 2007. MSRA-USTC-SJTU at TRECVID 2007: High-level feature extraction and search. In TREC Video Retrieval Evaluation Online Proceedings.

[104]

T. Mei and Y. Rui. 2009. Image similarity. In Ling Liu and M. Tamer Ozsu (Eds.), Encyclopedia of Database Systems, Springer. 1379--1384 pages.

[105]

Nobuyuki Morioka and Jingdong Wang. 2011. Robust visual reranking via sparsity and ranking constraints. In Proceedings of ACM Multimedia. 533--542.

Digital Library

[106]

D. Myoupo, A. Popescu, H. L. Borgne, and P.-A. Moellic. 2009. Visual reranking for image retrieval over the Wikipedia corpus. In CLEF.

[107]

S.-H. Na, I.-S. Kang, and J.-H. Lee. 2008. Structural re-ranking with cluster-based retrieval. In Advanced in Information Retrieval. 658--662.

Digital Library

[108]

M. Naphade, J. R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis. 2006. Large-scale concept ontology for multimedia. IEEE MultiMedia 13, 3 (2006), 86--91.

Digital Library

[109]

A. Natsev, A. Haubold, J. Tesic, Lexing Xie, and R. Yan. 2007. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In Proceedings of ACM Multimedia. 991--1000.

Digital Library

[110]

C. W. Ngo. 2009. VIREO/DVMM at TRECVID 2009: High-level feature extraction, automatic video search, and content-Based copy detection. In TREC Video Retrieval Evaluation Online Proceedings.

[111]

C.-W. Ngo, S. A. Zhu, W. Zhang, C.-C. Tan, T. Yao, L. Pang, and H.-K. Tan. 2011. VIREOTRECVID 2011: Instance search, semantic indexing, multimedia event detection and known-item search. In TREC Video Retrieval Evaluation Online Proceedings.

[112]

D. Nister and H. Stewenius. 2006. Scalable recognition with a vocabulary tree. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2161--2168.

Digital Library

[113]

NUS-WIDE. http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm.

[114]

X. Olivares, M. Ciaramita, and R. Zwol. 2008. Boosting image retrieval through aggregating search results based on visual annotations. In Proceedings of ACM Multimedia. 189--198.

Digital Library

[115]

G. Park, Y. Baek, and H.-K. Lee. 2005. Re-ranking algorithm using post-retrieval clustering for content-based image retrieval. Information Processing & Management 41, 2 (March 2005), 177--194.

Digital Library

[116]

PASCAL. http://pascallin.ecs.soton.ac.uk/challenges/VOC/.

[117]

James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. 2007. Object retrieval with large vocabularies and fast spatial matching. In Proceedings of CVPR.

[118]

A. Popescu, P.-A. Moellic, I. Kanellos, and R. Landais. 2009. Lightweight web image reranking. In Proceedings of ACM Multimedia. 657--660.

Digital Library

[119]

M. Elena Renda and U. Straccia. 2003. Web metasearch: Rank vs. score based rank aggregation methods. In Proceedings of the ACM Symposium on Applied Computing. 841--846.

Digital Library

[120]

U. Rohini and V. Varma. 2007. A novel approach for re-ranking of search results using collaborative filtering. In Proceedings of the International Conference on Computing: Theory and Applications.

Digital Library

[121]

D. E. Rose and D. Levinson. 2004. Understanding user goals in web search. In Proceedingos of WWW. 13--19.

Digital Library

[122]

S. Rudinac, M. Larson, and A. Hanjalic. 2009. Exploiting visual reranking to improve pseudo-relevance feedback for spoken-content-based video retrieval. In Proceedings of the International Workshop on Image Analysis for Multimedia Interactive Services. 17--20.

[123]

Y. Rui, T. S. Huang, and S.-F. Chang. 1999. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation 13, 10 (1999), 39--62.

Digital Library

[124]

Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra. 1998. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. on Circuits and Video Technology 8, 5 (September 1998), 644--655.

Digital Library

[125]

R. Yan and A. G. Hauptmann. 2003. The combination limit of video retrieval. In Proceedings of ACM Multimedia. 339--342.

Digital Library

[126]

G. Salton, A. Wong, and C. S. Yang. 1975. A vector space model for automatic indexing. Commun. ACM 18, 11 (1975), 611--620.

Digital Library

[127]

SavvySearch. 2013. Search.com. Retrieved from http://www.search.com/images.

[128]

J. Sivic and A. Zisserman. 2003. Video Google: A tex retrieval approach to object matching in videos. In Proceedings of ICCV.

Digital Library

[129]

Alan F. Smeaton, Paul Over, Cash J. Costello, Arjen P. De Vries, David Doermann, Alexander Hauptmann, Er Hauptmann, Mark E. Rorvig, John R. Smith, and Lide Wu. 2002. The TREC2001 video track: Information retrieval on digital video information. In Proceedings of the European Conference on Research and Advanced Technology for Digital Libraries.

Digital Library

[130]

A. F. Smeaton, P. Over, and W. Kraaij. 2006. Evaluation campaigns and TRECVid. In Proceedings of the ACM Workshop on Multimedia Information Retrieval.

Digital Library

[131]

A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 12 (2000), 1349--1380.

Digital Library

[132]

J. R. Smith, A. Jaimes, C.-Y. Lin, M. Naphade, A. P. Natsev, and B. Tseng. 2003. Interactive search fusion methods for video database retrieval. In Proceedings of the IEEE International Conference on Image Processing.

[133]

C. G. M. Snoek, J. C. van Gemert, Th. Gevers, B. Huurnink, D. C. Koelma, M. Van Liempt, O. De Rooij, K. E. A. van de Sande, F. J. Seinstra, A. W. M. Smeulders, A. H. C. Thean, C. J. Veenman, and M. Worring. 2006. The MediaMill TRECVID 2006 Semantic Video Search Engine. In TREC Video Retrieval Evaluation Online Proceedings.

[134]

Cees G. M. Snoek and Marcel Worring. 2009. Concept-based video retrieval. Foundations and Trends in Information Retrieval 4, 2 (2009), 215--322.

Digital Library

[135]

Cees G. M. Snoek, Marcel Worring, Ork de Rooij, Koen E. A. van de Sande, Rong Yan, and Alexander G. Hauptmann. 2008. VideOlympics: Real-time evaluation of multimedia retrieval systems. IEEE MultiMedia 15, 1 (2008), 86--91.

Digital Library

[136]

C. G. M. Snoek, M. Worring, and A. W. M. Smeulders. 2005. Early versus late fusion in semantic video analysis. In Proceedings of ACM Multimedia. Singapore, 399--402.

Digital Library

[137]

Cees G. M. Snoek, M. Worring, Jan C. van Gemert, J. M. Geusebroek, and Arnold W. M. Smeulders. 2006. The challenge problem for automated detection of 101 semantic concepts in multimedia. In Proceedings of the ACM International Conference on Multimedia.

Digital Library

[138]

K. Song, Y. Tian, W. Gao, and T. Huang. 2006. Diversifying the image retrieval results. In Proceedings of ACM Multimedia. 707--710.

Digital Library

[139]

D. Sontag, K. Collins-Thompson, P. N. Bennett, R. W. White, S. Dumais, and B. Billerbeck. 2012. Probabilistis models for personalizing web search. In Proceedings of the ACM International Conference on Web Search and Data Mining.

Digital Library

[140]

J. Tang, X.-S. Hua, G.-J. Qi, and X. Wu. 2007. Typicality ranking via semi-supervised multiple-instance learning. In Proceedings of ACM Multimedia. 297--300.

Digital Library

[141]

J. Teevan, E. Adar, R. Jones, and M. A. S. Potts. 2007. Information re-retrieval: Repeat queries in Yahoo¡¯s logs. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 151--158.

Digital Library

[142]

Xinmei Tian, Dacheng Tao, Xian-Sheng Hua, and Xiuqing Wu. 2010. Active reranking for web image search. IEEE Transactions on Image Processing 19, 3 (March 2010), 805--820.

Digital Library

[143]

X. Tian, L. Yang, J. Wang, Y. Yang, X. Wu, and X.-S. Hua. 2008. Bayesian video search reranking. In Proceedings of ACM Multimedia. 131--140.

Digital Library

[144]

A. Torralba. 2008. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 11 (2008), 1958--1970.

Digital Library

[145]

TRECVID. 2013. TREC Video Retrieval Evaluation: TRECVID. Retrieved from http://www-nlpir.nist.gov/projects/trecvid/.

[146]

Michele Trevisiol, Luca Chiarandini, Luca Maria Aiello, and Alejandro Jaimes. 2012. Image ranking based on user browsing behavior. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 445--454.

Digital Library

[147]

Y.-H. Tseng, C.-Y. Tsai, and Z.-J. Chuang. 2008. On the robustness of document re-ranking technique, a comparison of label propagation, KNN, and relevance feedback. In Proceedings of NTCIR-6 Workshop Meeting.

[148]

Panu Turcot and David G. Lowe. 2009. Better matching with fewer features: The selection of useful features in large database recognition problems. In Proceedings of ICCV Workshop on Emergent Issues in Large Amounts of Visual Data.

[149]

VIDEOSURF. (2010). http://www.videosurf.com/.

[150]

G. Wang and D. Forsyth. 2008. Object image retrieval by exploiting online knowledge resources. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[151]

J. Wang and X.-S. Hua. 2011. Interactive image search by color map. ACM Transactions on Intelligent Systems and Technology 3, 1 (2011).

Digital Library

[152]

L. Wang, L. Yang, and X. Tian. 2009b. Query aware visual similarity propagation for image search reranking. In Proceedings of ACM Multimedia. 725--728.

Digital Library

[153]

M. Wang, H. Li, D. Tao, and K. Lu. 2012. Multimodal graph-based reranking for web image search. IEEE Transactions on Image Processing 21, 11 (Nov. 2012), 4649--4661.

Digital Library

[154]

M. Wang, K. Yang, X.-S. Hua, and H.-J. Zhang. 2010a. Towards a relevant and diverse search of social images. IEEE Transactions on Multimedia 12, 8 (Dec. 2010), 829--842.

Digital Library

[155]

M. Wang, L. Yang, and X.-S. Hua. 2009a. MSRA-MM: Bridging research and industrial societies for multimedia information retrieval. Microsoft Technical Report.

[156]

S. Wang, F. Jing, J. He, Q. Du, and L. Zhang. 2007. IGroup: Presenting web image search results in semantic clusters. In Proceeding of the SIGCHI Conference on Human Factors in Computing Systems. 587--596.

Digital Library

[157]

X. Wang, Z. Li, and D. Tao. 2011. Subspaces indexing model on Grassmann manifold for image search. IEEE Trans. on Image Processing 20, 9 (Sept. 2011), 2627--2635.

Digital Library

[158]

Xin-Jing Wang, Lei Zhang, Ming Liu, Yi Li, and Wei-Ying Ma. 2010b. ARISTA--image search to annotation on billions of web photos. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 2987--2994.

[159]

Yang Wang, Tao Mei, Jingdong Wang, Houqiang Li, and Shipeng Li. 2011. JIGSAW: Interactive mobile visual search with multimodal queries. In Proceedings of ACM Multimedia. 73--82.

Digital Library

[160]

S. Wei, Y. Zhao, Z. Zhu, and N. Liu. 2009. Multimodal fusion for video search Reranking. IEEE Trans. on Knowledge and Data Engineering (June 2009).

Digital Library

[161]

Ryen W. White, Matthew Richardson, Mikhail Bilenko, and Allison P. Heath. 2008. Enhancing web search by promoting multiple search engine use. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 43--50.

Digital Library

[162]

Wikipedia. 2013. Click-Through Rate. Retrieved from http://en.wikipedia.org/wiki/Click-through_rate.

[163]

P. Wilkins, P. Ferguson, and A. F. Smeaton. 2006. Using score distribution for query-time fusion in multimedia retrieval. In Proceedings of the ACM Workshop on Multimedia Information Retrieval. 51--60.

Digital Library

[164]

P. Wilkins, A. F. Smeaton, and P. Ferguson. 2010. Properties of optimally weighted data fusion in CBMIR. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 643--650.

Digital Library

[165]

K. Wnuk and S. Soattoh. 2008. Filtering internet image search results towards keyword based category recognition. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.

[166]

M. Worring, C. G. M. Snoek, O. de Rooij, G. P. Nguyen, and A. W. M. Smeulders. 2007. Recent advances and challenges of semantic image/video search. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 1213--1216.

[167]

X. Wu, A. G. Hauptmann, and C.-W. Ngo. 2007. Novelty detection for cross-lingual news stories with visual duplicates and speech transcripts. In Proceedings of ACM Multimedia. 168--177.

Digital Library

[168]

Hao Xu, Jingdong Wang, Xian-Sheng Hua, and Shipeng Li. 2010. Image search by concept map. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 275--282.

Digital Library

[169]

Yahoo&excl; 2013. Homepage. Retrieved from http://www.yahoo.com/.

[170]

T. Yamamoto, S. Nakamura, and K. Tanaka. 2007. Rerank-by-example: Efficient browsing of web search results. Database and Expert Systems Applications (2007), 801--810.

Digital Library

[171]

R. Yan and A. G. Hauptmann. 2004. Co-retrieval: A boosted reranking approach for video retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval.

[172]

R. Yan and A. G. Hauptmann. 2006. Probabilistic latent query analysis for combining multiple retrieval sources. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. 324--331.

Digital Library

[173]

R. Yan and A. G. Hauptmann. 2007a. Query expansion using probabilistic local feedback with application to multimedia Retrieval. In Proceedings of the ACM Conference on Information and Knowledge Management. 361--370.

Digital Library

[174]

R. Yan and A. G. Hauptmann. 2007b. A review of text and image retrieval approaches for broadcast news video. Information Retrieval 10, 4--5 (October 2007), 445--484.

Digital Library

[175]

R. Yan, A. G. Hauptmann, and R. Jin. 2003. Multimedia search with pseudo-relevance feedback. In Proceedings of the ACM International Conference on Image and Video Retrieval.

Digital Library

[176]

R. Yan, J. Yang, and A. Hauptmann. 2004. Learning query-class dependent weights in automatic video retrieval. In Proceedings of ACM Multimedia. 548--555.

Digital Library

[177]

A. Yanagawa, S. F. Chang, L. Kennedy, and W. Hsu. 2007. Columbia University’s baseline detectors for 374 LSCOM semantic visual concepts. Columbia University ADVENT Technical Report.

[178]

K. Yang, M. Wang, X.-S. Hua, and H.-J. Zhang. 2010. Social image search with diverse relevance ranking. In Proceedings of the International MultiMedia Modeling Conference. 174--184.

Digital Library

[179]

Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan. 2012. A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans. on Pattern Analysis and Machine Intelligence 34, 4 (April 2012), 723--742.

Digital Library

[180]

YouTube. 2013. Homepage. Retrieved from http://www.youtube.com/.

[181]

Eric Zavesky and Shih-Fu Chang. 2008. CuZero: Embracing the frontier of interactive visual search for informed users. In Proceedings of ACM Multimedia Information Retrieval. 237--244.

Digital Library

[182]

Zheng-Jun Zha, Linjun Yang, Tao Mei, Meng Wang, and Zengfu Wang. 2009. Visual query suggestion. In Proceedings of ACM Multimedia. 15--24.

Digital Library

[183]

Zheng-Jun Zha, Linjun Yang, Tao Mei, Meng Wang, Zengfu Wang, Tat-Seng Chua, and Xian-Sheng Hua. 2010. Visual query suggestion: Towards capturing user intent in internet image search. ACM Transactions on Multimedia Computing, Communications, and Applications 6, 3 (2010).

Digital Library

[184]

Wengang Zhou, Yijuan Lu, Houqiang Li, Yibing Song, and Qi Tian. 2010. Spatial coding for large scale partial-duplicate web image search. In Proceedings of ACM Multimedia. 511--520.

Digital Library

[185]

X. S. Zhou and T. S. Huang. 2002. Relevance feedback in content-based image retrieval: Some recent advances. Information Sciences 148, 1--4 (Dec. 2002), 129--137.

Digital Library

[186]

X. S. Zhou and T. S. Huang. 2003. Relevance feedback for image retrieval: A comprehensive review. Multimedia Systems 8, 6 (April 2003), 536--544.

[187]

G. Zhu, S. Yan, and Y. Ma. 2010. Image tag refinement towards low-rank, content-tag prior and error sparsity. In Proceedings of the ACM International Conference on Multimedia. 461--470.

Digital Library

[188]

Jinfeng Zhuang, Tao Mei, Steven C. H. Hoi, Xian-Sheng Hua, and Shipeng Li. 2011a. Modeling social strength in social media community via kernel-based learning. In Proceedings of ACM Multimedia. 113--122.

Digital Library

[189]

Jinfeng Zhuang, Tao Mei, Steven C. H. Hoi, Ying-Qing Xu, and Shipeng Li. 2011b. When recommendation meets mobile: contextual and personalized recommendation on the go. In Proceedings of the ACM International Conference on Ubiquitous Computing. 153--162.

Digital Library

[190]

Z. Zhuang and S. Cucerzan. 2006. Re-ranking search results using query logs. In Proceedings of the ACM Conference on Information and Knowledge Management. 860--861.

Digital Library

[191]

H. Zitouni, S. Sevil, D. Ozkan, and P. Duygulu. 2008. Re-ranking of web image search results using a graph algorithm. In Proceedings of the International Conference on Pattern Recognition. 1--4.

[192]

M. M. Zloof. 1975a. Query by example. In Proceedings of the AFIPS National Compute Conference. 431--438.

Digital Library

[193]

Moshe M. Zloof. 1975b. Query-by-example: The invocation and definition of tables and forms. In Proceedings of VLDB. 1--24.

Digital Library

Cited By

Wu MWu FLin W(2024)Improving the Precision of Image Search Engines with the Psychological Intention DiagramElectronics10.3390/electronics1301020813:1(208)Online publication date: 2-Jan-2024
https://doi.org/10.3390/electronics13010208
Zhang YQian QWang HLiu CChen WWang F(2024)Graph Convolution Based Efficient Re-Ranking for Visual RetrievalIEEE Transactions on Multimedia10.1109/TMM.2023.327616726(1089-1101)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3276167
Chen AZhou FWang ZLi X(2024)Cliprerank: An Extremely Simple Method For Improving Ad-Hoc Video SearchICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446902(7850-7854)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446902
Show More Cited By

Index Terms

Multimedia search reranking: A literature survey
1. Information systems
  1. Information retrieval
  2. Information systems applications
    1. Multimedia information systems

Recommendations

User Intent in Multimedia Search: A Survey of the State of the Art and Future Challenges

Today's multimedia search engines are expected to respond to queries reflecting a wide variety of information needs from users with different goals. The topical dimension (“what” the user is searching for) of these information needs is well studied; ...
Optimizing Visual Search Reranking via Pairwise Learning

Visual search reranking is defined as reordering visual documents (images or video clips) based on the initial search results or some auxiliary knowledge to improve the search precision. Conventional approaches to visual search reranking empirically ...
Online reranking via ordinal informative concepts for context fusion in concept detection and video search

To exploit the co-occurrence patterns of semantic concepts while keeping the simplicity of context fusion, a novel reranking approach is proposed in this paper. The approach, called ordinal reranking, adjusts the ranking of an initial search (or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 46, Issue 3

January 2014

507 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/2578702

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2014

Accepted: 01 August 2013

Revised: 01 July 2013

Received: 01 June 2012

Published in CSUR Volume 46, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

156
Total Citations
View Citations
2,225
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu MWu FLin W(2024)Improving the Precision of Image Search Engines with the Psychological Intention DiagramElectronics10.3390/electronics1301020813:1(208)Online publication date: 2-Jan-2024
https://doi.org/10.3390/electronics13010208
Zhang YQian QWang HLiu CChen WWang F(2024)Graph Convolution Based Efficient Re-Ranking for Visual RetrievalIEEE Transactions on Multimedia10.1109/TMM.2023.327616726(1089-1101)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3276167
Chen AZhou FWang ZLi X(2024)Cliprerank: An Extremely Simple Method For Improving Ad-Hoc Video SearchICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446902(7850-7854)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446902
Baikadolla MBaditha SPatta MMuktha K(2024)Thumbnail Personalization in Movie Recommender SystemCommunication and Intelligent Systems10.1007/978-981-97-2079-8_22(277-296)Online publication date: 11-May-2024
https://doi.org/10.1007/978-981-97-2079-8_22
Zhao MLiu LDing YDeng HLiang HWang HWang Y(2023)Lightweight and Privacy-Preserving Multi-Keyword Search over Outsourced DataApplied Sciences10.3390/app1305284713:5(2847)Online publication date: 22-Feb-2023
https://doi.org/10.3390/app13052847
Yanagi RTogo ROgawa THaseyama M(2023)Recallable Question Answering-Based Re-Ranking Considering Semantic Region for Cross-Modal RetrievalIEEE Open Journal of Signal Processing10.1109/OJSP.2023.32382804(1-11)Online publication date: 2023
https://doi.org/10.1109/OJSP.2023.3238280
Gasmi KAouadi HTorjmen M(2023)Link-Driven Study to Enhance Text-Based Image Retrieval: Implicit Links Versus Explicit LinksIEEE Access10.1109/ACCESS.2023.330746411(90526-90537)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3307464
Guo SGeng HSu LHe SZhang X(2023)A Rankable Boolean Searchable Encryption Scheme Supporting Dynamic Updates in a Cloud EnvironmentIEEE Access10.1109/ACCESS.2023.328490411(63475-63486)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3284904
Zhang HYanagi RTogo ROgawa THaseyama M(2023)Cross-Modal Image Retrieval Considering Semantic Relationships With Many-to-Many Correspondence LossIEEE Access10.1109/ACCESS.2023.323985811(10675-10686)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3239858
Novas DPapakyriakopoulos DKartaloglou EGriva A(2023)Α ranking model based on user generated content and fuzzy logicInternational Journal of Hospitality Management10.1016/j.ijhm.2023.103561114(103561)Online publication date: Sep-2023
https://doi.org/10.1016/j.ijhm.2023.103561
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents