Abstract
This paper proposes a new approach for shot-based retrieval by optimal matching (OM), which provides an effective mechanism for the similarity measure and ranking of shots by one-to-one matching. In the proposed approach, a weighted bipartite graph is constructed to model the color similarity between two shots. Then OM based on Kuhn–Munkres algorithm is employed to compute the maximum weight of a constructed bipartite graph as the shot similarity value by one-to-one matching among frames. To improve the speed efficiency of OM, two improved algorithms are also proposed: bipartite graph construction based on subshots and bipartite graph construction based on the same number of keyframes. Besides color similarity, motion feature is also employed for shot similarity measure. A motion histogram is constructed for each shot, the motion similarity between two shots is then measured by the intersection of their motion histograms. Finally, the shot similarity is based on the linear combination of color and motion similarity. Experimental results indicate that the proposed approach achieves better performance than other methods in terms of ranking and retrieval capability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen L, Chua TS (2001) A match and tiling approach to content-based video retrieval. International Conference on Multimedia and Expo
Deng Y, Manjunath BS (1997) Content-based search of video using color, texture and motion. International Conference on Image Processing 534–537
Fan J, Elmagarmid AK, Zhu X, Aref WG, Wu L (2004) Classview: hierarchical video shot classification, indexing, and accessing. IEEE Trans Multimedia 6(1):70–86
Hauptmann A, Chen M-Y, Christel M et al Confounded expectations: Informedia at TRECVID 2004. http://www-nlpir.nist.gov/projects/tvpubs/ tvpapers04/
Jain AK, Vailaya A, Wei X (1999) Query by video clip. Multimedia Syst 7:369–384
Lienhart R, Effelsberg W, Jain R (1998) VisualGREP: a systematic method to compare and retrieve video sequences. In: SPIE Conference on Storage and Retrieval for Image and Video Databases. pp 271–282
Lin T, Ngo CW, Zhang HJ et al (2001) Integrating color and spatial features for content-based video retrieval. In: IEEE International Conference on Image Processing (ICIP 2001). pp 592–595
Liu X, Zhuang Y, Pan Y (1999) A new approach to retrieve video by example video clip. ACM Multimedia Conference
MPEG video group (1999) Description of Core Experiments for MPEG-7 Color/Texture Descriptions. ISO/MPEGJTC1/SC29/WG11 MPEG98/M2819
Ngo CW, Pong TC, Chin RT (2001) Video partitioning by temporal slice coherency. IEEE Trans Circuits Syst Video Technol 11(8):941–953
Ngo CW, Pong TC, Zhang HJ (2002) Motion-based video representation for scene change detection. Int J Comput Vis 50(2):127–143 (Nov)
Over P, Kraaij W, Laneva T, Smeaton A, Buckland L TREC 2005 video retrieval evaluation introductions. http://www-nlpir.nist.gov/projects/tvpubs/ tv.pubs.org.html
Peng Y, Ngo CW (2006) Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Trans Circuits Syst Video Technol 16(5):612–627 (May)
Schrijver A (2003) Combinatorial optimization: Polyhedra and efficiency, vol A. Springer Heidelberg New York
Smeaton A, Laneva T TRECVID 2005: Search task. http://www-nlpir.nist.gov/projects/tvpubs/tv.pubs.org.html
Smeaton A, Over P, Arlandis J TRECVID-2004: Search task overview. http://www-nlpir.nist.gov/projects/tvpubs/tv.pubs.org.html
Souvannavong F, Merialdo B, Huet B (2004) Latent semantic analysis for an effective region-based video shot retrieval system. In: The 6th ACM international workshop on multimedia information retrieval. New York, pp 243–250 (October)
Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32
Taskiran C, Chen J-Y, Albiol A, Torres L, Bouman CA, Delp EJ (2004) ViBE: a compressed video database structured for active browsing and search. IEEE Trans Multimedia 6(1):103–118
Wu Y, Zhuang Y, Pan Y (2000) Content-based video similarity model. In: ACM Multimedia Conference.
Xiao WS (1993) Graph theory and its algorithms. Aviation Industrial Press, Beijing
Yuan J, Duan L-Y, Tian Q, Wu C (2004) Fast and robust short video clip search using an index structure. In: The 6th ACM international workshop on multimedia information retrieval. New York, pp 61–68 (October)
Zhao L, Qi W, Li SZ et al (2000) “Key-frame extraction and shot retrieval using nearest feature line (NFL). In: ACM SIGMM international workshop on multimedia information retrieval.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Peng, Y., Ngo, CW. & Xiao, J. OM-based video shot retrieval by one-to-one matching. Multimed Tools Appl 34, 249–266 (2007). https://doi.org/10.1007/s11042-006-0085-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-006-0085-4