article

Real time repeated video sequence identification

Authors:

Susan E. Gauch,

Jedrzej Z. MiadowiczAuthors Info & Claims

Computer Vision and Image Understanding, Volume 93, Issue 3

Pages 310 - 327

https://doi.org/10.1016/j.cviu.2003.10.005

Published: 01 March 2004 Publication History

Abstract

In this paper, we present a real time system for detecting repeated video clips from a live video source such as news broadcasts. Our system utilizes customized temporal video segmentation techniques to automatically partition the digital video signal into semantically sensible shots and scenes. As each frame of the video source is processed, we extract auxiliary information to facilitate repeated sequence detection. When the video transition marking the end of the shot/scene is detected, we are able to rapidly locate all previous occurrences of the video clip. Our objective is to use repeated sequence information in our multimedia content analysis application to deduce semantic relationships among shots/scenes in the input video. Our real time video processing techniques are independent of source and domain and can be applied to other applications such as commercial detection and improved video compression.

References

[1]

{1} J. Allan, V. Laverenko, R. Papka, On-line new event detection and tracking, in: Proc. of ACM SIGIR 1998, 1998, pp. 37-45.

[2]

{2} F. Arman, R. Depommier, A. Hsu, M.Y. Chiu, Content-based browsing of video sequences, in: ACM Multimedia 1994, San Francisco, 1994, pp. 97-103.

[3]

{3} S. Bouix, VISION: segmentation, indexing and retrieval of digital videos, Master of Science Thesis, The University of Kansas, 1998.

[4]

{4} C. Breiteneder, H. Eidenberger, Content-based image retrieval and digital libraries, in: Kyoto Int. Conf. on Digital Libraries, 2000, pp. 67-74.

[5]

{5} S.F. Chang, W. Chen, H. Meng, H. Sundaran, D. Zhong, VideoQ: an automated content based video search system using visual cues, in: Proc. of ACM Multimedia 1997, Seattle, November 1997, pp. 313-324.

[6]

{6} V. Chitkara, Color-based image retrieval using compact binary signatures, Master's Thesis, Department of Computing Science, University of Alberta, 2001.

[7]

{7} S. Dagtas, T. McGee, A.M. Mohamed, SmartWatch: an automated video event finder, in: ACM Multimedia '00, Los Angeles, 2000, pp. 503-508.

[8]

{8} C. Decleir, M.S. Hacid, J. Kouloumdjian, A database approach for modeling and querying video data, in: Proc. 15th Int. Conf. on Data Engineering (ICDE'99), Sydney, March 1999, pp. 6-13.

[9]

{9} A. Del-Bimbo, M. Mugnaini, P. Pala, F. Turco, Visual querying by color perceptive regions, Pattern Recog. 3 (9) 1241-1253.

[10]

{10} A. Ekin, A.M. Tekalp, R. Mehrotra, Object-based video description: from low-level features to semantics, in: Symp. Electronic Imaging: Science and Technology: Storage and Retrieval for Image and Video Databases IV, IS&T/SPI01, San Jose, January 2001.

[11]

{11} A. Ekin, A.M. Tekalp, Integrated semantic video querying using an integrated semantic-syntactic model, in: Proc. IEEE ICIP, Rochester, September 2002.

[12]

{12} M. Flickner, J.A. Sawhney, Q. Huang, B. Dom, M. Gorkani, D. Lee, D. Petkovic, D. Steele, P. Yanker, Query by image and video content: the QBIC system, IEEE Comput. 28 (9) (1995) 23-32.

Digital Library

[13]

{13} Y. Fu, A. Ekin, A.M. Tekalp, R. Mehrotra, Temporal segmentation of video objects for hierarchical object-based motion description, IEEE Trans. Image Process. 11 (2) (2002) 135-145.

Digital Library

[14]

{14} S.E. Gauch, W. Li, J.M. Gauch, The vision digital video library, Informat. Process. Manag. 33 (4) (1997) 413-426.

Digital Library

[15]

{15} J.M. Gauch, Real time feature-based video stream validation and distortion analysis system using color moment, US Patent 6, 2001, pp, 246-803.

[16]

{16} B. Gunsel, A.M. Ferman, A.M. Tekalp, Temporal video segmentation using unsupervised clustering and semantic object tracking, J. Electron. Imaging 7 (3) (1998) 592-604.

[17]

{17} A.G. Hauptmann, M.J. Witbrock, Informedia: news on demand multimedia information acquisition and retrieval, in: Intelligent Multimedia Information Retrieval, AAAI Press, 1997, pp. 213-239.

Digital Library

[18]

{18} E. Izquierdo, J. Casas, R. Leonardi, P. Migliorati, N. O'Connor, I. Kompatsiaris, M. Strintzis, Advanced content-based semantic scene analysis and information retrieval: the SCHEMA Project, in: WIAMIS 2003-4th Workshop on Image Analysis for Multimedia Interactive Service, London, April 2003.

[19]

{19} A.A.C. Kalker, J.A. Haitsma, J. Oostveen, Issues with digital watermarking and perceptual hashing, SPIE Conf. Multimedia Systems Appl. (2001).

[20]

{20} C.M. Lee, Q. Li, W. Xiong, VIMS: a video information management system, Multimedia Tools Appl. 4 (1) (1997) 7-28.

[21]

{21} W. Li, S.E. Gauch, J.M. Gauch, K.M. Pua, VISION: a digital video library, in: ACM Digital Libraries '96, Baltimore, March 1996 pp. 19-27.

[22]

{22} W.K. Li, S.H. Lai, Integrated video shot segmentation algorithm, in: Storage and Retrieval for Media Databases 2003, SPIE, vol. 5021, 2003.

[23]

{23} R. Lienhart, W. Effelsberg, R. Jain, Visualgrep: a systematic method to compare and retrieve video sequences, Technical Report TR-97-005, Praktische Informatik IV, University of Mannheim, October 1997.

[24]

{24} A. Nagasaka, T. Tanaka, Automatic video indexing and full-video search for object appearances, in: Visual Database Systems II, Elsevier Science, Amsterdam, 1992, pp. 119-133.

[25]

{25} V. Ogle, M. Stonebraker, Chabot: Retrieval from a relational database of images, IEEE Comput. 28 (9) (1995) 40-48.

Digital Library

[26]

{26} J.C. Oostveen, A.A.C. Kalker, J.A. Haitsma, Visual hashing of digital video: applications and techniques, in: SPIE Applications of Digital Image Processing XXIV, San Diego, July 2001.

[27]

{27} G. Pass, R. Zabih, Comparing images using joint histograms, ACM J. Multimedia System (1998) 234-240.

[28]

{28} R. Qian, H. Haering, I. Sezan, A computational approach to semantic event detection, in: IEEE CVPR '99, Fort Collins, CO, June 1999, pp. 200-206.

[29]

{29} K. Rodden, Evaluating similarity-based visualisations as interfaces for image browsing, Technical Report UCAM-CL-TR-543, University of Cambridge Computer Laboratory, Septemeber 2002.

[30]

{30} Y. Rui, T.S. Huang, S.F. Chang, Image retrieval: past, present, and future, in: Proc. of Int. Symposium on Multimedia Information Processing, Taiwan, December 1997.

[31]

{31} Y. Rui, T.S. Huang, S. Mehrotra, Constructing table of contents for videos, ACM J. Multimedia Systems (1998) 359-368.

[32]

{32} C.L. Sabharwal, S.K. Bhatia, Perfect hash table algorithm for image databases using negative associated values, Pattern Recog. 28 (7) (1995) 1091-1101.

[33]

{33} H.S. Sawhney, S. Ayer, Compact representation of videos through dominant and multiple motion estimation, IEEE Pattern Anal. Mach. Intell. 18 (8) (1996) 814-830.

Digital Library

[34]

{34} R. Schettini, G. Ciocca, S. Zuffi, A survey on methods for colour image indexing and retrieval in image databases, in: R. Luo, L. MacDonald (Eds.), Color Imaging Science: Exploiting Digital Media, Wiley, New York, 2001.

[35]

{35} J. Smith, S.F. Chang, Tools and techniques for color image retrieval, SPIE Proc. 2670 (1996) 1630-1639.

[36]

{361 J.R. Smith, S.-F. Chang. Visualseek: a fully automated content-based image query system, in: Proc. of ACM Multimedia 96, Boston, MA, USA, 1996, pp. 87-98.

Digital Library

[37]

{37} M. Stricker, A. Dimai, Color indexing with weak spatial constraints, in: SPIE Proc. 2670 (1996) 29-40.

[38]

{38} M.J. Swain, D.H. Ballard, Color indexing, Int. J. Comput. Vision 7 (1991) 11-32.

Digital Library

[39]

{39} S.M. Tanveer, S. Srinivasan, Detecting topical events in digital video, in: ACM Multimedia '00, Los Angeles, 2000 pp. 85-94.

[40]

{40} L. Teodosio, W. Bender, Salient video stills: content and context preserved, in: Proc. ACM Multimedia '93, 1993, pp. 39-46.

Digital Library

[41]

{41} H.D. Wactlar, M.G. Christel, Y. Gong, A.G. Hauptmann, Lessons learned from the creation and deployment of a terabyte digital video library, IEEE Comput. 32 (2) (1999) 66-73.

Digital Library

[42]

{42} H.D. Wactlar, A.G. Hauptmann, M.G. Christel, R.A. Houghton, A.M. Olligschlaeger, Complementary video and audio analysis for broadcast news archives, Commun. ACM 43 (2) (2000) 42-47.

Digital Library

[43]

{43} W. Wolf, Key frame selection by motion analysis, in: Proc. ICASSP '96, 1996, pp. 1228-1231.

[44]

{44} K. Yoon, D. DeMentbon, D. Doermann, Event detection from MPEG video in the compressed domain, in: Proc. of ICPR '00, 2000, pp. 819-822.

[45]

{45} H.J. Zhang, S.Y. Tan, S.W. Smoliar, Y.H. Gong, Automatic parsing and indexing of news video, Multimedia Systems 2 (1995) 256-266.

Digital Library

Cited By

Yuan JGravier GCampion SLiu XJégou H(2012)Efficient mining of repetitions in large-scale TV streams with product quantization hashingProceedings of the 12th international conference on Computer Vision - Volume Part I10.1007/978-3-642-33863-2_27(271-280)Online publication date: 7-Oct-2012
https://dl.acm.org/doi/10.1007/978-3-642-33863-2_27
Döhring ILienhart RKompatsiaris YMarchand-Maillet SAvrithis YConnor NGatica-Perez DChua T(2009)Mining TV broadcasts for recurring video sequencesProceedings of the ACM International Conference on Image and Video Retrieval10.1145/1646396.1646432(1-8)Online publication date: 8-Jul-2009
https://dl.acm.org/doi/10.1145/1646396.1646432
Naturel XGros P(2008)Detecting repeats for video structuringMultimedia Tools and Applications10.1007/s11042-007-0180-138:2(233-252)Online publication date: 1-Jun-2008
https://dl.acm.org/doi/10.1007/s11042-007-0180-1
Show More Cited By

Index Terms

Real time repeated video sequence identification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Video segmentation
      2. Computer vision tasks
        Scene understanding
        Video summarization
  2. Computer graphics
    1. Image compression
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Zero-Correlation Zone Sequence Set Constructed from a Perfect Sequence and a Complementary Sequence Pair

The present paper introduces the construction of a class of sequence sets with zero-correlation zones called zero-correlation zone sequence sets. The proposed zero-correlation zone sequence set can be generated from an arbitrary perfect sequence and an ...
Binary Zero-Correlation Zone Sequence Set Constructed from an M-Sequence

The present paper introduces an improved construction of a class of binary sequences having a zero-correlation zone (hereafter binary ZCZ sequence set). The cross-correlation function and the side-lobe of the auto-correlation function of the proposed ...
Finding and identifying unknown commercials using repeated video sequence detection

Automated commercial detection can be performed by matching features extracted from commercials or by detecting embedded codes that are hidden within the commercial. In both cases, it is necessary to create a database of known commercials that contain ...

Comments

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 93, Issue 3

March 2004

126 pages

ISSN:1077-3142

Issue’s Table of Contents

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 March 2004

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yuan JGravier GCampion SLiu XJégou H(2012)Efficient mining of repetitions in large-scale TV streams with product quantization hashingProceedings of the 12th international conference on Computer Vision - Volume Part I10.1007/978-3-642-33863-2_27(271-280)Online publication date: 7-Oct-2012
https://dl.acm.org/doi/10.1007/978-3-642-33863-2_27
Döhring ILienhart RKompatsiaris YMarchand-Maillet SAvrithis YConnor NGatica-Perez DChua T(2009)Mining TV broadcasts for recurring video sequencesProceedings of the ACM International Conference on Image and Video Retrieval10.1145/1646396.1646432(1-8)Online publication date: 8-Jul-2009
https://dl.acm.org/doi/10.1145/1646396.1646432
Naturel XGros P(2008)Detecting repeats for video structuringMultimedia Tools and Applications10.1007/s11042-007-0180-138:2(233-252)Online publication date: 1-Jun-2008
https://dl.acm.org/doi/10.1007/s11042-007-0180-1
Poli J(2008)An automatic television stream structuring system for television archives holdersMultimedia Systems10.1007/s00530-008-0140-214:5(255-275)Online publication date: 1-Nov-2008
https://dl.acm.org/doi/10.1007/s00530-008-0140-2
Yuan JWang WMeng JWu YLi DLienhart RPrasad AHanjalic AChoi SBailey BSebe N(2007)Mining repetitive clips through finding continuous pathsProceedings of the 15th ACM international conference on Multimedia10.1145/1291233.1291294(289-292)Online publication date: 29-Sep-2007
https://dl.acm.org/doi/10.1145/1291233.1291294
Zobel JHoad T(2006)Detection of video sequences using compact signaturesACM Transactions on Information Systems10.1145/1125857.112585824:1(1-50)Online publication date: 1-Jan-2006
https://dl.acm.org/doi/10.1145/1125857.1125858
Gauch JShivadas A(2006)Finding and identifying unknown commercials using repeated video sequence detectionComputer Vision and Image Understanding10.1016/j.cviu.2006.03.002103:1(80-88)Online publication date: 1-Jul-2006
https://dl.acm.org/doi/10.1016/j.cviu.2006.03.002
Naturel XGros PAmsaleg LJónsson BOria V(2005)A fast shot matching strategy for detecting duplicate sequences in a television streamProceedings of the 2nd international workshop on Computer vision meets databases10.1145/1160939.1160947(21-27)Online publication date: 17-Jun-2005
https://dl.acm.org/doi/10.1145/1160939.1160947
Yang XXue PTian QZhang HChua TSteinmetz RKankanhalli MWilcox L(2005)A repeated video clip identification systemProceedings of the 13th annual ACM international conference on Multimedia10.1145/1101149.1101189(227-228)Online publication date: 6-Nov-2005
https://dl.acm.org/doi/10.1145/1101149.1101189
Yuan JDuan LTian QRanganath SXu C(2004)Fast and robust short video clip search for copy detectionProceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing - Volume Part II10.1007/978-3-540-30542-2_59(479-488)Online publication date: 30-Nov-2004
https://dl.acm.org/doi/10.1007/978-3-540-30542-2_59

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents