Article

Near-duplicate keyframe retrieval with visual keywords and semantic context

Authors:

Chong-Wah NgoAuthors Info & Claims

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

Pages 162 - 169

https://doi.org/10.1145/1282280.1282309

Published: 09 July 2007 Publication History

Abstract

Near-duplicate keyframes (NDK) play a unique role in large-scale video search, news topic detection and tracking. In this paper, we propose a novel NDK retrieval approach by exploring both visual and textual cues from the visual vocabulary and semantic context respectively. The vocabulary, which provides entries for visual keywords, is formed by the clustering of local keypoints. The semantic context is inferred from the speech transcript surrounding a keyframe. We experiment the usefulness of visual keywords and semantic context, separately and jointly, using cosine similarity and language models. By linearly fusing both modalities, performance improvement is reported compared with the techniques with keypoint matching. While matching suffers from expensive computation due to the need of online nearest neighbor search, our approach is effective and efficient enough for online video search.

References

[1]

J. Allan, C. Wade, and A. Bolivar. Retrieval and Novelty Detection at the Sentence Level. ACM SIGIR'03.

Digital Library

[2]

J. Amores, N. Sebe, P. Radeva, T. Gevers, and A. Smeulders. Boosting Contextual Information in Content-Based Image Retrieval. ACM MIR'04, 2004

Digital Library

[3]

S.-F. Chang and et al. Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction. TRECVID 2005, Washington DC, 2005.

[4]

M. Davis, S. King, N. Good, and R. Sarvas. From Context to Content: Leveraging Context to Infer Media Metadata. ACM MM'04, 2004.

Digital Library

[5]

P. Duygulu, J.-Y. Pan and D. A. Forsyth. Towards Auto-Documentary: Tracking the Evolution of News Stories. ACM MM'04, USA, Oct. 2004, pp. 820--827.

Digital Library

[6]

W. H. Hsu and S.-F. Chang. Topic Tracking across Broadcast News Videos with Visual Duplicates and Semantic Concepts. ICIP'06, Atlanta, GA, October 2006.

[7]

J. L. Gauvain, L. Lamel, and G. Adda. The LIMSI Broadcast News Transcription System. Speech Communication, 2002.

Digital Library

[8]

J. Li. A Mutual Semantic Endorsement Approach to Image Retrieval and Context Provision. ACM MIR'05, 2005.

Digital Library

[9]

Y. Ke, R. Suthankar, and L. Huston. Efficient Near-Duplicate Detection and Sub-Image Retrieval. ACM MM'04.

Digital Library

[10]

D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91--110, 2004.

Digital Library

[11]

K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors. CVPR'03, pp. 257--263.

Digital Library

[12]

K. Mikolajczyk and C. Schmid. Scale and Affine Invariant Interest Point Detectors. IJCV, 60 (2004), pp. 63--86.

Digital Library

[13]

C.-W. Ngo, W.-L. Zhao and Y.-G. Jiang. Fast Tracking of Near-Duplicate Keyframes in Broadcast Domain with Transitivity Propagation. ACM MM'06, pp. 845--854, 2006.

Digital Library

[14]

J. M. Ponte and W. B. Croft. A Language Modeling Approach to Information Retrieval. ACM SIGIR'98.

Digital Library

[15]

J. Sivic and A. Zisserman. Video Google: A Text Retrieval Approach to Object Matching in Videos. ICCV'03.

Digital Library

[16]

TRECVID, http://www-nlpir.nist.gov/projects/trecvid/.

[17]

X. Wu, C.-W. Ngo and Q. Li. Threading and Autodocumenting News Videos. IEEE Signal Processing Magazine, vol. 23, no. 2, pp. 59--68, March 2006.

[18]

C. Zhai and J. Lafferty. A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval. ACM SIGIR'01, USA, pp. 334--342, Sep. 2001.

Digital Library

[19]

Y. Zhai and M. Shah. Tracking News Stories across Different Sources. ACM MM'05, Singapore, Nov. 2005.

Digital Library

[20]

D.-Q. Zhang and S.-F. Chang. Detecting Image Near-Duplicate by Stochastic Attributed Relational Graph Matching with Learning. ACM MM'04, pp. 877--884, 2004.

Digital Library

[21]

Y. Zhang, J. Callan, and T. Minka. Novelty and Redundancy Detection in Adaptive Filtering. ACM SIGIR'02, 2002.

Digital Library

[22]

Y. Zhang, W. Xu, and J. Callan. Exact Maximum Likelihood Estimation for Word Mixtures. Text Learning Workshop at ICML'02.

[23]

W.-L. Zhao, C.-W. Ngo, H.-K. Tan and X. Wu. Near-Duplicate Keyframe Identification with Interest Point Matching and Pattern Learning. IEEE Trans. on MM, 2007.

Digital Library

[24]

Y. Zheng, S.-Y. Neo, T.-S. Chua and Q. Tian. Fast Near-Duplicate Keyframe Detection in Large-Scale Video Corpus for Video Search. IWAIT'07.

Cited By

Sandeep RPrabin B(2022)Application of Perceptual Video Hashing for Near-duplicate Video RetrievalEvolutionary Computing and Mobile Sustainable Networks10.1007/978-981-16-9605-3_18(253-275)Online publication date: 22-Mar-2022
https://doi.org/10.1007/978-981-16-9605-3_18
Zhang CJin DXiao XChen GShyu M(2020)A Novel Collaborative Optimization Framework for Web Video Event Mining Based on the Combination of Inaccurate Visual Similarity Detection Information and Sparse Textual InformationIEEE Access10.1109/ACCESS.2020.29647148(10516-10527)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2964714
Shen LHong RHao Y(2020)Advance on large scale near-duplicate video retrievalFrontiers of Computer Science10.1007/s11704-019-8229-714:5Online publication date: 3-Jan-2020
https://doi.org/10.1007/s11704-019-8229-7
Show More Cited By

Index Terms

Near-duplicate keyframe retrieval with visual keywords and semantic context
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization
2. Information systems
  1. Information retrieval

Recommendations

Visual word proximity and linguistics for semantic video indexing and near-duplicate retrieval

Bag-of-visual-words (BoW) has recently become a popular representation to describe video and image content. Most existing approaches, nevertheless, neglect inter-word relatedness and measure similarity by bin-to-bin comparison of visual words in ...
Near-duplicate keyframe retrieval by semi-supervised learning and nonrigid image matching

Near-duplicate keyframe (NDK) retrieval techniques are critical to many real-world multimedia applications. Over the last few years, we have witnessed a surge of attention on studying near-duplicate image/keyframe retrieval in the multimedia community. ...
Novelty detection for cross-lingual news stories with visual duplicates and speech transcripts
MM '07: Proceedings of the 15th ACM international conference on Multimedia

An overwhelming volume of news videos from different channels and languages is available today, which demands automatic management of this abundant information. To effectively search, retrieve, browse and track cross-lingual news stories, a news story ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

July 2007

655 pages

ISBN:9781595937339

DOI:10.1145/1282280

General Chairs:
Nicu Sebe
Univ. of Amsterdam, The Netherlands
,
Marcel Worring
Univ. of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CIVR07

Sponsor:

SIGMM

CIVR07: International Conference on Image and Video Retrieval 2007

July 9 - 11, 2007

Amsterdam, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
527
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sandeep RPrabin B(2022)Application of Perceptual Video Hashing for Near-duplicate Video RetrievalEvolutionary Computing and Mobile Sustainable Networks10.1007/978-981-16-9605-3_18(253-275)Online publication date: 22-Mar-2022
https://doi.org/10.1007/978-981-16-9605-3_18
Zhang CJin DXiao XChen GShyu M(2020)A Novel Collaborative Optimization Framework for Web Video Event Mining Based on the Combination of Inaccurate Visual Similarity Detection Information and Sparse Textual InformationIEEE Access10.1109/ACCESS.2020.29647148(10516-10527)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2964714
Shen LHong RHao Y(2020)Advance on large scale near-duplicate video retrievalFrontiers of Computer Science10.1007/s11704-019-8229-714:5Online publication date: 3-Jan-2020
https://doi.org/10.1007/s11704-019-8229-7
Yang XShyu MYu HSun SYin NChen W(2019)Integrating Image and Textual Information in Human–Robot Interactions for Children With Autism Spectrum DisorderIEEE Transactions on Multimedia10.1109/TMM.2018.286582821:3(746-759)Online publication date: Mar-2019
https://doi.org/10.1109/TMM.2018.2865828
Kleinrouweler JCabrero SCesar P(2017)An SDN Architecture for Privacy-Friendly Network-Assisted DASHACM Transactions on Multimedia Computing, Communications, and Applications10.1145/309283813:3s(1-22)Online publication date: 28-Jun-2017
https://dl.acm.org/doi/10.1145/3092838
Yu XBaek S(2017)Energy-Efficient Collection of Sparse Data in Wireless Sensor Networks Using Sparse Random MatricesACM Transactions on Sensor Networks10.1145/308557613:3(1-36)Online publication date: 16-Aug-2017
https://dl.acm.org/doi/10.1145/3085576
Vardasbi AFaili HAsadpour M(2017)SWIMACM Transactions on Information Systems10.1145/307265236:1(1-33)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.1145/3072652
Zhang QYang LLiu XChen ZLi P(2017)A Tucker Deep Computation Model for Mobile Multimedia Feature LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/306359313:3s(1-18)Online publication date: 10-Aug-2017
https://dl.acm.org/doi/10.1145/3063593
Wang XNie LSong XZhang DChua T(2017)Unifying Virtual and Physical WorldsACM Transactions on Information Systems10.1145/305277436:1(1-26)Online publication date: 6-Apr-2017
https://dl.acm.org/doi/10.1145/3052774
Jiang JTong YLu HCui BLei KYu L(2017)GVoSACM Transactions on Information Systems10.1145/304165736:1(1-36)Online publication date: 5-Jun-2017
https://dl.acm.org/doi/10.1145/3041657
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents