Article

Video summarization based on user log enhanced link analysis

Authors:

Klara Nahrstedt,

Hong-Jiang ZhangAuthors Info & Claims

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

Pages 382 - 391

https://doi.org/10.1145/957013.957095

Published: 02 November 2003 Publication History

Abstract

Efficient video data management calls for intelligent video summarization tools that automatically generate concise video summaries for fast skimming and browsing. Traditional video summarization techniques are based on low-level feature analysis, which generally fails to capture the semantics of video content. Our vision is that users unintentionally embed their understanding of the video content in their interaction with computers. This valuable knowledge, which is difficult for computers to learn autonomously, can be utilized for video summarization process. In this paper, we present an intelligent video browsing and summarization system that utilizes previous viewers' browsing log to facilitate future viewers. Specifically, a novel ShotRank notion is proposed as a measure of the subjective interestingness and importance of each video shot. A ShotRank computation framework is constructed to seamlessly unify low-level video analysis and user browsing log mining. The resulting ShotRank is used to organize the presentation of video shots and generate video skims. Experimental results from user studies have strongly confirmed that ShotRank indeed represents the subjective notion of interestingness and importance of each video shot, and it significantly improves future viewers' browsing experience.

References

[1]

A. Hanjialic and H.J. Zhang, An Integrated Scheme for Automated Video Summarization Based on Unsupervised Cluster-Validity Analysis, in IEEE Transactions on Circuits and Systems for Video Technology, VOL. 9, NO. 8, PP. 1280--1289, December 1999.

Digital Library

[2]

D. DeMenthon, V. Kobla and D. Doermann, in Proc. of Video Summarization by Curve Simplification, ACM MM 1998, United Kingdom, September 1998.

Digital Library

[3]

M. Smith and T. Kanade, Video Skimming for Quick Browsing Based on Audio and Image Characterization, Carnegie Mellon University, Technical Report CMU-CS-95-186, July 1995.

[4]

H.J. Zhang, C.Y. Low, S.W. Smoliar and J.H. Wu, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, in Proc. of ACM MM 1995, California, November 1995.

Digital Library

[5]

V. Kobla and D. Doermann, VideoTrails: Representing and Visualizing Structure in Video Sequences, in Proc. of ACM MM 1997, Seattle, November 1997.

Digital Library

[6]

H. Aoki, S. Shimotsuji and O. Hori, A Shot Classification Method of Selecting Effective Key-Frames for Video Browsing, in Proc. of ACM MM 1996, Massachusetts, November 1996.

Digital Library

[7]

Y. Rui, A. Gupta and A. Acero, Automatically Extracting Highlights for TV Baseball Programs, ACM MM 2000, California, November 2000.

Digital Library

[8]

S. Uchihashi, J. Foote, A. Girgensohn and J. Boreczky, Video Manga: Generating Semantically Meaningful Video Summaries, in Proc. of ACM MM 1999, Florida, October 1999.

Digital Library

[9]

J. Nam and A. H. Tewfik, Dynamic Video Summarization and Visualization, ACM MM 1999, Florida, October 1999.

Digital Library

[10]

M. M. Yeung and B. L. Yeo, Time-constrained Clustering for Segmentation of Video into Story Units, in Proc. of International Conference on Pattern Recognition, August 1996.

Digital Library

[11]

L.W. He, E. Sanocki, A. Gupta and J. Grudin, Auto-Summarization of Audio-Video Presentations, in Proc. of ACM MM 1999, Florida, October 1999.

Digital Library

[12]

R. Lienhart, Abstracting Home Video Automatically, in Proc. of ACM MM 1999, Florida, October 1999.

Digital Library

[13]

Y.F. Ma, L. Lu, H.J. Zhang and M. Li, An Attention Model for Video Summarization, in Proc. of ACM MM 2002, France, December 2002.

Digital Library

[14]

Informedia Project, http://www.informedia.cs.cmu.edu/

[15]

DirecHit: http://www.directhit.com

[16]

S. Acharya, B. Smith, and P. Parnes, Characterizing User Access To Videos On the World Wide Web, in Proc. of Multimedia Computing and Networking 2000, California, January 2000.

[17]

T. F. Syeda-Mahmood and D. Ponceleon, Learning Video Browsing Behavior and its Application in the Generation of Video Previews, in Proc. of ACM MM 2001, Canada, October 2001.

Digital Library

[18]

X. Zhu, J. Fan and A.K. Elmagarmid, Hierarchical Video Summarization and Content Description Joint Semantic and Visual Similarity, in ACM Multimedia Systems, VOL 8, NO 5, 2003.

Digital Library

[19]

S. Brin and L. Page, The Anatomy of a Large-Scale Hypertextual Web Search Engine, in Proc. of 7th International WWW Conference, Australia, April 1998.

Digital Library

[20]

Y. Wang, P. Zhao, D. Zhang, M. Li and H.J. Zhang, MyVideos - A System for Home Video Management, in Proc. of ACM MM 2002, France, December 2002.

Digital Library

[21]

H. J. Zhang, A. Kankanhalli, S. W. Smoliar, Automatic Partitioning of Full-motion Video, in ACM Multimedia Systems, VOL. 1, NO.1, PP. 10--28, 1993.

Digital Library

[22]

P. Bouthemy, Y. Dufournaud, R. Fablet, R. Mohr, S. Peleg, and A. Zomet, Video Hyper-links Creation for Content-Based Browsing and Navigation, in Proc. of Workshop on Content-Based Multimedia Indexing, France, October 1999.

[23]

W.-Y. Ma and H.J. Zhang, An Indexing and Browsing System for Home Video, In Proc. of European Conference on Signal Processing, Greece, September 2000.

[24]

K. Ntalianis, A. Doulamis, N. Doulamis, and S. Kollias, Non-Sequential Video Structuring Based on Video Object Linking: An Efficient Tool for Video Browsing and Indexing, in Proc. of IEEE Int. Conf. Image Processing, Greece, October 2001.

[25]

Z. Chen, L. Tao, J. Wang, W. Liu and W.Y. Ma, A Unified Framework for Web Link Analysis, in Proc. of WISE 2002, Singapore, December 2002

Digital Library

[26]

J. Kleinberg, Authoritative Sources in a Hyperlinked Environment, in Journal of ACM, VOL. 46, NO. 5, PP. 604--632, 1999.

Digital Library

[27]

H. Zhang, S. Smoliar and J. Wu. Content-based Video Browsing Tools, in A. Rodriguez and J. Maitan, editors, Symposium on Electronic Imaging Science and Technology: Multimedia Computing and Networking, SPIE VOL. 2417, PP. 389--398, 1995.

Cited By

Kansal KKansal NBavana SVamshi BGoyal NKankanhalli MPatras ILiu JWong YKomamizu T(2023)A Systematic Study on Video Summarization: Approaches, Challenges, and Future DirectionsProceedings of the 2nd Workshop on User-centric Narrative Summarization of Long Videos10.1145/3607540.3617139(65-73)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3607540.3617139
Brunyé TDrew TWeaver DElmore J(2019)A review of eye tracking for understanding and improving diagnostic interpretationCognitive Research: Principles and Implications10.1186/s41235-019-0159-24:1Online publication date: 22-Feb-2019
https://doi.org/10.1186/s41235-019-0159-2
Mollashahi EUddin MGutwin CCatarci TNorman KMecella M(2018)Improving revisitation in long documents with two-level artificial-landmark scrollbarsProceedings of the 2018 International Conference on Advanced Visual Interfaces10.1145/3206505.3206554(1-9)Online publication date: 29-May-2018
https://dl.acm.org/doi/10.1145/3206505.3206554
Show More Cited By

Index Terms

Video summarization based on user log enhanced link analysis
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization

Recommendations

A user attention model for video summarization
MULTIMEDIA '02: Proceedings of the tenth ACM international conference on Multimedia

Automatic generation of video summarization is one of the key techniques in video management and browsing. In this paper, we present a generic framework of video summarization based on the modeling of viewer's attention. Without fully semantic ...
A generic framework of user attention model and its application in video summarization

Due to the information redundancy of video, automatically extracting essential video content is one of key techniques for accessing and managing large video library. In this paper, we present a generic framework of a user attention model, which ...
Rushes video summarization using audio-visual information and sequence alignment
TVS '08: Proceedings of the 2nd ACM TRECVid Video Summarization Workshop

This paper describes our system and methodologies for the BBC rushes video summarization task of TRECVID 2008. The procedure of the system is composed of three major steps: shot detection, irrelevant and repetitive subshot removal, and final summary ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

November 2003

670 pages

ISBN:1581137222

DOI:10.1145/957013

General Chairs:
Lawrence Rowe
University of California, Berkeley
,
Harrick Vin
University of Texas, Austin
,
Program Chairs:
Thomas Plagemann
University of Oslo
,
Prashant Shenoy
University of Massachusetts, Amherst
,
John R. Smith
IBM T.J. Watson Research Center

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM03

Sponsor:

MM03: 2003 11th Annual ACM International Conference on Multimedia

November 2 - 8, 2003

CA, Berkeley, USA

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
1,167
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kansal KKansal NBavana SVamshi BGoyal NKankanhalli MPatras ILiu JWong YKomamizu T(2023)A Systematic Study on Video Summarization: Approaches, Challenges, and Future DirectionsProceedings of the 2nd Workshop on User-centric Narrative Summarization of Long Videos10.1145/3607540.3617139(65-73)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3607540.3617139
Brunyé TDrew TWeaver DElmore J(2019)A review of eye tracking for understanding and improving diagnostic interpretationCognitive Research: Principles and Implications10.1186/s41235-019-0159-24:1Online publication date: 22-Feb-2019
https://doi.org/10.1186/s41235-019-0159-2
Mollashahi EUddin MGutwin CCatarci TNorman KMecella M(2018)Improving revisitation in long documents with two-level artificial-landmark scrollbarsProceedings of the 2018 International Conference on Advanced Visual Interfaces10.1145/3206505.3206554(1-9)Online publication date: 29-May-2018
https://dl.acm.org/doi/10.1145/3206505.3206554
Uddin MGutwin CGoguey ASimeone AJohnsen KTeather RSandor C(2017)Using artificial landmarks to improve revisitation performance and spatial learning in linear control widgetsProceedings of the 5th Symposium on Spatial User Interaction10.1145/3131277.3132184(48-57)Online publication date: 16-Oct-2017
https://dl.acm.org/doi/10.1145/3131277.3132184
Zhang HSun MWang XSong ZTang JSun JBarrett RCummings RAgichtein EGabrilovich E(2017)Smart JumpProceedings of the 26th International Conference on World Wide Web Companion10.1145/3041021.3054166(331-339)Online publication date: 3-Apr-2017
https://dl.acm.org/doi/10.1145/3041021.3054166
Uddin MGutwin CCockburn AMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)The Effects of Artificial Landmarks on Learning and Performance in Spatial-Memory InterfacesProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025497(3843-3855)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025497
Cheng ZWu XLiu YHua X(2017)Video eCommerce++IEEE Transactions on Multimedia10.1109/TMM.2016.264738619:6(1170-1183)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1109/TMM.2016.2647386
Raiturkar PKleinsmith AKeil ABanerjee AJain EJain EJoerg S(2016)Decoupling light reflex from pupillary dilation to measure emotional arousal in videosProceedings of the ACM Symposium on Applied Perception10.1145/2931002.2931009(89-96)Online publication date: 22-Jul-2016
https://dl.acm.org/doi/10.1145/2931002.2931009
Zen Gde Juan PSong YJaimes AKender JSmith JLuo JBoll SHsu W(2016)Mouse Activity as an Indicator of Interestingness in VideoProceedings of the 2016 ACM on International Conference on Multimedia Retrieval10.1145/2911996.2912005(47-54)Online publication date: 6-Jun-2016
https://dl.acm.org/doi/10.1145/2911996.2912005
Avlonitis M(2016)On the problem of early detection of users interaction outbreaks via stochastic differential modelsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2016.01.00851:C(92-96)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1016/j.engappai.2016.01.008
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents