poster

Music video affective understanding using feature importance analysis

Authors:

Shiliang Zhang,

Qi TianAuthors Info & Claims

CIVR '10: Proceedings of the ACM International Conference on Image and Video Retrieval

Pages 213 - 219

https://doi.org/10.1145/1816041.1816074

Published: 05 July 2010 Publication History

Abstract

Music video is a popular type of entertainment by viewers. Currently, the novel indexing and retrieval approach based on the affective cues contained in music videos becomes more and more attractive to users. Music video affective analysis and understanding is one of the most popular topics in current multimedia community. In this paper, we propose a novel feature importance analysis approach to select most representative arousal and valence features for arousal and valence modeling. Compared with state-of-the-art work by Zhang on music video affective analysis, our main contributions are in the following aspects: (1) Another 3 affect-related features are extracted to enrich the feature set and exploit their correlation with arousal and valence. (2) All extracted features are ordered via feature importance analysis, and then optimal feature subset is selected after ordering. (3) Different regression methods are compared for arousal and valence modeling in order to find the fittest estimation function. Our method achieves 33.39% and 42.17% deduction in terms of mean absolute error compared with Zhang's method. Experimental results demonstrate our proposed method has a considerable improvement on music video affective understanding.

References

[1]

S. Arifin and P. Y. Cheung. User attention based arousal content modeling. In Proceedings of IEEE International Conference on Image Processing (ICIP), pages 433--436, March 2006.

[2]

S. Arifin and P. Y. Cheung. Affective level video segmentation by utilizing the pleasure-arousal-dominance information. IEEE Transactions on Multimedia, 10(7):1325--1341, November 2008.

Digital Library

[3]

R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. Wiley Interscience, New York, 2001.

Digital Library

[4]

S. R. Gunn. Support vector machines for classification and regression. Technical report, Image Speech and Intelligent Systems Research Group, University of Southampton, U.K., 1998.

[5]

A. Hanjalic and L.-Q. Xu. Affective video content representation and modeling. IEEE Transactions on Multimedia, 7(1):143--154, February 2005.

Digital Library

[6]

I. Jolliffe. Principle Component Analysis. Springer-Verlag, New York, 1986.

[7]

D. Li, I. K. Sethi, N. Dimitrova, and T. McGee. Classification of general audio data for content-based retrieval. Pattern Recognition Letters, 22:533--544, 2001.

Digital Library

[8]

L. Lu, D. Liu, and H.-J. Zhang. Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio, Speech, and Language Processing, 14(1):5--18, January 2006.

Digital Library

[9]

M. M. Ruxanda, B. Y. Chua, Alexandros, and C. S. Jensen. Emotion-based music retrieval on a well-reduced audio feature space. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 181--184, April 2009.

Digital Library

[10]

M. Soleymani, G. Chanel, J. J. Kierkels, and T. Pun. Affective characterization of movie scenes based on multimedia content analysis and user's physiological emotional responses. In Proceedings of the Tenth IEEE International Symposium on Multimedia, pages 228--235, 2008.

Digital Library

[11]

K. Sun, J. Yu, Y. Huang, and X. Hu. An improved valence-arousal emotion space for video affective content representation and recognition. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME), pages 566--569, 2009.

Digital Library

[12]

P. Valdez and A. Mehrabian. Effects of color on emotions. Journal of Experimental Psychology, 123:394--409, 1994.

[13]

V. N. Vapnik. Statistical Learning Theory. John Wiley and Sons, Inc., New York, 1998.

Digital Library

[14]

H. L. Wang and L.-F. Cheong. Affective understanding in film. IEEE Transactions on Circuits and Systems for Video Technology, 16(6):689--704, June 2006.

Digital Library

[15]

S. Weisberg. Applied Linear Regression. Wiley/Interscience, New York, 2005.

[16]

M. Xu, J. S. Jin, S. Luo, and L. Duan. Hierarchical movie affective content analysis based on arousal and valence features. In Proceedings of ACM Multimedia, pages 677--680, 2008.

Digital Library

[17]

S. Zhang, Q. Huang, Q. Tian, S. Jiang, and W. Gao. i. mtv - an integrated system for mtv affective analysis. In Proceedings of ACM Multimedia (demenstration), pages 985--986, 2008.

Digital Library

[18]

S. Zhang, Q. Huang, Q. Tian, S. Jiang, and W. Gao. Personalized mtv affective analysis using user profile. In Proceedings of the 9th Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, pages 327--337, 2008.

Digital Library

[19]

S. Zhang, Q. Tian, Q. Huang, W. Gao, and S. Li. Utilizing affective analysis for effective movie browsing. In Proceedings of IEEE International Conference on Image Processing (ICIP), pages 677--680, 2009.

Digital Library

[20]

S. Zhang, Q. Tian, S. Jiang, Q. Huang, and W. Gao. Affective mtv analysis based on arousal and valence features. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME), pages 1369--1372, 2008.

[21]

T. Zhang and C.-C. J. Kuo. Audio content analysis for online audiovisual data segmentation and classification. IEEE Transactions on Speech and Audio Processing, 9(4):441--457, May 2001.

Cited By

Wang SWang CChen TWang YShu YJi Q(2021)Video Affective Content Analysis by Exploring Domain KnowledgeIEEE Transactions on Affective Computing10.1109/TAFFC.2019.291237712:4(1002-1017)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TAFFC.2019.2912377
Jiang DJin DZhuang JTan DChen DLiang Y(2021)A computational model of emotion based on audio‐visual stimuli understanding and personalized regulation with concurrencyConcurrency and Computation: Practice and Experience10.1002/cpe.626933:17Online publication date: 31-May-2021
https://doi.org/10.1002/cpe.6269
Jiang DWu KChen DTu GZhou TGarg AGao L(2020)A probability and integrated learning based classification algorithm for high-level human emotion recognition problemsMeasurement10.1016/j.measurement.2019.107049150(107049)Online publication date: Jan-2020
https://doi.org/10.1016/j.measurement.2019.107049
Show More Cited By

Index Terms

Music video affective understanding using feature importance analysis
1. Information systems
  1. Information retrieval
    1. Document representation
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Abstraction

Recommendations

Hierarchical movie affective content analysis based on arousal and valence features
MM '08: Proceedings of the 16th ACM international conference on Multimedia

Emotional factors directly reflect audiences' attention, evaluation and memory. Affective contents analysis not only create an index for users to access their interested movie segments, but also provide feasible entry for video highlights. Most of the ...
Affective content analysis of music video clips
MIRUM '11: Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies

Nowadays, the amount of multimedia contents is explosively increasing and it is often a challenging problem to find a content that will be appealing or matches users' current mood or affective state. In order to achieve this goal, an effcient indexing ...
Automatic highlights extraction for drama video using music emotion and human face features

This paper describes a novel system that uses music emotion and human face as features for automatic highlights extraction for drama video. These high-level audiovisual features are used because music evokes emotion response from the viewer and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '10: Proceedings of the ACM International Conference on Image and Video Retrieval

July 2010

492 pages

ISBN:9781450301176

DOI:10.1145/1816041

Conference Chairs:
Shipeng Li
Microsoft Research Asia, China
,
Xinbo Gao
Xidian University, China
,
Nicu Sebe
University of Trento, Italy

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIVR' 10

Sponsor:

SIGMM

CIVR' 10: International Conference on Image and Video Retrieval

July 5 - 7, 2010

Xi'an, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
492
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang SWang CChen TWang YShu YJi Q(2021)Video Affective Content Analysis by Exploring Domain KnowledgeIEEE Transactions on Affective Computing10.1109/TAFFC.2019.291237712:4(1002-1017)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TAFFC.2019.2912377
Jiang DJin DZhuang JTan DChen DLiang Y(2021)A computational model of emotion based on audio‐visual stimuli understanding and personalized regulation with concurrencyConcurrency and Computation: Practice and Experience10.1002/cpe.626933:17Online publication date: 31-May-2021
https://doi.org/10.1002/cpe.6269
Jiang DWu KChen DTu GZhou TGarg AGao L(2020)A probability and integrated learning based classification algorithm for high-level human emotion recognition problemsMeasurement10.1016/j.measurement.2019.107049150(107049)Online publication date: Jan-2020
https://doi.org/10.1016/j.measurement.2019.107049
Xing BZhang HZhang KZhang LWu XShi XYu SZhang S(2019)Exploiting EEG Signals and Audiovisual Feature Fusion for Video Emotion RecognitionIEEE Access10.1109/ACCESS.2019.29148727(59844-59861)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2914872
Chen TWang YWang SChen SLiu QLienhart RWang HChen SBoll SChen PFriedland GLi JYan S(2017)Exploring Domain Knowledge for Affective Video Content AnalysesProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123352(769-776)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3123266.3123352
Niu JSu YMo SZhu Z(2016)A Novel Affective Visualization System for Videos Based on Acoustic and Visual FeaturesMultiMedia Modeling10.1007/978-3-319-51814-5_2(15-27)Online publication date: 31-Dec-2016
https://doi.org/10.1007/978-3-319-51814-5_2
Wang SWang JWang ZJi Q(2015)Multiple Emotion Tagging for Multimedia Data by Exploiting High-Order Dependencies Among EmotionsIEEE Transactions on Multimedia10.1109/TMM.2015.248496617:12(2185-2197)Online publication date: Dec-2015
https://doi.org/10.1109/TMM.2015.2484966
Wang SJi Q(2015)Video Affective Content Analysis: A Survey of State-of-the-Art MethodsIEEE Transactions on Affective Computing10.1109/TAFFC.2015.24327916:4(410-430)Online publication date: 1-Oct-2015
https://doi.org/10.1109/TAFFC.2015.2432791
Acar EHopfgartner FAlbayrak S(2014)Understanding Affective Content of Music Videos through Learned RepresentationsProceedings of the 20th Anniversary International Conference on MultiMedia Modeling - Volume 832510.1007/978-3-319-04114-8_26(303-314)Online publication date: 6-Jan-2014
https://dl.acm.org/doi/10.1007/978-3-319-04114-8_26
Acar EJaimes ASebe NBoujemaa NGatica-Perez DShamma DWorring MZimmermann R(2013)Learning representations for affective video understandingProceedings of the 21st ACM international conference on Multimedia10.1145/2502081.2502215(1055-1058)Online publication date: 21-Oct-2013
https://dl.acm.org/doi/10.1145/2502081.2502215
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents