Article

Online video recommendation based on multimodal fusion and relevance feedback

Authors:

Xian-Sheng Hua,

Shi-Qiang Yang,

Mingjing LiAuthors Info & Claims

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

Pages 73 - 80

https://doi.org/10.1145/1282280.1282290

Published: 09 July 2007 Publication History

Abstract

With Internet delivery of video content surging to an un-precedented level, video recommendation has become a very popular online service. The capability of recommending relevant videos to targeted users can alleviate users' efforts on finding the most relevant content according to their current viewings or preferences. This paper presents a novel online video recommendation system based on multimodal fusion and relevance feedback. Given an online video document, which usually consists of video content and related information (such as query, title, tags, and surroundings), video recommendation is formulated as finding a list of the most relevant videos in terms of multimodal relevance. We express the multimodal relevance between two video documents as the combination of textual, visual, and aural relevance. Furthermore, since different video documents have different weights of the relevance for three modalities, we adopt relevance feedback to automatically adjust intra-weights within each modality and inter-weights among different modalities by users' click-though data, as well as attention fusion function to fuse multimodal relevance together. Unlike traditional recommenders in which a sufficient collection of users' profiles is assumed available, this proposed system is able to recommend videos without users' profiles. We conducted an extensive experiment on 20 videos searched by top 10 representative queries from more than 13k online videos, reported the effectiveness of our video recommendation system.

References

[1]

http://soapbox.msn.com/.

[2]

http://video.google.com/.

[3]

http://video.msn.com/.

[4]

http://video.yahoo.com/.

[5]

http://www.myspace.com/.

[6]

http://www.youtube.com/.

[7]

R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison Wesley, 1999.

Digital Library

[8]

M. Balabanovic. Exploring versus exploiting when learning user models for text recommendation. User Modeling and User-Adapted Interaction, 8(4):71--102, Nov 1998.

Digital Library

[9]

C. Christakou and A. Stafylopatis. A hybrid movie recommender system based on neural networks. In Proceedings of the 2005 5th International Conference on Intelligent Systems Design and Applications, Wroclaw, Poland, 2005.

Digital Library

[10]

A. G. Hauptmann, W. H. Lin, R. Yan, J. Yang, and M. Y. Chen. Extreme video retrieval: Joint maximization of human and computer performance. In Proceedings of the ACM International Conference on Multimedia, Santa Barbara, USA, 2006.

Digital Library

[11]

W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Video search reranking via information bottleneck principle. In Proceedings of the ACM International Conference on Multimedia, Santa Barbara, USA, 2006.

Digital Library

[12]

X.-S. Hua, L. Lu, and H.-J. Zhang. Optimization-based automated home video editing system. IEEE Trans. on Circuit and System for Video Technology, 14(5):572--583, May 2004.

Digital Library

[13]

X.-S. Hua, T. Mei, W. Lai, and et al. Microsoft Research Asia TRECVID 2006 high-level feature extraction and rushes exploitation. In TREC Video Retrieval Evaluation Online Proceedings, 2006.

[14]

X.-S. Hua and H.-J. Zhang. An attention-based decision fusion scheme for multimedia information retrieval. In Proceedings of IEEE Pacific-Rim Conference On Multimedia, Tokyo, Japan, 2004.

Digital Library

[15]

M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. on Multimedia Computing, Communications and Applications, 2(1):1--19, Feb 2006.

Digital Library

[16]

H. Mak, I. Koprinska, and J. Poon. INTIMATE: A web-based movie recommender using text categorization. In Proceedings of the IEEE/WIC International Conference on Web Intelligence, Beijing, China, 2003.

Digital Library

[17]

Online Publishers. http://www.online-publishers.org/.

[18]

P. Resnick and H. R. Varian. Recommender systems. Communications of the ACM, 40(3):56--58, May 1997.

Digital Library

[19]

Y. Rui, T. S. Huang, and M. Ortega. Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans. on Circuit and System for Video Technology, 8(5):644--655, Sep 1998.

Digital Library

[20]

M. V. Setten and M. Veenstra. Prediction strategies in a TV recommender system - method and experiments. In Proceedings of International World Wide Web Conference, Budapest, Hungary, 2003.

[21]

Y. Yang and X. Liu. A re-examination of text categorization methods. In Proceedings of ACM SIGIR conference on Research and development in information retrieval, California, USA, 1999.

Digital Library

Cited By

Mondal PKapoor PSingh SSaha SSingh JSingh A(2024)Genre Effect Toward Developing a Multi-Modal Movie Recommendation System in Indian SettingIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332400970:1(2517-2526)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3324009
Motamedi EKholgh DSaghari SElahi MBarile FTkalcic M(2024)Predicting movies’ eudaimonic and hedonic scores: A machine learning approach using metadata, audio and visual featuresInformation Processing & Management10.1016/j.ipm.2023.10361061:2(103610)Online publication date: Mar-2024
https://doi.org/10.1016/j.ipm.2023.103610
Mu YWu Y(2023)Multimodal Movie Recommendation System Using Deep LearningMathematics10.3390/math1104089511:4(895)Online publication date: 10-Feb-2023
https://doi.org/10.3390/math11040895
Show More Cited By

Index Terms

Online video recommendation based on multimodal fusion and relevance feedback
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Information systems
  1. World Wide Web
    1. Web applications
    2. Web services

Recommendations

Contextual Video Recommendation by Multimodal Relevance and User Feedback

With Internet delivery of video content surging to an unprecedented level, video recommendation, which suggests relevant videos to targeted users according to their historical and current viewings or preferences, has become one of most pervasive online ...
An online video recommendation framework using rich information
ICIMCS '11: Proceedings of the Third International Conference on Internet Multimedia Computing and Service

Automatic video recommendation is involved in an attempt to tackle the information-overload problem, aiming to present the personalized video list to the user. This paper presents a novel approach to improve the accuracy of the video recommendation by ...
Multimodal retrieval with relevance feedback based on genetic programming

This paper presents a framework for multimodal retrieval with relevance feedback based on genetic programming. In this supervised learning-to-rank framework, genetic programming is used for the discovery of effective combination functions of (multimodal)...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '07: Proceedings of the 6th ACM international conference on Image and video retrieval

July 2007

655 pages

ISBN:9781595937339

DOI:10.1145/1282280

General Chairs:
Nicu Sebe
Univ. of Amsterdam, The Netherlands
,
Marcel Worring
Univ. of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CIVR07

Sponsor:

SIGMM

CIVR07: International Conference on Image and Video Retrieval 2007

July 9 - 11, 2007

Amsterdam, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

117
Total Citations
View Citations
1,858
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)6

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mondal PKapoor PSingh SSaha SSingh JSingh A(2024)Genre Effect Toward Developing a Multi-Modal Movie Recommendation System in Indian SettingIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332400970:1(2517-2526)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3324009
Motamedi EKholgh DSaghari SElahi MBarile FTkalcic M(2024)Predicting movies’ eudaimonic and hedonic scores: A machine learning approach using metadata, audio and visual featuresInformation Processing & Management10.1016/j.ipm.2023.10361061:2(103610)Online publication date: Mar-2024
https://doi.org/10.1016/j.ipm.2023.103610
Mu YWu Y(2023)Multimodal Movie Recommendation System Using Deep LearningMathematics10.3390/math1104089511:4(895)Online publication date: 10-Feb-2023
https://doi.org/10.3390/math11040895
Zioutos KKondylakis HStefanidis K(2023)Healthy Personalized Recipe Recommendations for Weekly Meal PlanningComputers10.3390/computers1301000113:1(1)Online publication date: 20-Dec-2023
https://doi.org/10.3390/computers13010001
Ho TLe AVu D(2023)Multiview Fusion Using Transformer Model for Recommender Systems: Integrating the Utility Matrix and Textual SourcesApplied Sciences10.3390/app1310632413:10(6324)Online publication date: 22-May-2023
https://doi.org/10.3390/app13106324
Wang YLee A(2023)Movie Account Recommendation on InstagramACM Transactions on Internet Technology10.1145/357985223:1(1-21)Online publication date: 13-Jan-2023
https://dl.acm.org/doi/10.1145/3579852
Pati USwain SKumari RMishra VBandyopadhyay A(2023)Excavating Multimodal Correlation for Movie Recommendation System2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT56998.2023.10306669(1-7)Online publication date: 6-Jul-2023
https://doi.org/10.1109/ICCCNT56998.2023.10306669
Chen XWang YYang J(2023)Multi-modal Recommendation based on Knowledge Graph2023 9th International Conference on Computer and Communications (ICCC)10.1109/ICCC59590.2023.10507494(2383-2388)Online publication date: 8-Dec-2023
https://doi.org/10.1109/ICCC59590.2023.10507494
Zhang HMu XYan HRen LMa J(2023)A survey of online video advertisingWIREs Data Mining and Knowledge Discovery10.1002/widm.148913:2Online publication date: 18-Jan-2023
https://doi.org/10.1002/widm.1489
Kang SChung K(2022)Preference-Tree-Based Real-Time Recommendation SystemEntropy10.3390/e2404050324:4(503)Online publication date: 2-Apr-2022
https://doi.org/10.3390/e24040503
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents