Personalized Content Recommender System via Non-verbal Interaction Using Face Mesh and Facial Expression
Authors: 
Yuya Moroto, Rintaro Yanagi, Naoki Ogawa, Kyohei Kamikawa, Keigo Sakurai, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki HaseyamaAuthors Info & Claims
Yuya Moroto, Rintaro Yanagi, Naoki Ogawa, Kyohei Kamikawa, Keigo Sakurai, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki HaseyamaAuthors Info & ClaimsPages 9399 - 9401
Abstract
Multimedia content recommendation needs to consider users' preferences for each content. Conventional recommender systems consider them with wearable sensors, however, wearing such sensors can lead to a burden on users. In this paper, we construct a recommender system that can explicitly estimate users' preferences without wearable sensors. Specifically, by constructing lightweight but strong machine learning models suitable for our system, the users' interest levels for contents can be estimated from facial images obtained from a widely used webcam. In addition, through the interaction that the user selects displayed contents, our system finds the tendency of personal preferences for recommending contents with high user satisfaction. Our system is available on https://www.lmd-demo.org/2022/start_eng.html.
Supplemental Material
MP4 File
The first half of the video shows the background of our demo system development and its architecture. The second half of the video introduces the features of the system while showing the actual operation of the demo system.
- Download
- 184.11 MB
References
[1]
Raman Arora and Karen Livescu. 2013. Multi-view CCA-based acoustic features for phonetic recognition across speakers and domains. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 7135--7139.
[2]
Octavio Arriaga, Matias Valdenegro-Toro, and Paul Plöger. 2017. Real-time convolutional neural networks for emotion and gender classification. arXiv preprint arXiv:1710.07557 (2017).
[3]
Deger Ayata, Yusuf Yaslan, and Mustafa E Kamasak. 2018. Emotion based music recommendation system using wearable physiological sensors. IEEE Transactions on Consumer Electronics, Vol. 64, 2 (2018), 196--203.
[4]
Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov, Karthik Raveendran, and Matthias Grundmann. 2019. Blazeface: Sub-millisecond neural face detection on mobile gpus. arXiv preprint arXiv:1907.05047 (2019).
[5]
Christiane Fellbaum. 1998. WordNet: An electronic lexical database. MIT Press.
[6]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 770--778.
[7]
Guang Huang, Qin Zhu, and Chee Siew. 2004. Extreme learning machine: A new learning scheme of feedforward neural networks. In Proceedings of IEEE International Joint Conference on Neural Networks, Vol. 2. 985--990.
[8]
Yury Kartynnik, Artsiom Ablavatski, Ivan Grishchenko, and Matthias Grundmann. 2019. Real-time facial surface geometry from monocular video on mobile GPUs. arXiv preprint arXiv:1907.06724 (2019).
[9]
Hyeyoung Ko, Suyeon Lee, Yoonseo Park, and Anna Choi. 2022. A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics, Vol. 11, 1 (2022), 141.
[10]
Qidong Liu, Jiaxi Hu, Yutian Xiao, Jingtong Gao, and Xiangyu Zhao. 2023. Multimodal Recommender Systems: A Survey. arXiv preprint arXiv:2302.03883 (2023).
[11]
Yoh Pao, Gwang Park, and Dejan Sobajic. 1994. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing, Vol. 6, 2 (1994), 163--180.
[12]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, Vol. 115 (2015), 211--252.
[13]
Luz Santamaria-Granados, Juan Francisco Mendoza-Moreno, Angela Chantre-Astaiza, Mario Munoz-Organero, and Gustavo Ramirez-Gonzalez. 2021. Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data. Sensors, Vol. 21, 23 (2021), 7854.
[14]
Wouter Schmidt, Martin Kraaijveld, and Robert Duin. 1992. Feedforward neural networks with random weights. In Proceedings of International Conference on Pattern Recognition Methodology and Systems. 1--4.
[15]
Marko Tkalcic, Ante Odic, Andrej Kosir, and Jurij Tasic. 2012. Affective labeling in a content-based recommender system for images. IEEE Transactions on Multimedia, Vol. 15, 2 (2012), 391--400.
[16]
Dandan Wang and Xiaoming Zhao. 2022. Affective video recommender systems: A survey. Frontiers in Neuroscience, Vol. 16 (2022), 984404
Index Terms
- Personalized Content Recommender System via Non-verbal Interaction Using Face Mesh and Facial Expression
Recommendations
Collaborative user modeling for enhanced content filtering in recommender systems
Recommender systems, which have emerged in response to the problem of information overload, provide users with recommendations of content suited to their needs. To provide proper recommendations to users, personalized recommender systems require ...
Adaptive personalized recommender system using learning automata and items clustering
AbstractThe personalized recommender systems provide user-related services based on user preferences; these preferences are recorded in an individual profile. Therefore, the more complete and precise each user profile leads more successful the ...
Highlights- Construct Adaptive user’s interest model using learning automata.
- Present ...
Improving Accuracy of Recommender System by Item Clustering
Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Demonstration
Funding Sources
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Acceptance Rates
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 100Total Downloads
- Downloads (Last 12 months)94
- Downloads (Last 6 weeks)4
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in