Evaluating a recommendation application for online video content: an interdisciplinary study
Authors: 
Katrien De Moor, Toon De Pessemier, 
Peter Mechant, Cédric Courtois, Adrian Juan, Lieven De Marez, Luc MartensAuthors Info & Claims
Katrien De Moor, Toon De Pessemier, Peter Mechant, Cédric Courtois, Adrian Juan, Lieven De Marez, Luc MartensAuthors Info & ClaimsPages 115 - 122
Abstract
In this paper, we discuss the set-up and results from an interdisciplinary study aimed at evaluating a recommendation application for online video content, called PersonalTV. By involving (possible) users (i.e. a panel of test users), we tried to gather insights that might help to optimize and refine the application. In this respect, implicit and explicit user feedback were complemented. This paper explores the relation between the PersonalTV suggestions (recommended content) and the consumption percentage (objective data) (RQ 1) and between the recommended content and the reported satisfaction (subjective data) (RQ 2) of the test users. We also investigated whether the objective and subjective measures converge (RQ 3) and collected feedback that suggests measures for further improvement and optimization of the application.
References
[1]
Jones, Q., Ravid, G., & Rafaeli, S. (2004). Information overload and the message dynamics of online interaction spaces. Information Systems Research, 15(2), 194--210.
[2]
Tidline, T. (2002). Information Overload. In A. Kent (Ed.), Encyclopedia of Library and Information Science (Vol. 72, pp. 217--233).
[3]
Toffler, A. (1981). The third wave. New York: Bantam Books Trott, P. ( 2003). Innovation and Market Research. In: L.V. Shavinina (Ed.), The International Handbook on Innovation (pp. 835--844). Oxford: Pergamon, Elsevier, pp. 835--844.
[4]
De Pessemier, T., Deryckere, T. & Martens, L. (2009). Context-aware recommendations for user-generated content on a social network site. Proceedings of EuroITV'09.
[5]
De Pessemier, T., Ide, M. & Martens, L. (2008). Consumption context and personalization. Proceedings of EuroIVT'08.
[6]
De Marez, L. and Verleye, G. (2004). Innovation diffusion: The need for more accurate consumer insight. Illustration of the PSAP scale as a segmentation instrument. Journal of Targeting, Measurement and Analysis for Marketing, 13(1), 32--49.
[7]
Veryzer, R. W. and Borja de Mozota, B. (2005). The impact of User-Oriented Design on New Product Development: An Examination of Fundamental Relationships. The Journal of Product Innovation Management, 22, 128--143.
[8]
Von Hippel, E 2005. Democratizing Innovation. Cambridge: MIT Press.
[9]
Haddon, L., Mante, E., Sapio, B., Kommonen, K.-H., Fortunati, L. and Kant, A. (Eds)(2005). Everyday Innovators: Researching the role of users in shaping ICT's. Dordrecht: Springer.
[10]
Khurana, A. and Rosenthal, S.R. (1998). Toward Holistic 'Front End' in New Product Development. Journal of Product Innovation Management, 15(1), 57--74.
[11]
Kristensson, P., Gustafsson, A. and Archer, T. (2004). Harnessing the Creative Potential among Users. Journal of Product Innovation Management, 21, 4--14.
[12]
Haddon, L, Paul, G. (2001). Design in the ICT industry: the role of users. In: R. Coombs, K. Green, A. Richards and V. Walsh (Eds.), Technology and the Market: Demand, Users and Innovation, Cheltenham: Edward Elgar Publishing.
[13]
Lettl, C. (2007). User involvement competence for radical innovation export. Journal of Engineering and Technology Management, 24(1--2), 53--75.
[14]
Limonard, S. and de Koning, N. (2005). Dealing with Dilemmas in Pre-competitive ICT Development Projects: The Construction of 'The Social' in Designing New Technologies. In: L. Haddon, E. Mante, B. Sapio, K.-H. Kommonen, L. Fortunati and A. Kant (Eds.). Everyday Innovators: Researching the Role of Users in Shaping ICT's (pp. 155--167). Dordrecht: Springer.
[15]
Rohracher, H. (2005). From passive consumers to active participants: The diverse roles of users in innovation processes. In: H. Rohracher (Ed.). User Involvement in Innovation Processes. Strategies and Limitations from a Socio--Technical Perspective (9--35). Munich: Profil-Verlag.
[16]
Lievrouw, L. (2006). New media design and development: diffusions of innovations v social shaping of technology. In: Lievrouw, L. and Livingstone, S. (Eds.). The handbook of New Media (pp. 246--265), London: Sage.
[17]
Rickards, T. (2003). The Future of Innovation Research. In: L.V. Shavinina (Ed.), The International Handbook on Innovation (pp. 1094--1100). Oxford: Pergamon, pp. 1094--1100.
[18]
Trott, P. ( 2003). Innovation and Market Research. In: L.V. Shavinina (Ed.), The International Handbook on Innovation (pp. 835--844). Oxford: Pergamon, Elsevier.
[19]
Stone, A., Turkkan, J.S., Bachrach, C.A. et al. (Eds.)(2000). The Science of Self-report: Implications for Research and Practice. Mahwah: Lawrence Erlbaum Associates, Inc.
[20]
Burke, R. (2007). Hybrid web recommender systems. In P. Brusilovsky, A. Kobsa & W. Nejdl (Eds.), The adaptive Web (pp. 377--408). Berlin: Springer-Verlag.
[21]
Resnick, P., & Varian, H. (1997). Recommender Systems. Communications of the ACM, 40(3), 56--58.
[22]
Chorianopoulos, K. (2008). Personalized and mobile digital TV applications. Multimedia Tools and Applications, 36(1--2), 1--10.
[23]
Pazzani, M. (1999). A framework for collaborative, content-based and demographic filtering. Artifical Intelligence Review 13(5/6), 393--408.
[24]
Barragáns Martínez, A.B., Pazos Arias, J.J., Fernández Vilas, A et al. (2009). What's on TV Tonight? An efficient and effective personalized recommender system of TV Programs. IEEE Transactions on Consumer Electronics, 55(1), 286--294.
[25]
Schein, A., Popescul, A., Ungar. L., Pennock, D. (2002). Methods and metrics for cold-start recommendations. Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, 253--260.
[26]
Papagelis, M., Plexousakis, D., Kutsuras, T. (2005). Alleviating the Sparsity Problem of Collaborative Filtering Using Trust Inferences. Lecture Notes in Computer Science -- Trust Management 3477, 224--239.
[27]
Weng, J., Miao, C., Goh, A., Shen, Z., Gay, R. (2006). Trust-based agent community for collaborative recommendation. Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems, 1260--1262.
[28]
McCracken, G. (2007). How social networks work: the puzzle of exhaust data {Electronic Version} from http://www.cultureby.com/trilogy/2007/07/how-social-netw.html.
[29]
Najjar, J., Wolpers, M., & Duval, E. (2006). Attention Metadata: Collection and Management {Electronic Version} from http://ariadne.cs.kuleuven.be/empirical/papers/www2006.pdf.
[30]
Hill, W. C., Hollan, J. D., Wroblewski, D., & McCandless, T. (1992). Edit Wear and Read Wear. Paper presented at the ACM Conference on Human Factors in Computing Systems (CHI'92), New York City, New York.
[31]
Kedrosky, P. (2005). Drive-By Data & Web 2.0 {Electronic Version} from http://paul.kedrosky.com/archives/2005/06/driveby_communi.html.
[32]
Najjar, J., Wolpers, M., & Duval, E. (2006). Attention Metadata: Collection and Management {Electronic Version} from http://ariadne.cs.kuleuven.be/empirical/papers/www2006.pdf.
[33]
Boyd, S. (2005). Starting From Scratch: Social Design Is Hard {Electronic Version} from http://getreal.corante.com/archives/2005/07/07/starting_from_scratch_social_design_is_hard.php.
[34]
Jaokar, A. (2006). Tim O' Reilly's seven principles of web 2.0 make a lot more sense if you change the order {Electronic Version} from http://opengardensblog.futuretext.com/archives/2006/04/a_web_20_faq.html.
[35]
Warr, W. A. (2008). Social software: fun and games, or business tools. Journal of Information Science, 34(4), 591--604.
[36]
Hoegg, R., Martignoni, R., Meckel, M., & Stanoevska-Slabeva, K. (2006). Overview of business models for web 2.0 communities. Paper presented at the GeNeMe 2006, Dresden.
Index Terms
- Evaluating a recommendation application for online video content: an interdisciplinary study
Recommendations
Users' (Dis)satisfaction with the personalTV application: Combining objective and subjective data
Theoretical and Practical Computer Applications in EntertainmentThe overabundance of content on online video platforms has made intelligent recommender systems that assist users in finding content matching their personal preferences indispensable. This article reports on a study in which “PersonalTV,” an online ...
Tag-Based Collaborative Filtering Recommendation in Personal Learning Environments
The personal learning environment (PLE) concept offers a learner-centric view of learning and suggests a shift from knowledge-push to knowledge-pull approach to learning. One concern with a PLE-driven knowledge-pull approach to learning, however, is ...
Jointly modeling content, social network and ratings for explainable and cold-start recommendation
Model-based approach to collaborative filtering (CF), such as latent factor models, has improved both accuracy and efficiency of predictions on large and sparse dataset. However, most existing methods still face two major problems: (1) the ...
Comments
Information & Contributors
Information
Published In
June 2010
328 pages
ISBN:9781605588315
DOI:10.1145/1809777
- Conference Chairs:
- Petri Vuorimaa,
- Pertti Naranen,
- General Chair:
- Artur Lugmayr,
- Program Chairs:
- Célia Quico,
- Gunnar Harboe
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 June 2010
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
EuroITV '10
Acceptance Rates
Overall Acceptance Rate 52 of 149 submissions, 35%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 178Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in