Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

Exploiting temporal context has been proved to be an effective approach to improve recommendation performance, as shown, e.g. in the Netflix Prize competition. Time-aware recommender systems (TARS) are indeed receiving increasing attention. A wide range of approaches dealing with the time dimension in user modeling and recommendation strategies have been proposed. In the literature, however, reported results and conclusions about how to incorporate and exploit time information within the recommendation processes seem to be contradictory in some cases. Aiming to clarify and address existing discrepancies, in this paper we present a comprehensive survey and analysis of the state of the art on TARS. The analysis show that meaningful divergences appear in the evaluation protocols used—metrics and methodologies. We identify a number of key conditions on offline evaluation of TARS, and based on these conditions, we provide a comprehensive classification of evaluation protocols for TARS. Moreover, we propose a methodological description framework aimed to make the evaluation process fair and reproducible. We also present an empirical study on the impact of different evaluation protocols on measuring relative performances of well-known TARS. The results obtained show that different uses of the above evaluation conditions yield to remarkably distinct performance and relative ranking values of the recommendation approaches. They reveal the need of clearly stating the evaluation conditions used to ensure comparability and reproducibility of reported results. From our analysis and experiments, we finally conclude with methodological issues a robust evaluation of TARS should take into consideration. Furthermore we provide a number of general guidelines to select proper conditions for evaluating particular TARS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. A public test dataset used for evaluating performance of participants in the Netflix Prize competition.

  2. In this case it may be more precise to talk about score prediction, but for the sake of simplicity we will refer to this also as rating prediction.

  3. http://www.merriam-webster.com/dictionary/time

  4. There is no general definition of what good recommendations are. Nonetheless, a commonly used approach is to establish the quality (goodness) of recommendations by computing different metrics that assess various desired characteristics of a RS output.

  5. We abuse of the notation by denoting as \(t(e)\) the function returning the time of event \(e\), and denoting as \(t\) a particular value of \(T\).

  6. For the sake of simplicity we consider latent factors vectors of the same size, but they can be of different size.

  7. These metrics are also referred to as usage prediction accuracy metrics (Gunawardana and Shani 2009) and classification accuracy metrics (Herlocker et al. 2004).

  8. In a strict sense, a user-centered split ensures that most users will have training and test data, but there may be some users without enough ratings for both training and test sets. This will depend not only on the number of ratings of each user, but also on the definition of other evaluation conditions like the size condition.

  9. Note that each user’s rating sequence \(Seq_u^{ti}\) is generated independently in case of using a user-centered base set condition.

  10. In case of ties, they could be broken by sorting the tied ratings by user_id. If still there are tied ratings, then they could be sorted by item_id (note that in real datasets it is possible to find users with several ratings with the same timestamp due to, e.g. inconsistencies in the log subsystem).

  11. In case of a non-integer size value, it could be rounded to the nearest integer value.

  12. In this work we use a pure collaborative filtering approach for the music recommendation domain. Content-based approaches—exploiting special characteristics of music, such as chord, melody, lyrics, and composer—could be used instead, but they fall out of the scope of this study.

  13. MovieLens movie recommendations, http://movielens.umn.edu

  14. Neflix on-demand video streaming, http://www.netflix.com

  15. Last.fm Internet radio, http://www.lastfm.es

  16. These differences are statistically significant (Wilcoxon p \(<\) 0.05).

References

  • Adomavicius, G., Sankaranarayanan, R., Sen, S., Tuzhilin, A.: Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Inf. Syst. 23(1), 103–145 (2005)

    Article  Google Scholar 

  • Adomavicius, G., Tuzhilin, A.: Multidimensional recommender systems: a data warehousing approach. Second International Workshop on Electronic Commerce, pp. 180–192, Heidelberg (2001).

  • Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)

    Article  Google Scholar 

  • Adomavicius, G., Tuzhilin, A.: Context-aware recommender systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.), Recommender Systems Handbook. ISBN 978-0-387-85820-3 (2011)

  • Ardissono, L., Gena, C., Torasso, P., Bellifemine, F., Difino, A., Negro, B.: User modeling and recommendation techniques for personalized electronic program guides. Personalized Digital Television. Springer. ISBN 978-1-4020-2164-0 (2004).

  • Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  • Baeza-Yates, R.A., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York. ISBN 020139829X (1999).

  • Baltrunas, L.: Context-aware collaborative filtering recommender systems. PhD Dissertation, Free University of Bozen-Bolzano (2011)

  • Baltrunas, L., Amatriain, X.: Towards time-dependant recommendation based on implicit feedback. RecSys 2009 Workshop on Context-Aware Recommender Systems. New York, NY (2009).

  • Baltrunas, L., Ricci, F.: Experimental evaluation of context-dependent collaborative filtering using item splitting. User Model. User-Adapt. Interact., Special issue on Context-Aware Recommender Systems (2013)

  • Bell, R.M., Koren, Y.: Scalable collaborative filtering with fointly derived neighborhood interpolation weights. Seventh IEEE International Conference on Data Mining, pp. 43–52, Omaha (2007)

  • Bell, R.M., Koren, Y., Volinsky, C.: The Bellkor 2008 solution to the Netflix Prize. Netflix prize report (2008)

  • Bellogín, A., Castells, P., Cantador, I.: Precision-based evaluation of recommender systems: an algorithmic comparison. Fifth ACM Conference on Recommender Systems, pp. 333–336, Chicago (2011)

  • Bennet, J., Lanning, S.: The Netflix Prize. KDD Cup and Workshop, San Jose (2007)

    Google Scholar 

  • Brenner, A., Pradel, B., Usunier, N., Gallinari, P.: Predicting most rated items in weekly recommendation with temporal regression. Workshop on Context-Aware Movie Recommendation, pp. 24–27, Barcelona (2010)

  • Burke, R.: Hybrid recommender systems: survey and experiments. User Model. User-Adapt. Interact. 12(4), 331–370 (2002)

    Article  MATH  Google Scholar 

  • Burke, R.: Hybrid web recommender systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web, pp. 377–408. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Campos, P.G., Bellogín, A., Díez, F., Chavarriaga, J.E.: Simple time-biased KNN-based recommendations. Workshop on Context-Aware Movie Recommendation, pp. 20–23, Barcelona (2010)

  • Campos, P.G., Díez, F., Bellogín, A.: Temporal rating habits: a valuable tool for rater differentiation. Second Workshop on Context-Aware Movie Recommendation, pp. 29–35, Chicago (2011a)

  • Campos, P.G., Díez, F., Sánchez-Montañés, M.: Towards a more realistic evaluation: testing the ability to predict future tastes of matrix factorization-based recommenders. Fifth ACM Conference on Recommender Systems, pp. 309–312, Chicago (2011b)

  • Cao, H., Chen, E., Yang, J., Xiong, H.: Enhancing recommender systems under volatile user interest drifts. Eighteenth ACM conference on Information and knowledge management, pp. 1257–1266, Hong Kong (2009)

  • Celma, O.: Music recommendation and discovery in the long tail. PhD Dissertation, Universitat Pompeu Fabra, Spain (2010)

  • Chen, P., Tsai, C., Chen, Y., Chou, K., Li, C., Tsai, C., Wu, K., Chou, Y., Li, C., Lin, W., Yu, S., Chiu, R., Lin, C., Wang, C., Wang, P., Su, W., Wu, C., Kuo, T., McKenzie, T.G., Chang, Y., Ferng, C., Ni, C., Lin, H., Lin, C., Lin, S.: A linear ensemble of individual and blended models for music rating prediction. KDD-Cup Report, San Diego (2011a)

    Google Scholar 

  • Chen, T., Zheng, Z., Lu, Q., Jiang, X., Chen, Y., Zhang, W., Chen, K., Yu, Y.: Informative ensemble of multi-resolution dynamic factorization models’. KDD-Cup Report, San Diego (2011b)

    Google Scholar 

  • Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J.: Is seeing believing? How recommender interfaces affect users’ opinions’. SIGCHI Conference on Human Factors in Computing Systems, pp. 585–592, Fort Lauderdale (2003)

  • Cremonesi, P., Garzotto, F., Negro, S., Papadopoulos, A.V., Turrin, R.: Looking for “Good” recommendations: a comparative evaluation of recommender systems’. Thirteenth IFIP TC 13 International conference on Human-computer interaction, pp. 152–168, Lisbon (2011)

  • Cremonesi, P., Koren, Y., Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. Fourth ACM conference on Recommender systems, pp. 39–46, Barcelona (2010)

  • Cremonesi, P., Turrin, R.: Analysis of cold-start recommendations in IPTV systems. Third ACM Conference on Recommender Systems, pp. 233–236, New York (2009)

  • Cremonesi, P., Turrin, R.: Time-evolution of IPTV recommender systems. Eighth International Interactive Conference on Interactive TV & Video, pp. 105–114, Tampere (2010)

  • Dey, A.K.: Understanding and using context. Pers. Ubiquitous Comput. 5(1), 4–7 (2001)

    Article  Google Scholar 

  • Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (1998)

    Article  Google Scholar 

  • Ding, Y., Li, X.: Time weight collaborative filtering. Fourteenth ACM international Conference on Information and Knowledge Management, pp. 485–492, Bremen (2005)

  • Ding, Y., Li, X., Orlowska, M.E.: Recency-based collaborative filtering. Seventeenth Australasian Database Conference, pp. 99–107, Hobart, Tasmania (2006)

  • Dourish, P.: What we talk about when we talk about context. Pers. Ubiquitous Comput. 8(1), 19–30 (2004)

    Article  Google Scholar 

  • Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley. ISBN 0471056693 (2000)

  • Gantner, Z., Rendle, S., Schmidt-Thieme, L.: Factorization models for context-/time-aware movie recommendations. Workshop on Context-Aware Movie Recommendation, pp. 14–19, Barcelona (2010)

  • Gordea, S., Zanker, M.: Time filtering for better recommendations with small and sparse rating matrices. Eighth International Conference on Web Information Systems Engineering, pp. 171–183, Nancy (2007)

  • Gorgoglione, M., Panniello, U.: Including context in a transactional recommender system using a pre-filtering approach: two real e-commerce applications. 2009 International Conference on Advanced Information Networking and Applications Workshops, pp. 667–672, Bradford (2009)

  • Gorgoglione, M., Paniello, U., Tuzhilin, A.: The effect of context-aware recommendations on customer purchasing behavior and trust’. Fifth ACM Conference on Recommender Systems, pp 85–82, Chicago (2011)

  • Gunawardana, A., Shani, G.: A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 2935–2962 (2009)

    MATH  MathSciNet  Google Scholar 

  • Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative Filtering. Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 230–237, Berkeley (1999)

  • Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5–53 (2004)

    Article  Google Scholar 

  • Hermann, C.: Time-based recommendations for lecture materials. 2010 World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1028–1033, Toronto (2010)

  • Hussein, T., Linder, T., Werner, G., Ziegler, J.: Hybreed: A software framework for developing context-aware hybrid recommender systems. User Model. User-Adapt. Interact, Special Issue on Context-aware Recommender Systems (2014)

  • Iofciu, T., Demartini, G.: Time based tag recommendation using direct and extended users sets’. ECML PKDD discovery Challenge 2009, pp. 99–107, Paris (2009)

  • Jahrer, M., Töscher, A.: Collaborative filtering ensemble. KDD-Cup Report, San Diego (2011)

    Google Scholar 

  • Jahrer, M., Töscher, A., Legenstein, R.: Combining predictions for accurate recommender systems. Sixteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 693–702, Washington (2010)

  • Karatzoglou, A.: Collaborative temporal order modeling. Fifth ACM Conference on Recommender Systems, pp. 313–316, Chicago (2011)

  • Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: N-dimensional tensor factorization for context-aware collaborative filtering. Fourth ACM Conference on Recommender Systems, pp. 79–86, Barcelona (2010)

  • Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adapt. Interact. 22, 441–504 (2012)

    Article  Google Scholar 

  • Koenigstein, N., Dror, G., Koren, Y.: Yahoo! Music recommendations: modeling music ratings with temporal dynamics and item taxonomy’. Fifth ACM Conference on Recommender Systems, pp. 165–172, Chicago (2011)

  • Kohavi, R., Longbotham, R., Sommerfield, D., Henne, R.: Controlled experiments on the web: survey and practical guide. Data Mining Knowl. Discov. 18(1), 140–181 (2009)

    Article  MathSciNet  Google Scholar 

  • Konstan, J.A., Riedl, J.: Recommender systems: from algorithms to user experience. User Model. User-Adapt. Interact. 22, 101–123 (2012)

    Article  Google Scholar 

  • Koren, Y.: The BellKor solution to the Netflix grand prize’. Netflix Prize, Report (2009a)

  • Koren, Y.: Collaborative filtering with temporal dynamics. Fifteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 447–456, Paris (2009b)

  • Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl 21(4), 1253–1278 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  • Lathia, N., Hailes, S., Capra, L.: Temporal collaborative filtering with adaptive neighbourhoods. Thirty-Second International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 796–797, Boston (2009)

  • Lathia, N., Hailes, S., Capra, L., Amatriain, X.: Temporal diversity in recommender systems. Thirty-Third International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 210–217, Geneva (2010)

  • Lee, D., Park, S., Kahng, M., Lee, S., Lee, S.: Exploiting contextual information from event logs for personalized recommendation’. In Lee, R. (ed.), Computer and Information Science. ISBN 978-3-642-15404-1 (2010)

  • Lee, T.Q., Park, Y., Park, Y.: A time-based approach to effective recommender systems using implicit feedback. Expert Syst. Appl. 34(4), 3055–3062 (2008)

    Article  Google Scholar 

  • Lee, T.Q., Park, Y., Park, Y.: An empirical study on effectiveness of temporal information as implicit ratings. Expert Syst. Appl. 36(2), 1315–1321 (2009)

    Article  Google Scholar 

  • Li, R., Li, B., Jin, C., Xue, X., Zhu, X.: Tracking user-preference varying speed in collaborative filtering. Twenty-Fifth AAAI Conference on Artificial Intelligence, San Francisco (2011)

    Google Scholar 

  • Ling, C., Huang, J., Zhang, H.: AUC: A better measure than accuracy in comparing learning algorithms. In: Xiang, Y., Chaib-draa B. (eds.). Advances in Artificial Intelligence 2671, pp. 329–341, LNCS (2003)

  • Lipczak, M., Hu, Y., Kollet, Y., Milios, E.: Tag sources for recommendation in collaborative tagging systems. ECML PKDD Discovery Challenge 2009, pp. 157–172, Paris (2009)

  • Liu, N.N., Cao, B., Zhao, M., Yang, Q.: Adapting neighborhood and matrix factorization models for context aware recommendation. Workshop on Context-Aware Movie Recommendation, pp. 7–13, Barcelona (2010a)

  • Liu, N.N., Zhao, M., Xiang, E., Yang, Q.: Online evolutionary collaborative filtering. Fourth ACM Conference on Recommender Systems, pp. 95–102, Barcelona (2010b)

  • Lu, Z., Agarwal, D., Dhillon, I.S.: A spatio-temporal approach to collaborative filtering. Third ACM Conference on Recommender Systems, pp. 13–20, New York (2009)

  • Ma, S., Li, X., Ding, Y., Orlowska, M.E.: A recommender system with interest-drifting. Eighth International Conference on Web Information Systems Engineering, pp. 633–642, Nancy (2007)

  • Min, S., Han, I.: Detection of the customer time-variant pattern for improving recommender systems. Expert Syst. Appl. 28(2), 189–199 (2005)

    Article  Google Scholar 

  • Montañés, E., Quevedo, J.R., Díaz, I., Ranilla, J.: Collaborative tag recommendation system based on logistic regression. ECML PKDD Discovery Challenge 2009, pp. 173–188, Paris (2009)

  • Oku, K., Nakajima, S., Miyazaki, J., Uemura, S.: Context-aware SVM for context-dependent information recommendation. Seventh International Conference on Mobile Data Management, pp. 109–109, Nara (2006)

  • Palmisano, C., Tuzhilin, A., Gorgoglione, M.: Using context to improve predictive modeling of customers in personalization applications. IEEE Trans. Knowl. Data Eng. 20(11), 1535–1549 (2008)

    Article  Google Scholar 

  • Panniello, U., Gorgoglione, M., Palmisano, C.: Comparing pre-filtering and post-filtering approach in a collaborative contextual recommender system: an application to e-commerce. Tenth International Conference on E-Commerce and Web Technologies, pp. 348–359, Linz (2009a)

  • Panniello, U., Tuzhilin, A., Gorgoglione, M., Palmisano, C., Pedone, A.: Experimental comparison of pre- vs. post-filtering approaches in context-aware recommender systems. Third ACM Conference on Recommender Systems, pp. 265–268, New York (2009b)

  • Panniello, U., Tuzhilin, A., Gorgoglione, M.: Comparing context-aware recommender systems in terms of accuracy and diversity: which contextual modeling, pre-filtering and post-filtering methods perform the best. User Model. User-Adapt. Interact, Special Issue on Context-Aware Recommender Systems (2013)

  • Park, M.-H., Hong, J.-H., Cho, S.-B.: Location-based recommendation system using Bayesian user’s preference model in mobile devices. In: Indulska, J., Yang, L., Ungerer, L., Cao, J. (eds.) Proceedings of Ubiquitous Intelligence and Computing, 4611, pp. 1130–1139. Springer, Berlin (2007)

  • Parra, D., Amatriain, X.: Walk the talk: analyzing the relation between implicit and explicit feedback for preference elicitation. Nineteenth International Conference on User Modeling, Adaption, and Personalization, pp. 255–268, Girona (2011)

  • Pradel, B., Sean, S., Delporte, J., Guérif, S., Rouveirol, C., Usunier, N., Fogelman-Soulié, F., Dufau-Joel, F.: A case study in a recommender system based on purchase data. Seventeenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377–385, San Diego (2011)

  • Rendle, S.: Time-variant factorization models. In Rendle, S. (ed.), Context-aware ranking with factorization models. ISBN 978-3-642-16897-0 (2011)

  • Rendle, S., Gantner, Z., Freudenthaler, C., Schmidt-Thieme, L.: Fast context-aware recommendations with factorization machines. Thirty-Fourth International ACM SIGIR Conference on Research and Development in Information, pp. 635–644, Beijing (2011)

  • Sarwar, B., Karypis, G., Konstan, J., Riedl., J.: Analysis of recommendation algorithms for e-commerce. Second ACM Conference on Electronic Commerce, pp. 158–167, Minneapolis (2000)

  • Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.), Recommender Systems Handbook. ISBN 978-0-387-85820-3 (2011).

  • Stormer, H.: Improving e-commerce recommender systems by the identification of seasonal products. Twenty-second Conference on Artificial Intelligence, pp. 92–99, Vancouver, British Columbia (2007).

  • Töscher, A., Jahrer, M.: The bigchaos solution to the Netflix prize 2008. Netflix Prize, Report (2008)

  • Töscher, A., Jahrer, M., Legenstein, R.: Improved neighborhood-based algorithms for large-scale recommender systems. Second KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, pp. 4:1–4:6, Las Vegas (2008)

  • Tang, T.Y., Winoto, P., Chan, K.C.C.: On the temporal analysis for improved hybrid recommendations. 2003 IEEE/WIC International Conference on Web Intelligence, pp. 214–220, Beijing (2003).

  • Vargas, S., Castells, P.: Rank and relevance in novelty and diversity metrics for recommender systems. Fifth ACM Conference on Recommender Systems, pp. 109–116, Chicago (2011).

  • Vildjiounaite, E., Kyllönen, V., Hannula, T., Alahuhta, P.: Unobtrusive dynamic modelling of TV program preferences in a household. In: Tscheligi, M., Obrist, M., Lugmayr, A. (eds.), Changing Television Environments. ISBN 978-3-540-69477-9 (2008)

  • Weng, S.-S., Binshan, L., Chen, W.-T.: Using contextual information and multidimensional approach for recommendation. Expert Syst. Appl. 36(2009), 1268–1279 (2009)

    Article  Google Scholar 

  • Whitrow, G. J.: Time in history: views of time from prehistory to the present day. Oxford University Press, Oxford. ISBN 0-19-285211-6 (1988)

  • Wu, Y., Yan, Q., Bickson, D., Low, Y., Yang, Q.: Efficient multicore collaborative filtering. KDD-Cup Report, San Diego (2011)

    Google Scholar 

  • Xiang, L., Yang, Q.: Time-dependent models in collaborative filtering based recommender system. 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pp. 450–457, Milano (2009)

  • Xiang, L., Yuan, Q., Zhao, S., Chen, L., Zhang, X., Yang, Q., Sun, J.: Temporal recommendation on graphs via long- and short-term preference fusion. Sixteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 723–732, Washington (2010)

  • Xiong, L., Chen, X., Huang, T., Schneider, J., Carbonell, J.G.: Temporal collaborative filtering with Bayesian probabilistic tensor factorization. SIAM Data Mining 2010, 211–222 (2010)

    Google Scholar 

  • Zhan, S., Gao, F., Xing, C., Zhou, L.: Addressing concept drift problem in collaborative filtering systems. ECAI 2006 Workshop on Recommender Systems, pp. 34–39, Riva del Garda (2006)

  • Zhang, M., Hurley, N.: Novel item recommendation by user profile partitioning. 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pp. 508–515, Milano (2009)

  • Zheng, N., Li, Q.: A recommender system based on tag and time information for social tagging systems. Expert Syst. Appl. 38(4), 4575–4587 (2011)

    Article  Google Scholar 

  • Zhou, T., Kuscsik, Z., Liu, J.-G., Medo, M., Wakeling, J.R., Zhang, Y.-C.: Solving the apparent diversity-accuracy dilemma of recommender systems. Natl. Acad. Sci. 107(10), 4511–4515 (2010)

    Article  Google Scholar 

  • Ziegler, C., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. Fourteenth International Conference on World Wide Web, pp. 22–32, Chiba (2005)

  • Zimdars, A., Chickering, D.M., Meek, C.: Using temporal data for making recommendations. Seventeenth Conference in Uncertainty in Artificial Intelligence, pp. 580–588, Seattle (2001)

Download references

Acknowledgments

This work was supported by the Spanish Government (TIN2011-28538-C02), and by Comunidad Autónoma de Madrid and Universidad Autónoma de Madrid (CCG10-UAM/TIC-5877). The authors thank Centro de Computación Científica at UAM for its technical support. The authors also thank the anonymous reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Information Systems, Universidad del Bío-Bío, Concepción, Chile

    Pedro G. Campos

  2. Department of Computer Science, Universidad Autónoma de Madrid, Madrid, Spain

    Pedro G. Campos, Fernando Díez & Iván Cantador

Authors
  1. Pedro G. Campos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Díez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iván Cantador
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro G. Campos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Campos, P.G., Díez, F. & Cantador, I. Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols. User Model User-Adap Inter 24, 67–119 (2014). https://doi.org/10.1007/s11257-012-9136-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-012-9136-x

Keywords

Search

Navigation