Practical diversified recommendations on youtube with determinantal point processes

M Wilhelm, A Ramanathan, A Bonomo, S Jain… - Proceedings of the 27th …, 2018 - dl.acm.org
M Wilhelm, A Ramanathan, A Bonomo, S Jain, EH Chi, J Gillenwater
Proceedings of the 27th ACM International Conference on Information and …, 2018dl.acm.org
Many recommendation systems produce result sets with large numbers of highly similar
items. Diversifying these results is often accomplished with heuristics, which are
impoverished models of users' desire for diversity. However, integrating more complex
statistical models of diversity into large-scale, mature systems is challenging. Without a good
match between the model's definition of diversity and users' perception of diversity, the
model can easily degrade users' perception of the recommendations. In this work we …
Many recommendation systems produce result sets with large numbers of highly similar items. Diversifying these results is often accomplished with heuristics, which are impoverished models of users' desire for diversity. However, integrating more complex statistical models of diversity into large-scale, mature systems is challenging. Without a good match between the model's definition of diversity and users' perception of diversity, the model can easily degrade users' perception of the recommendations. In this work we present a statistical model of diversity based on determinantal point processes (DPPs). We train this model from examples of user preferences with a simple procedure that can be integrated into large and complex production systems relatively easily. We use an approximate inference algorithm to serve the model at scale, and empirical results on live YouTube homepage traffic show that this model, coupled with a re-ranking algorithm, yields substantial short- and long-term increases in user engagement.
ACM Digital Library