Building Human Values into Recommender Systems: An Interdisciplinary Synthesis

Generally, such an effort begins with the study of potential mechanisms to incorporate this value. In the case of diversity, designers may research the history of attempts to create inter-group tolerance through diversity of exposure, including the ways in which this has failed [258, 263]. User studies may be used to test the exposure diversity hypothesis with users of the platform, assessing the impact of seeing news items from diverse perspectives on people's knowledge and attitudes. In particular, it will be important to test for backfire effects where people reject diverse information [18, 95] or other unintended consequences. There must also be a reasonable expectation that the changes to the recommendation system won't adversely affect other values that the system embodies. In concrete terms, this often means that any proposed change should not decrease other metrics (see below) more than a specified amount.

As discussed in Section 4, metrics are central to any attempt to build values into recommender systems. A variety of diversity metrics have been proposed, both for news content [75, 357] and for recommender systems more generally [60, 194]. Choosing or developing a metric requires settling on a definition of diversity suitable for that particular system. It may be that multiple metrics are needed to capture different aspects of diversity. While product teams typically choose metrics, there has been experimentation involving external stakeholders in order to increase the legitimacy of such decisions [33, 201, 320].

Here we focus on a conception of “productive” diversity, where people disagree in ways that are ultimately constructive [321]. Given this concept, developing a method to measure the diversity of articles on the platform may be broken into several phases, such as

Before implementing a specific product change, different product-based approaches to increasing diversity will be explored. For example, the designer could change the user interface, perhaps by showing related articles from other viewpoints below each item [325], or change the ranking algorithm to try to nudge people to consume more articles that represent this type of diversity [216]. For the sake of example, we assume below that the latter change has been selected.

Implementing this specific product refinement requires incorporating the diversity prediction model into the item ranking procedure. Whereas many current recommender systems score each item independently, diversity is a property of sets of items. One challenge is that scoring entire lists is both more complicated and more expensive than scoring individual items [160, 369] which may necessitate the development of more efficient algorithms for ranking such item sets.

Once implemented, the new product feature will typically be tested using offline data [55, 128] followed by A/B tests with a small group of users. If these tests show positive results, the new diversity prediction model will be deployed in production, and monitored to see whether the target diversity metric improves (this may involve gradual ramping up of the deployment to larger numbers of users, holdback tests, etc.). There may also be side effects—for example, while increasing diversity often increases engagement [194, 369], this is not always the case. Suppose that as a by-product of the model deployment, there is a drop in engagement, declines in other values-relevant outcomes such as quality or safety metrics, or some cost imposed on other stakeholders (e.g., content producers). In practice, this requires negotiation among internal stakeholders to decide if the increase in diversity is significant enough to justify a drop in other metrics.

Once deployed, product teams often establish a review process so that subsequent product changes, within the originating team or elsewhere, don't indirectly revert the diversity improvement. This might include numeric “guardrails” that specify the maximum allowable decrease in diversity induced by other product or algorithmic changes.

In practice, product teams will continually monitor the diversity of recommended items to detect operational failures (e.g., bugs or system failures), or (induced or exogenous) changes in the user distribution and user behavior. A survey which asks users or human raters to detect “productive” diversity may also be regularly employed to detect model drift and produce updated training data. Finally, determining whether the ultimate goal is being achieved, or is still worthy of being achieved, requires ongoing evaluation. This could involve survey methods to assess metrics such as affective polarization, and ethnographic research to understand what it means for users to be encouraged to be “more tolerant” in this way.

The properties of an item being considered for recommendation—whether a social media post, news article, or musical track—play a key role in determining whether it is of interest to the user, and, therefore, whether showing it to a user can serve one of their values.

The “topic” of an item is an important signal. A variety of techniques have been used for text analysis in recommender systems, including latent semantic indexing in Google News [79], Latent Dirichlet allocation [173], and transformers. Image and video analysis are also used for topic assessment [82], as is audio analysis [381]. Modern recommender topic taxonomies can encompass tens of thousands, or even millions, of distinct categories and sub-categories. One of the challenges is that these classifiers are typically accurate for the popular elements of the taxonomy of topics, but often do not perform well for posts on less popular topics.

Topics do not directly correspond to specific user needs or to values. A post about “football” can be about organizing a football viewing party (possibly contributing to the value of connection) or just reporting the latest scores or player injuries (the value of knowledge). To support the value of empathy and care it would be useful to identify posts where the poster could benefit from support of their network (e.g., after losing a loved one, needing advice on a certain matter, or announcing an important event in their life). An open challenge is inferring the way a particular user will relate to an item, rather than analyzing properties of the item alone. There is nascent work on predicting the experience a user may have when viewing content, e.g., whether they are entertained, angry, or inspired [67, 163]. These sorts of inferences could also enable better targeted or more persuasive advertising, which raises policy concerns [304].

Content analysis is also heavily used by social media platforms in order to determine whether a post violates community policies or is of low quality in some way, and should therefore be removed or demoted. The predominant method for determining such violations uses ML models trained on content labels provided by paid raters. These models increasingly use multimodal techniques, simultaneously considering text, speech/audio, images, and video. This is often crucial to understanding the intent of a post, e.g., the caption “love the way you smell today” means something different when superimposed over the image of a rose vs. a skunk [184].

Increasingly, items are evaluated not just in terms of their content but their context, including properties of the poster and audience, previous user behavior, reactions from other users, and so on [142]. Incorporating embeddings of sequences of user interactions has been shown to increase accuracy in misinformation and hate speech classification on Facebook [246] while “toxic” conversations on Twitter are better predicted by including network structure features [290].

These models are generally trained on labels or annotations generated by human raters. This process itself is potentially subject to error and particular forms of bias, both in the instructions given to evaluators and their assessment biases [287]. Also, many of these categories are both complex and politically contested [46, 214] and raters often disagree, which complicates the evaluation of model accuracy [134]. The promotion or demotion of particular topics or types of language, and the errors and biases in this process, have major implications for freedom of expression, which implies a deep connection between these technical methods and broader policy considerations [91, 181, 380].

We use “trajectory” to refer both to the sequence of items a user saw over time and the reactions those items evoked. A user's past trajectory often provides information that can be used to make better predictions about their future preferences and behaviors. At the same time, we want to ensure that the user's future trajectory supports the values they care about. Moreover, trajectories are the basic unit for studying some problems users experience on platforms, as they are central to discussions about potential long term effects on, say, well-being and polarization. For example, the “filter bubble” critique is essentially a statement about the typical course of user trajectories.

The majority of recommenders deployed in practice are “myopic” in the sense that they make predictions of a user's immediate response to the next slate of items presented, and rank items based on these predictions. However, many recommenders use past sequences of user and system behavior in sophisticated ways.

Advances in deep learning have made possible the practical deployment of sequence models, such as recurrent neural networks or transformers. For example, Beutel et al. [36] describe a recurrent (specifically, an LSTM) model deployed at YouTube. Recurrent approaches explicitly model a user's “latent” or “hidden” state, that is, they include variables which represent aspects of the user's situation or psychological state which are unobserved but have effects on what the user wants to see next. This state might encode aspects such as user satisfaction, frustration, or current topic focus, but interpretation of the hidden state embodied in such models is challenging and is tightly coupled to the engagement metrics being predicted and optimized. Inferring user state from behavior is an important challenge and a key step toward better supporting many values. For example, a recommender could detect a user's dissatisfaction with a certain type of content, or with too little topic diversity in the recommendation stream, or even that someone was developing an eating disorder [377].

Current ranking techniques do not offer the means to (directly) optimize a user's future trajectory. A promising direction is the use of reinforcement learning (RL) for optimizing such futures non-myopically [5, 297, 330]. In particular, the use of RL allows the system to consider the impact on the user of not just immediate recommendations, but of the entire sequence of recommendations it might make over an extended period, and plan that sequence accordingly.

In our news diversity example, the redesigned recommender considered diversity within each slate of recommended items independently. An RL-based recommender would be able to consider the diversity of items over days or weeks as well. In an educational setting—where a user's understanding of a topic might best be served by an individualized sequence of content— an RL recommender resulted in faster learning and more course completions than a baseline linear sequence [24]. As such, RL offers considerable promise as a technology for individual or user-level value alignment. To date it has been used largely to optimize user engagement over the long term, but with suitable metrics and objective criteria, it could play a vital role in better aligning recommendation trajectories with user well-being by adaptively planning recommendation sequences.

As with sequence models, advances in deep RL methods have made it more practical to deploy RL in practical recommender systems [66, 126, 160]. That said, a number of challenges remain. First, since RL relies on sequential interaction data with real users, such models are often trained offline using data generated by past recommendations, which may induce bias because the user was interacting with a different model [66]. The second challenge involves choosing from a large action space ranging from hundreds to millions depending on the recommendation task [94, 160]. A related problem is that item recommendations are often made jointly in slates or scrolling feeds, where the interactions or interference among the visible items makes interpreting and optimizing for user responses challenging [160]. Finally, adopting RL for true value-alignment requires sophisticated models of various aspects of user latent state (preferences, satisfaction, awareness/knowledge levels, fatigue/boredom, etc.) and their dynamics. These psychological and situational states are challenging to uncover from observable user-response behavior, and may require planning over extremely long “event horizons” as user adaptation to recommender changes may take 3–6 months to fully materialize [57, 229].

Some work has considered trajectories that might be unhealthy or harmful. One problem concerns minimizing the likelihood that trajectories will end up with users viewing large amounts of questionable content, such as videos that might promote unhealthy behavior, false statements about COVID vaccines [61], or other content that may not explicitly violate the platform's policies. This might occur not because users explicitly ask for such material, but because they end up in places (e.g., groups) where this material is common. In addition to downranking such content, another solution is to avoid recommending low-quality groups or content sources to users.

Another strand of work concerns items that are acceptable or appropriate when considered in isolation but could be harmful if consumed too much or by certain vulnerable people. For example, there may be nothing wrong with a diet video but perhaps someone with an eating disorder should not be presented with an endless stream of such videos; or it may be unhealthy to watch exclusively violent movies. Singh et al. [301] address this possibility by ranking user trajectories by proportion of “unhealthy” content, then using the mean proportion of this content in the top αth percentile of user trajectories as a regularization metric. They demonstrate a safe RL approach that improves both worst-case and average-case outcomes, in terms of the fraction of “unhealthy” items recommended to any one user.

The most complex concern about trajectories is the possibility that recommender systems might change user preferences in a manner that increases engagement but harms some other value. This is commonly associated with the idea of “manipulation,” meaning unwanted changes in attitude or behavior (even if unintentional).

This sort of optimization-driven shift has been frequently suggested as a mechanism driving filter bubble, echo chamber, or polarization effects, though the empirical evidence is mixed [50, 139, 249, 321, 392]. Such models posit a feedback loop where users choose particular items (as in selective exposure effects [265]) and the recommender responds to that engagement by narrowing its output to those topics, which in turn shifts user preferences further in that direction. This effect may be a particular concern for RL systems, which may learn how to make their users more predictable so as to maximize engagement [283]. A polarizing preference shift effect has been demonstrated in multiple simulations with different specifications [57, 105, 175, 177, 192, 284] which suggests that it could be a robust effect.

A range of work has attempted to determine the causal effect of recommender systems on content consumption trajectories. This is particularly important if those trajectories are correlated with offline outcomes such as violence [129]. There is replicated evidence that strongly moralizing expression spreads faster than other content on social networks [47] and such moralizing seems to precede offline violence [232]. A number of researchers interested in categories such as  “far right,” “conspiracy,” or “radical” have studied the network structure of recommendations between YouTube channels [110, 200, 275]. While this showed that more extreme channels often link to each other, these studies do not analyze user trajectories because they were conducted without any personalization. A different approach is to program bots to selectively engage with certain topics. This has generally shown that engaging with some type of content increases its frequency [360, 368] but this design models users as unchanging, so it does not provide evidence on persuasive effects. A user-tracking study of “far right” content on YouTube from 2016 to 2019 found that consumption there matched broader patterns across the web, including consumption from sources without recommender systems [156]. Separating consumption due to recommenders and consumption due to users’ seeking behavior is difficult without on-platform experiments. Using a set of Twitter users randomly assigned to receive a baseline chronological feed, Huszár et al. [159] found that Twitter's home timeline recommender reduced the consumption of more politically extreme material.

In general, inference of the causal effects of recommenders on user preferences and outcome is a major open problem. The feedback effects between algorithms and societies are at the cutting edge of social science research, in need of both interdisciplinary and cross-sector collaboration.

The field of explainable recommender systems has grown into a vibrant research area [386] in part because users respond positively to having good justifications for why certain items are suggested. Tintarev and Masthoff [337] argue that a good explanation can increase transparency, scrutability, trust, effectiveness, efficiency, persuasiveness, and satisfaction in the system. Each of these terms expresses somewhat different goals: transparency shows how the system works and can be instrumental in accountability; scrutability allows the users to tell the system it is wrong; trust increases confidence in the system; and satisfaction increases users’ sense of derived benefit; effectiveness helps the user employ the system toward their own aims; persuasiveness convinces the user to change their beliefs or behavior; efficiency can increase the value of the system. Some of these values can be contradictory and may not be achieved at the same time. For example, an explanation that increases transparency of the system does not necessarily increase trust if the explanation is not understandable or reveals undesirable behavior. Therefore, what constitutes a “good” explanation very much depends on the goal, and the field is rapidly developing [145, 241, 248].

With the rise of increasingly complex machine learning models, it has also become increasingly difficult to give intuitive and understandable explanations of why a user received a specific recommendation. Explanations may be shown to the user in different forms (e.g., text, visuals, highlighting relevant features) and may either attempt to explain the workings of the recommendation model itself, or may be the result of training a separate model that generates post-hoc explanations from model inputs and outputs [386]. There are also recommender algorithms specifically designed to be explainable [376].  The system of Balog et al. [19], for instance, operates based on set intersections, e.g., “recommended for you because you don't like science fiction movies except those about space exploration.” This explainable-by-design approach avoids the challenges of interpreting learned models [282], but generating understandable explanations from deep learning models is an active area of research that may yet prove fruitful [185, 189, 328].

People express their values in their everyday use of recommender systems. Indeed, many recommender controls are designed specifically for this purpose, everything from upvoting and swiping left/right to reporting violating content. This creates the problem of aggregating preferences, and negotiating between competing desires of individuals, groups, and stakeholders. The framework of social choice, originally developed in economics [125, 294], provides a foundational tool for addressing tradeoffs at all of these levels.

Abstractly, this framework assumes that each stakeholder has a utility function over a set of possible outcomes for them. This is motivated by the idea that someone could say which of several situations they'd prefer, that is, that each person has preferences. Note that this utility function can be purely “local” (a user may care only about whether they get good recommendations) or it can involve societal values (a user may care that other users also like what they like, or that vendors are treated fairly). A social welfare function is a voting process or a way of “adding up” or combining stakeholder preferences to produce a single number, a “societal utility” or social welfare. The aim is then to adopt a recommender policy which maximizes the expected social welfare.

This formulation can encompass virtually any criteria that express preferences over short or long-term, individual or group outcomes. There are relatively direct mathematical expressions for penalizing addictive behaviors, group-level diversity of consumed content, fairness across individuals, and so on. Conversely, any collaborative recommender system aggregates the explicit or implicit preferences of users in some mechanical way, as when signals like upvoting and watch time are combined across many users to decide how items are ranked for a different user. Social choice theory is a key bridge between the normative and the algorithmic, useful for analysis and design.

Although social choice theory is foundational, there are two major difficulties to applying it in practice. First, determining what an individual's “preferences” actually are is quite challenging both in theory and in practice [334]. There is a long history of formal elicitation methods such as asking users to repeatedly say which of two options they prefer, or asking them to play various kinds of economic games [53, 65], but formal preference elicitation can be involved and is quite demanding of the participants. Furthermore, someone might be uninformed, coerced, addicted, have lowered expectations [9] or want something that isn't available [119, 277]. In addition, attempts to elicit preferences can lead to strategic behavior where people misrepresent their beliefs to try to induce more favorable recommender outcomes, e.g., a content creator usually has an incentive to argue that their content is relevant to as many users as possible [29, 30]. While many different kinds of feedback can provide crucial signals of what people value, behavior cannot be naively interpreted as true preferences.

Second, there is no entirely bottom-up or value-neutral method of ethics. Simply specifying the outcomes over which stakeholder preferences are defined is inextricably tied to the values being considered [32]. For example, there is the choice of what will be voted upon. There is also the question of whose preferences count in what situations, e.g., community administrators may have special voting privileges, and it may be important to encode “rights” that cannot be infringed by the preferences of others. Hence, social choice approaches cannot excuse system designers from making consequential normative choices [27]. There is more to democracy than voting systems.

Many of the values in our list involve making tradeoffs between different stakeholders (such as users vs. content providers) or among members of a stakeholder group (such as different subgroups of content providers which compete for attention). Correspondingly, there are a wide variety of notions of “fairness,” which often (but not always) are framed as some sort of tradeoff. The extent to which these tradeoffs are inherent is an open question in the research literature because there are cases where it is possible to improve performance for one user subgroup without decreasing performance for other users. There are multiple challenges in this area, including defining fairness, measuring it in practice, and designing algorithms for efficient recommendation. See Ekstrand et al. [98] for an overview.

Several major categories of fairness have been proposed in the context of recommender systems, roughly corresponding to the interests of different stakeholders. “C fairness” considers how well individual information consumers are served, “P fairness” is concerned with the distribution of attention between items or providers of content, while “CP fairness” considers both simultaneously, as in a rental property recommender designed to protect the rights of both minority renters and minority landlords [52, 361].

Recommender systems are mostly evaluated based on average performance across all users, but different user subgroups, such as age or gender groups, might be served with differing performance or error rates. Subgroup performance disparities can happen for a variety of reasons, including differences in group size or activity that affect the amount of training data available [100, 204].  There is a large body of work on mitigating group-level unfairness in classifier models, some of which can be adapted to recommender systems. For example, [34, 35] use pairwise comparisons of the ranking of different items to generalize the well-known “equality of opportunity” and “equality of odds” measures, showing that it is possible to equalize prediction error rates between user groups on a large commercial platform. However, algorithmic approaches that aim at equalizing effectiveness disparities between user groups may make inappropriate tradeoffs: increasing recommendation utility for one user does not necessarily require decreasing it for other users, so it is not clear that allowing a solution that may decrease utility for well-served users is appropriate, as opposed to other approaches such as feature prioritization in the engineering process (Ekstrand et al., 2021 Section 5.4).

Items and their producers, on the other hand, necessarily compete for user attention. This leads to the concept of “exposure fairness” which may be formulated in a variety of ways [302]. Even if predicted “relevance” scores correctly measure the value of an item to an individual user, slightly less relevant items may get disproportionately less attention simply because they appear farther down, an effect known as “position bias.” Several algorithms have been proposed to ensure item attention is proportional to item value on average, either across a group of items or across multiple slates of recommendations [37, 85, 361]. Such algorithms can be considered a correction to a type of error or “technical bias” in the fair ranking typology of Zehlike et al. [383]. More normative definitions of item fairness are often desirable. For example, Spotify strives to give exposure to less popular artists to counteract the “superstar economics” of cultural production [222], while a “demographic parity” conception of fairness may be appropriate when qualified candidates from different groups (say, men and women) should be shown to prospective employers at the same rate [127]. A wide variety of fairness metrics concerning the exposure of items, groups of items, or producers have been proposed, though many of these are closely related [193, 270, 288, 384].

Where it is possible to produce reasonable quantitative estimates of utility to different stakeholders, multi-objective optimization (MOO) can be used to balance multiple conflicting stakeholder utility and fairness objectives. One approach is to ensure that the recommender is Pareto efficient, meaning that there should be no way to modify a slate of recommendations to make it more fair without reducing utility, or to increase utility without reducing fairness [374]. Mladenov et al. [228] go beyond this by proposing a recommendation method that maximizes user social welfare (total user utility) by allowing small sacrifices in utility from well-served users to drive large gains for less well-served users. RL has also been applied to multi-sided fairness, through contextual multi-armed bandits which simultaneously optimize stakeholder utility and fairness objectives [222, 223, 375]. Recently, several researchers have taken a game-theoretic approach to the study of recommender systems. Ben-Porat and Tennenholtz [30] and Ben-Porat et al. [29] develop approaches that account for the strategic behavior of content providers while aiming at maximizing user engagement. While all these methods hold promise for value alignment in complex ecosystems, they have not seen practical deployment to date.

Because there is no purely “bottom up” way of making tradeoffs, system designers must ultimately choose some set of overall objectives or “ground truth” signals to serve as overall measures of value. Increasingly, AI tools are used to help make tradeoffs over the complex design space of recommender parameters, particularly the relative “weighting” of the signals that feed into item ranking functions.

Milli et al. [227] determined the relative value of different user actions including viewing a Tweet, sharing it, liking it, and so on by connecting these interactions to the sparse use of the “see less often” control in a causal Bayesian network. This network represents dependencies such as the fact that a user has to view a Tweet before they can share it. By taking “see less often” as a ground truth signal of negative value it was possible to infer the value of all of the other, more common interactions. This idea generalizes to more complex methods for finding multiple weights that optimize multiple metrics.

Bayesian optimization [130, 274] can be used to find the weights of a ranking function that maximizes relevant (perhaps long-term) metrics, and automatically run data-gathering experiments to improve those predictions [140]. This approach requires that the designer be able to assess the overall “utility” of any vector of performance metrics, which itself induces various tradeoffs. For example, is a product that has a greater number of items viewed but less total time spent and lower reported satisfaction better than the opposite? The tools of interactive optimization and utility elicitation [42, 388] could play an important role here, though these approaches have not yet found widespread use in practical recommender design.