SustAInable: How Values in the Form of Individual Motivation Shape Algorithms’ Outcomes. An Example Promoting Ecological and Social Sustainability

Algorithmic systems are designed to generate output (e.g., a decision) on the basis of patterns in data from past instances (i.e., training data). These data, especially if they are social data, frequently contain biases and incorporate social stereotypes [55], which are then inherited and perpetuated by the model [20]. Mehrabi et al. [37] provided a comprehensive framework classifying various potential sources of bias in algorithmic output into three categories: data-to-algorithm bias, algorithm-to-user bias, and user-to-data bias. The first category, data-to-algorithm bias, highlights the limitations of models when they do not adequately represent the population (e.g., a lack of diversity in image data sets can cause facial recognition algorithms to perform differently on the basis of a person's skin type and gender [7]). The second category, algorithm-to-user bias, is especially prominent in recommendation and content management algorithms, as these influence the visibility and ranking of content. Users, in turn, are more likely to engage with content that is more visible or ranked higher, thus reinforcing its prominence. The third category is user-to-data bias. This category encompasses different biases, such as historical bias, where patterns inherent in human-generated data [8] are learned by the algorithm and eventually lead to discrimination (this bias is evident in scenarios such as occupational roles traditionally held by men or women, leading to biased representations in the training data). Another example of user-to-data bias is content production bias. It arises from variations in how different populations generate content, for instance, because they differ in their use of language. This bias can arise from any person who generates data—not only from users but also from curators who train the algorithm. Values influencing an algorithm via stakeholders’ motivations belong to the category of user-to-data bias. The training data would be skewed in such a way that it reflects a set of values that might not correspond to the overall population of users.

We draw from research from Psychology and Communication Science to understand the emergence of user-to-data bias. Like human decisions in general, decisions made during the training of algorithms (e.g., when curators rate the responses of a chatbot as appropriate or inappropriate, or when emails are classified as spam or not spam) may depend on the (value-relevant) involvement—that is, individuals’ motivation to behave in a manner that is aligned with their values regarding the issue [16, 19, 35]. This involvement influences how individuals process information and results in people being less easily persuaded to change their attitudes [19]. Research from environmental psychology has found that involvement (i.e., the motivation to protect the environment—with values being an integral part of it) is related to conservation behavior, e.g., [ 22, 16]. Applied to the context of algorithms, this association implies that, when training the algorithm, people with a high level of involvement (i.e., a high level of environmental motivation) are likely to make decisions that are consistent with their motivation to protect the environment. In terms of the case study of our spam-filter algorithm, this means that the extent to which people value environmental motivation may be related to the positivity of their perceptions of emails on this topic.

Researchers examined the consequences of this involvement with regard to sharing information in social networks [12, 51]. Similar to the training of various algorithms (e.g., spam filters, chatbots, or content moderation), people make decisions about the visibility of certain content for other people. For instance, by frequently sharing health-related news on social media, individuals influence their network's feed, determining which topics are more visible to their contacts. Scholz et al. [51] argued that such information sharing is guided by value-based decision-making. Thus, when deciding whether to share content, a person weighs the positive effects of sharing (e.g., providing access to relevant information) against possible costs (e.g., the risk that the message will not be well-received in one's social group) and chooses the option that offers the highest value for them. The likelihood of sharing increases the more a person perceives a message as relevant to themselves or to other people [12]. This inclination to share content that is relevant for oneself suggests that when training algorithms— especially in the aforementioned contexts—individuals’ decisions are similarly related to their perceptions of relevance and their personal involvement in the topic. As algorithm curators are usually anonymous, the costs (risks) are negligible compared with message sharing in social networks. Consequently, in the case of algorithms, the decision of what content should be visible likely depends even more on personal relevance. We expect that for people with high involvement (i.e., high environmental or altruistic motivation), the self-relevance of emails dealing with the respective issue will be higher.

Besides biases in the data used to train algorithms, human decisions made during the development process (e.g., when specifying model parameters) may still lead to biased algorithmic output [26, 27, 37, 42]. Thus, whenever the development process allows for degrees of freedom, it is likely that developers will base their programming decisions on their own needs and preferences, which form their motivation [45, 60], and they will thereby make value-consistent decisions. This self-referential approach, in which values (sometimes unconsciously) guide the decision-making process, extends to the training of algorithms, particularly in the case of machine learning models, which rely heavily on human curation. Therefore, we propose that developers’ and curators’ motivation can influence their decisions when developing and training an algorithm. Similarly, the decisions made by users interacting with these systems can potentially be shaped by their values. A relationship between values and decisions has already been empirically identified (e.g., with regard to social decisions [53]). We aim to apply these findings to the development and use of algorithms and to empirically show that stakeholders' values—through their motivations—affect their decisions and, in turn, may lead to biased algorithmic output.

Existing studies have predominantly focused on exploring unfair biases in algorithmic output [30], that is, biases that lead to systematic discrimination [15]. In this work, we aim to adopt a broader definition of bias, considering it in its original—morally neutral—sense as a simple “skewness” or “slant.” By using this broad definition, we could also focus on nondiscriminatory biases and thus discuss their potential opportunities.

In addition to research on the emergence of bias, another strand of research focuses on developing methods to reduce bias in algorithmic output. Such research includes approaches for raising stakeholders' awareness as well as the development of fairness measures and algorithms capable of mitigating bias, e.g., [3]. To reduce bias, researchers have emphasized the importance of caution and education on the emergence of bias in the development process, as well as a diverse workforce [3]. Continuous monitoring of algorithms even after the training period and algorithm audits are also recommended as tools for identifying biases [26, 32, 36]. Kleinberg et al. [27] further noted that with appropriate laws and regulatory systems in place, algorithms would potentially facilitate the detection and prevention of discriminating biases, as algorithmic decision rules—although often described as a black box—can in fact be made more transparent than human decision-making processes. From this angle, algorithms could even be regarded as a force for the common good by “nudging” user behavior in a less discriminatory (and thus more sustainable) direction [28]. Taking this idea further, not only the reduction of discrimination but also a bias toward the common good or ecological sustainability would lead to more sustainable algorithmic decisions, which might then nudge user behavior. The notion of incorporating values and ethical considerations into algorithms has also been discussed in research on machine ethics [58, 59], with representatives of this line of research emphasizing the need to incorporate values into algorithmic decisions in order to prevent unethical outcomes, although the success of this approach has been criticized [6]. Among those who advocate for incorporating values into algorithms, sustainability is a value that is increasingly recognized as a crucial aspect of AI design [58].

In software development in general, sustainability is also becoming increasingly relevant, as reflected in the development of guidelines to incorporate sustainability considerations into the development process [46]. However, besides guidelines, human factors—particularly with regard to stakeholders' values and the motivation related to these values—also play a role in software development. A diverse and educated workforce can help reduce discriminatory biases by identifying them and applying tools to reduce them [3]. Moreover, it is also likely that a more prosocial or environmentally motivated workforce would incorporate these values more strongly into an algorithm, as values may be reflected in the decisions people make during algorithm development and training. Lammert [31] postulated that the values of developers and other stakeholders of a software product are reflected in the product and thereby also influence its sustainability impact. However, this claim has yet to be empirically investigated.

The aim of this study was to explore whether stakeholders' prosocial and environmental motivation influences the preparation and selection of the training material that finally goes into the training of algorithms and in the end has an effect on algorithmic output and can thus create a bias toward ecological or social sustainability. We aimed to address the abovementioned research gaps by (a) examining the relationship between stakeholders’ motivations and algorithmic training material as the primary source of bias and (b) exploring how to potentially increase sustainability in algorithmic output. Specifically, we investigated the relationship between the motivation to benefit other people or, more generally, nature (i.e., environmental and altruistic motivation) and the decisions people make when training a spam-filter algorithm. We hypothesized:

H1: Stakeholders’ environmental motivation is negatively related to categorizing emails from environmental organizations as spam.

H2: Stakeholders’ altruistic motivation is negatively related to categorizing emails from humanitarian organizations as spam.

Previous research has shown that prosocial and environmental motivation are correlated [39]. Related to this finding, studies have shown that behaviors directed toward the well-being of other people and toward the environment are linked to the same personality traits [10, 44, 56]. Otto et al. [44] argued that there is an overarching factor that can be regarded as a prosocial propensity or an orientation toward the welfare of others and that it can be directed toward both human and nonhuman entities. This orientation can manifest in motivation for both pro-environmental and prosocial behaviors. Thus, due to the close connection between these two constructs, we expected a spillover effect in spam categorization. That is, a high degree of environmental motivation might lead not only to protecting the environment but also to helping people. In turn, this motivation influences the categorization of emails from social organizations (and vice versa). Therefore, we hypothesized:

H3a: Stakeholders’ altruistic motivation is also negatively related to categorizing emails from environmental organizations as spam (however, the effect is smaller than for humanitarian organizations).

H3b: Stakeholders’ environmental motivation is also negatively related to categorizing emails from humanitarian organizations as spam (however, the effect is smaller than for environmental organizations).

Data were collected between September 10 and 16, 2021, with the help of a research panel. Participants were incentivized by the market research company to participate in the study. All participants gave informed consent, and we adhered to the ethical guidelines of the American Psychological Association [1].

If not otherwise stated, all variables were assessed with 5-point scales, either ranging from 1 (does not apply at all) to 5 (fully applies) or with the following anchors for items assessing behaviors: 1 = never, 2 = rarely, 3 = occasionally, 4 = often, 5 = very often/always.

Data analyses were performed with RStudio [48]. For both environmental and altruistic motivation, we conducted a Rasch analysis [5, 49]. All items demonstrated an MS-infit below 1.3, indicating a satisfactory model fit [5]. For plausibility checks, we conducted a series of paired t tests to test whether (a) the real spam email was significantly more likely to be classified as spam than all other emails and (b) the email from the tax authority (not spam) was significantly less likely to be classified as spam than all other emails. The results confirmed that the genuine spam email received higher spam ratings, and the email from the tax authority received lower spam ratings than all other emails. Furthermore, we tested whether emails from humanitarian and environmental organizations differed in their spam ratings. Emails from humanitarian organizations were more frequently classified as spam (M = 3.78, SD = 1.19) than emails from environmental organizations (M = 3.31, SD = 1.32), t(660) = -9.70, p < .001, 95% CI[-0.50, -0.33].

In our study, we chose spam filters as a specific case study due to their relatively narrow but rather controlled environment. The behavioral shifts observed in the spam filter training task may significantly impact the flow of information when scaled to larger systems. In these broader contexts, slight biases, akin to those in this study, can impact the visibility and dissemination of content, thereby influencing public access to information. While the specific task of categorizing emails may seem limited in scope, it mirrors the decision-making processes found in many more complex algorithmic systems. In other, more complex contexts, the basic mechanism will be quite similar if not the same because even though an amalgamate of several motives (and values) can be at work in complex contexts, motives as well as values are usually compensatory, and thus, they simply add up independently.

Just like spam filters, such systems often involve sorting and classifying information. For instance, large language models learn through curators’ ratings of whether the content is appropriate or not. In the realm of social media, content moderation algorithms function similarly to spam filters to determine the appropriateness of content. Similarly, news feed algorithms curate a user's news stream on the basis of perceived interests. In recommendation systems—whether for e-commerce, streaming services, or online advertising—the sorting and recommending of items to users is another domain where the decision-making process can be subtly influenced by the values and biases of the stakeholders.

Whereas the systems described above are algorithms that are similar to spam filters with respect to their training (i.e., information is sorted by humans), the fundamental assumption that values can influence algorithmic output through motivation is applicable to a variety of contexts. Our research focused on content curators; however, similar effects could emerge among developers, users, and other stakeholders. Developers’ motivations could influence their decisions in the development process in a similar manner. On the other hand, user feedback further shapes algorithmic decision-making. Thus, the supposedly small influences of the values and motivations of certain groups could have a large impact all together.

Our findings contribute to the understanding and management of biases in algorithmic systems. Recognizing these underlying values can help developers and policymakers create algorithms that are sustainable and socially responsible. For instance, it may be instrumental to integrate sustainability-focused training and interventions into the educations or workplaces of those who develop or curate algorithms. Fostering a culture of environmental stewardship and social responsibility in tech companies might provide a basis for these motivations to be reflected in the decisions made during the development and training of algorithms and, thus, algorithmic output. Whereas on the one hand, our results suggest that the influence of individual values on algorithmic output could be utilized to promote positive outcomes, see also [61], our results also highlight that values may also have unintended consequences in the form of unintended biases that need to be mitigated. For instance, clients who want to apply an algorithm might not want sustainability motivation (or other motivations) to influence training decisions. The biased selection of training material, as observed in our study, echoes concerns similar to those found in research on social media, such as the selective sharing of posts. For instance, Shin and Thorson [52] showed that partisans tend to spread content that favors their perspective while criticizing the opposition—a bias that could similarly affect algorithms responsible for content selection and dissemination. Thus, raising awareness about such value- or motivation-based biases is crucial for developing strategies to reduce them. For instance, it may be necessary to discuss (a) whether and how the influence of stakeholders’ values and motivations could or should be taken into account in regulations and guidelines pertaining to algorithm development and (b) potential ways to reduce such influences. In their recent work, Hardy et al. [18] adapted a content selection algorithm to be used in a social network. The aim was for the algorithm to rearrange the content from individuals’ personal networks in such a way that it was representative of the perspectives of the population. This approach enables a more balanced flow of information and reduces echo chambers.

There are several limitations to consider when interpreting the results. First, the fact that we did not find an influence of altruistic motivation on the categorization of emails might be due to the measurement method and the material we used. The scale we used is widely applied to assess altruism; however, it might not have adequately captured the nuances of altruistic motivation that would affect spam ratings, or the ratings might be influenced by a negative image of humanitarian organizations. Future studies should assess this relationship using different measures and by using different and more diverse stimulus material. Second, we used a cross-sectional design; thus, the directions of the relationships can be derived only from theory. Third, our study focused on a specific use case, and the effect sizes were rather small. We chose spam filters as a specific case study, which, while narrow, provided a more controlled environment to observe the subtle influence of values on decision-making. Recognizing and understanding these effects lays the groundwork for further research that can investigate the implications of such value-based decisions in broader, real-world contexts where decisions are made at scale and where their impacts can be far-reaching.