Algorithmic Power or Punishment: Information Worker Perspectives on Passive Sensing Enabled AI Phenotyping of Performance and Wellbeing

One of the classical approaches to evaluate privacy for passive sensing is to evaluate it by proportionality to existing activities [75]. In information work, project management tools such as JIRA are already used to disclose an IW’s work activity to others on the project [107]. An IW might want to disclose their state of wellbeing to their manager to negotiate work breaks. In theory, this can be a compelling crutch to justify PSAI at work. Yet, it remains an open question if the algorithmic phenotyping of PSAI introduces uncertain imaginaries that cannot be reconciled by existing work practices. According to Nissenbaum’s framework of Contextual Integrity, user preferences for tracking systems are limited when privacy is considered intrinsic to the actors, spaces, or nature of information [102]. Instead, the adoption of systems must be studied by understanding the role of that information within the context of the user.

The contextual integrity framework is becoming increasingly significant to evaluate reasonable implementations of sensing technologies. Nicholas et al., have illuminated attitudes toward personal sensing in the health context [101]. Similarly, contextual integrity has been used to explain adoption of tracking systems for public health [135]. Closer to our scope, a recent study by Adler et al. described the norms of information flow for quantifying the stress of physicians in the workplace to respond to burnout [4]. Interestingly, in their context, workers felt that sharing information with a supervisor could be more valuable than self-reflection, as supervisors had actual power to make changes to assuage their stress. Studies indicate that workers are willing to adopt ambient technologies if they enhance their wellbeing [114] and location tracking when it improves work efficiency [1]. Can we transfer these expectations to the algorithmic inferences provided by PSAI? Information work provides a unique setting (Section 2.2). This motivates us to interpret our themes from an analytical lens that reconciles the expectations of emerging technologies in specific settings. We know that Contextual Integrity is upheld when the following information norms are maintained; (i) Norms of Appropriateness and (ii) Norms of Flow/Distribution [102]. As a result, we synthesize and scrutinize our findings based on these norms.

We describe our positionality as a way to situate our values that shaped this research. Three authors have conducted research in the past combining machine learning with passively collected data for digital phenotyping to support mental wellbeing. The authors have, however, no stake, financial, personal, professional, or other, in any of the technologies used to inspire scenarios (Table 2). However, their research advances the technologies like PSAI through novel methodologies as well as human–centered evaluations. Two authors also have experience working in typical information work organizations. In light of this, we consider ourselves “insiders” because this perspective critiques technology motivated by our own research and appraises our own sociotechnical reality as IWs. Our identities and experiences as researchers also help us construct meaning from our data and conceptualize our findings [15]. Broadly, this paper is influenced by our interactions with privacy researchers, organizational psychology researchers, IWs, and other data subjects of digital phenotyping. We borrow Chancellor et al.’s term to describe ourselves as “critical insiders” [22]. We are in a unique position to bridge disparate views and approaches on the future of work by pursuing a worker–centered approach.

PSAI systems provide indicators, which might be considered orthogonal to work-specific tasks (e.g., your performance was moderate or stress was high). On the one hand, participants found value in leveraging these insights to contextualize their experience with evidence and champion change. On the other hand, participants were anxious that these insights could be misappropriated to their own detriment.

The insights generated from PSAI can be empowering to IWs as it helps contextualize their work experience. P25 imagined such systems to support IW needs, “I think with the data, it would at least help you sit down at the table, so to speak.” Traditionally, workplace evaluations favor ends over means. As P2 puts it, “the work which is getting done is what is counted, but how we achieve it is never logged in anywhere.” P5, like P6 (quoted above), believed that PSAI insights could complement existing evaluations after seeing Sys 5. Reflecting on one of her past evaluations, P15 claimed that PSAI could have been useful as a reference (“let me give some numbers.”) It can give IWs a deeper understanding of their work patterns, present opportunities for learning effective work practices, and enable them to negotiate changes. P22 envisioned using Sys 7 to request time off for their wellbeing, “He can look into this report and it would be some kind of objective.” Alternatively, P17 believed this data would be more persuasive to reorganize his work expectations, “Having data that would support, I need a virtual assistant or we need to hire another PM or it’s not feasible for me to run this many projects and run this team at the same time.” P14 thought PSAI could support her in highlighting her role to others higher up in the organization. She said, “This would end up benefiting us more because it would help others see how much we actually do and change the current stigma.” Even at an aggregate level, it can help employers reflect on their organizational health. For instance, P25 found this a suitable approach for “the company to be aware of work–life balance.” Therefore, IWs find value in such systems when they can incorporate its insights into demonstrable change such as professional development or negotiation for wellbeing.

The existing power asymmetry of information work environments always engenders concerns about privacy and subsequent misappropriation of their passively sensed data. Antithetical to the empowering aspects PSAI, P3 was concerned that his employer could tap into these insights to stifle their career, even going on to call one PSAI system “destructive.” P1 had a more straightforward concern, “If my workday performance and how I work was released, it might affect how much I get paid.” Participants like P4 were anxious about the uncertain consequences of other stakeholders using this data. These concerns stemmed from the perceived lack of control over one’s data in the organizational context. “I download the app, the information is captured and then it goes to someone else. That’s the objection.” (P27). Furthermore, implementing PSAI like Sys 1 and Sys 4 — which are embedded in the physical infrastructure — can create an austere situation where IWs might feel that their choice to consent could affect their employment (P5, P11). In fact, some participants felt that the very decision to use such systems for deeper surveillance could reflect an organizations’ own values (P12). Eventually, such uncontrolled and imbalanced deployment of PSAI can detract IWs from choosing to work in such companies. However, even that choice is a function of job precarity in that sector. As a result, development of PSAI systems needs to be aware of the socio-economic conditions of employment.

Workers’ preferences for work–life boundary management reflected their perceived control of privacy in PSAI systems but also highlighted their expected value from the system. Post–COVID-19 pandemic, new emerging work practices are allowing many IWs to work remotely either in their entirety or on certain days of the week. Moreover, given the ubiquity of personal laptops and mobile phones, it is commonplace to bring some work home. Although we know non-work can influence work experiences, some workers found sensing beyond work invasive and irrelevant to improving work. Yet, some workers also believed that sensing non-work could be less consequential to their jobs and more holistic for reflection.

Different workers have different approaches to their work–life. Some demarcate the segmentation between the two using physical aspects. A common understanding is segmenting work–life based on the space the IW finds themselves in. In the quote above, P8 was willing to consent to PSAI if it is contained to their workspace. With more interleaving work–life practices, space is not the only indicator of work–life separation. Organizations often provide workers with work–specific devices or enforce a logical separation between work & personal profiles. For example, P5 thought Sys 5, which logs applications and browsing, was reasonable because she did not do personal activities on her work machine anyway. Although this might be to ensure security of organizational data, it also provides another method for IWs to segment work–life. P17 noted that he was willing to allow PSAI systems on his work device, “But if it’s a personal device and I’m doing work on, absolutely not.” P1 said Sys 2 was a “violation of personal space” because the webcam could capture their home environment. Understanding these constraints can help describe the limits within which privacy can be preserved. It is also worth noting the concern of some participants who believed that preserving the work–life boundary for PSAI made it more useful (P13, P16, P21). Similarly, on different occasions, both P9 and P10 stated the focus on the work context was more “accurate.”“If we can achieve only tracking the work applications that will definitely improve the efficiency and avoid a lot of other privacy arguments, if there’s any there,” said P8. Thus, for certain IWs, the work context is not only more private but can actually be more useful.

P7 presented an alternative viewpoint that shows work–only restriction of PSAI can elicit concerns about job consequences (Section 4.1.1). In fact, depending on what kind of data is being sensed, an IW might consider the privacy of their work activities to outweigh that of activities outside of it. Devices distinct from the work context can be considered more reasonable for sensing. P1 even described a greater willingness to accept a PSAI system provided by a third–party because of the apprehension that something provided by the organization can be misappropriated by HR. Again, the shifting of sensing to non-work devices and concepts is not only determined or shaped by privacy decisions, but P18 found other work–specific PSAI to be limited in “the world of working from home.” P6 felt that PSAI like Sys 4 could be more valuable. He said, “It would give me a true reflection of of how I work, it would give me a true performance evaluation report that I can actually make use of.” Thus, PSAI systems that model phenomena outside work could provide the opportunity for an IW to interrelate all aspects of their life and improve as a totality.

Choosing where to work is not the only freedom IWs have in determining their work styles. IWs often enjoy a broader sense of flexibility where they are rewarded and evaluated for outcomes. Unlike other forms of labor, an IW is not as heavily scrutinized on time–tracking. “Brain work” is often hard to quantify, and therefore workers can approach work tasks at their own rhythm. We found that our participants suspected this flexibility could be hindered with PSAI systems, even if their employment was unaffected or their work–life boundary was secure.

Among others, P1 felt that PSAI inferences might be reductive in quantifying varying work styles. In reference to IWs that work in bursts or “sprints,” P9 said, “it can it can adversely affect people who do not progress in a linear manner.” Similarly, P20 anticipated a simple case of regimenting where PSAI’s inferences would force her to work specific hours instead of simply being judged on her output. For IWs like her who work from home, these systems could disrupt how they choose to interleave their work–home responsibilities. P18 noted that Sys 1 could penalize her behaviors that do not look like work but actually are, such as when they “do laps around the office” or “lay on a beanbag chair.” P7 believed this to be the case when PSAI was limited to “just PC stuff,” such as Sys 3 and Sys 6. More generally, P28 believed that PSAI confined to work applications reinforces an “older view” of work that thinks “you’ve got to be in a place to be able to do a job.” With more IWs opting into remote or hybrid work options, these systems can be considered regressive. However, expanding sensing might not be the solution either. IWs like P1 found that tracking ecological factors like movement and space might not generalize to “varying situations” and could render false negatives. Meanwhile, P6 preferred work–specific tracking over ecological ones because “I could skew the data in favor of my performance being better than it actually is” (referring to screenshots taken by Sys 3). As a result, it would free him to work as he likes. These findings indicate that the very presence of PSAI could establish expectations of a rigid work style and discourage pluralistic approaches to work.

The threat to flexibility posed by PSAI can not only restrict activities but also lead to worker distress. In the quote above, P19 alluded to feeling further commoditized because algorithmic inferences tend to convert nuanced, complex human experiences into streams of numbers. P3 described enrolling into such a system as “a little intimidating” and feeling like a part of a “cold, hard, big institution.” These impressions were likely due to perspectives of PSAI as a tool for reducing the complexity of a worker’s experience into performance metrics. P10 felt that “continuously monitoring for what you do [...] could affect your job more” and P20 even thought it could be “distraction and counterproductive.” Similarly, P5 described that “I don’t like being overanalyzed […] I would be less likely to produce good work.” Other participants like P6 and P11 also alluded to the fact that PSAI systems can exacerbate the Hawthorne effect caused by supervision [3]. It could even lead to negative consequences to an IW’s affect. P24 was concerned that the continuous monitoring required by PSAI could be “stress [her] more” and P10 felt it would “put a lot of pressure on [him].” These perspectives mostly arose from discussions on performance measures and not so much on wellbeing inferences. However, to keep up with a camera-based system like Sys 1, P8 felt they would need to compromise their wellbeing by reducing breaks and socializing at work. Even when a worker might not lose sensitive data, the mere presence of these systems can impact their work effectiveness. This risk is ironic for systems that aim to improve worker wellbeing & performance.

Sharing knowledge in an IW’s workplace is essential for seamless communication and information flows. PSAI systems might develop insights on a worker passively, but how and where that information is distributed needs to be a deliberate process. Here, we describe the network of stakeholders within which a particular IW might want to share the insights provided by PSAI.

A common form of distribution described by our participants was the value of one-one sharing for personal improvement. Others echoed P25 in using PSAI to better explain their work context to their managers (e.g., Sys 2, Sys 3, and Sys 6). This perspective was often described as analogous to existing workplace practices, i.e., “I work with them” (P15), “they know more of your day-to-day” (P7), and “understand the way that thought processes work or deep thinking happens” (P18). What is apparent to be important in this sharing flow is that the manager should be viewed as a stakeholder with real expertise or valid opinion on an IW’s work behavior. Participants suggested alternative experts as well, such as senior collaborators (P7) or advisors and mentors (P6). Sharing PSAI inferences could also be seen as necessary because viewing them in isolation could cause harm. About Sys 7, P20 pointed out that she “would have to use it with some sort of guide or some sort of other coach in order to use it in a kind way rather than in a punitive way.” On inspecting P20’s sentiment further, it was clear that her preference was not for validating estimates but rather for “someone to tell me that what is happening is normal in this moment or common.” Note, however, that these one–one flows still reflect the power asymmetry of information work as, in most of these cases an IW is required to share information with those above them in the hierarchy. Refreshingly enough, we also found some perspectives that go against these expectations. Both P16 and P28 were willing to share their PSAI insights with those they mentor. Thus, a key component in this paradigm is the presence of stakeholders who have the know-how to reappraise the inferences produced by PSAI and, in turn generate more holistic feedback for IWs.

Another sharing metaphor that emerged from our findings was to share information for comparisons and coordination in a many-to-many fashion. The commonly stated purpose was to share PSAI insights to regulate their activities in accordance with the coworkers they aspire to resemble. Akin to P13, P22 felt that seeing others could help them aspire for better work processes. Of course, these many–to–many transactions need to be mutual as P28 said I’d be willing to share my price in order to get that back.” P10 also wanted to be compared but clarified that it did not have to “be a specific number” suggesting abstract methods of comparison. P7 wanted to view aggregated insights for “different types of roles like people that are managers or [...] how much are the analysts doing.” Thus this kind of aggregated benchmarking against other peers can also help an IW identify “normalized” patterns and improve social awareness within organizations. This could be cathartic but could also nudge an IW to understand that certain challenges might be a function of a poor workplace and are thus less “mutable.” Aside from self–regulation, some participants believed this form of sharing could help redistribute workload. P7 envisioned sharing PSAI estimates of her stress with coworkers so that, “they can at least take over some of the easier work.” To complement this P25 wanted to know how the people he was working with were feeling and support them during junctures of low wellbeing (“it beats them up”). Similarly, P18 thought this kind of sharing could help manage work in her team when one of her coworkers was “having a hard time personally.” An extension of the many–many paradigm discussed earlier is to share data completely anonymously for gross aggregation (e.g., Sys 4 and Sys 5). While P4 claimed the lack of specificity in such a paradigm would make their privacy more protected, P20 and P26 believed it could actually serve an altruistic purpose. She said, “I would certainly consent to that if my individual data were to be consolidated with others because I think that there would be a purpose to that.” Although this might not directly benefit the IW, who is a data subject, this could lead to eventual collective benefit, such as cultural change in the concerned organization. Therefore, sharing within a finite network of coworkers can help an IW make more sense of the inferences they receive and leverage the support of coworkers.

In information work, some information about an IW is constantly available for all coworkers to see. It can be momentary information such as availability or work–specific information such as which documents they worked on or when they committed their code last. In regards to this kind of information, an IW’s state can be visible to others unconditionally. However, with PSAI insights, our participants preferred more intentional sharing. Particularly, IWs referred to their ability to negotiate, perception of stakeholder roles, and accountability of the system. This section expands on the factors that can inform flows of sharing.

In the quote above, P15 wanted to assimilate and communicate her understanding of the algorithmic insights before passing it over to another stakeholder. Similarly, P11 said, “I feel like I have got some time to adjust and then sharing would make me feel more comfortable.” P7 remarked that she would share the insights if “it needs to be escalated,” a common way of describing work issues in information work that need more attention. “I would feel better, I would feel more in control,” said P14 when reflecting on the possibility of personally appraising the PSAI insights first. Without this agency, an IW might feel over–scrutinized. P20 even exclaimed that this kind of sharing can be seen as “manipulative”, but that social discretion allows her to fulfill any necessary disclosures to her supervisor. Essentially, IWs need some room to negotiate the algorithmic inferences of PSAI, before they can be distributed any further. Thus, the control that IWs seek is not only limited to whom the data is shared with but how and when it is shared.

The ability to negotiate the PSAI insights does not necessitate that newly generated information can be shared with anyone. Participants had varying attitudes towards different stakeholders in the information flow because of anxieties that the insights could be used against them. Several participants shared P8’s view that their manager is preferred over the HR as a receiver in the information flow (P7, P15, P19, P22). Ironically, we found strong resistance to HR being involved in systems designed for HRM. The responses to these situations were plain and clear, “I don’t want HR to be measuring anything” (P17). HR as an entity seemed to foster a negative connotation, described as a “bad word” (17), “scary” (P23), or “ominous” (P15). Both P13 and P24 anticipated they would be worried about what HR could interpret from PSAI or question them for. Aside from the social connotation, P22 thought HR was too “far removed” from the work context, while P23 thought this was not a part of HR’s role. Alternatively, not only was a manager more relevant to an IW’s functioning, but certain participants also noted that it was actually their manager’s primary function to improve their work experience. P13 even went on to say that they would rather have PSAI directly send insights to managers, “I don’t really care about how well I’m doing in terms of performance, I feel like that’s more of a important measure for my manager.” P4 and P10 described that managers could use these measures to coordinate work. By contrast, other participants expressed greater concern because of managers potentially micromanaging. “I also like the level of separation from my manager.,” said P5, who would prefer the insights be shared with Human Resources (HR). Similarly, P9 mentioned the trust deficit between him and his manager made it challenging to foresee favorable outcomes of sharing PSAI insights. Interestingly enough, P4 thought that HR should be the ones that educate managers on good practices for using PSAI. Therefore, the perception of the functional roles of different stakeholders can determine an IWs willingness to distribute their PSAI insights.

Against the backdrop of the above anxieties, IWs like P28 expressed a need to understand how the entire PSAI information flow was set up. An improved understanding can help an IW anticipate the consequences of misappropriating these insights. P5 called out for greater disclosure and transparency, “Some way to describe the limitations of the system would make me more comfortable with a system.” In fact, some participants noted their indifference to PSAI that they consider ineffective, and therefore invaluable in the information flow (e.g., both P1 and P7 thought that space usage was a poor measure). Besides the mechanics of PSAI’s collection and inference, P9 and P20 called for explicit disclosure of the stakeholders who could access these insights. P1 was skeptical about who manufactures the PSAI, “If they were using a product promoted by Apple to do this, I’m going to be more OK because I know this data is not going to go back to HR.” P11 urged that the flow of information needs to be established within organizational policy, “I think there has to be an agreement of purpose or expectations.” For instance, participants with smaller teams were concerned that many–to–many sharing can lead to negative consequences for their job role or employment (P1, P2, P21, P22). Participants also valued awareness of who else was sharing or how many people were being aggregated. As P16 noted, “given that the culture was such that everyone was having an openness to the material and they felt comfortable with it and it made sense to everybody.” Therefore, IWs tend to expect clear notice and guidance on the scope of distribution in the PSAI system.

The normalization of remote work has made IWs ambiguate which devices are considered work and nonwork, such as the mobile phone [36]. PSAI leverages such devices to model behaviors. In our sample, IWs had distinct preferences for the scope of PSAI vis-a-vis their work–life boundary. This preference was based not only on privacy preferences but also utility. As we see in Section 4.1.2, one stance was that modeling behaviors outside the work–context was intrusive and irrelevant (e.g., sleeping, physical activity, mobile use). The other stance subscribed to the idea of the Social-Ecological-model, which describes individual actions are the result of a multitude of intersecting factors that lie beyond the individual in their ecological contexts [20]. Yet, this must not be confused with using holistic sensing as an excuse for unchecked sensing. Very much in the spirit of Nissenbaum’s Contextual Integrity framework [102], we need to recognize that IWs have different preferences on how they combine and contrast their private and professional lives. For instance, some workers choose to disclose more of their personal situation to their managers than others who prefer to keep it separate from work (Section 4.1.2). Another way to view this dichotomy is by recognizing different strategies to adjust to the asymmetries at work. More information can lead to more power [124]. IWs who prefer segmentation in PSAI might want to ensure the organization does not get any more power. IWs who want complementary information from PSAI might want to increase the power they have. Arguably, it is challenging to reconcile both perspectives and arrive at a universal PSAI template. Having said that, designers of these systems must be sensitive to these individual preferences when trying to solicit consent and provide notice. Describing the technology on these lines can support more informed decision–making for adoption.

Information work allows workers to approach their work the way they like [120]. Certain IWs anticipated that algorithmic inferences produced by PSAI would take away that discretion. They suspected that such technology would measure all IWs by the same yardstick, and that would be unfair to their unique approach to work (Section 4.1.3). This concern could stem from the inability of IWs to determine their own evaluation criteria, which is common in asymmetrical power structures [104]. Hence, IWs resign to accepting that PSAI will also impose—or be used to impose— rigid terms. In fact, contemporary research shows that interventions to provide information workers more agency of their time can help them deal with job demands better [30]. The domain of organizational psychology already has some precedent for this pluralistic notion. Research on PSAI grounded in such work has incorporated measures beyond task proficiency, like organizational citizenship, to define performance [34, 94]. However, research on PSAI has also shown that algorithmic inferences can be semantically disconnected from what workers actually perceive when it comes to predicting abstract constructs like wellbeing [29]. The shift to hybrid work has further compelled the need for a more diverse view of effectiveness that might even include domestic activities [24]. Therefore, PSAI systems need to work more intimately with an IW to retrain themselves based on the data subject’s uniqueness but also infer insights that speak towards their goals.

Generally speaking, participants did not want to share the raw streams of data collected by PSAI, but they still acknowledged that in certain circumstances, the estimates output by PSAI needed to be distributed. For example, they might want feedback from coworkers or want to compare themselves (Section 4.2). The critical challenge in such flows is to protect the IW from being misrepresented by algorithmic estimates. Arguably, PSAI can produce evaluations automatically and at a higher frequency than traditional organizational methods, but these evaluations need to be complemented with human expertise and perspective. For instance, IWs want to only share their PSAI evaluations after they have had a chance to process and add context (Section 4.2.2). Recent work shows that IWs might have a very different understanding of their behaviors than what can be measured by PSAI [29, 66]. Conversely, IWs might not be able to interpret personalized insights or conceive actionable changes without the support of their managers or mentors (Section 4.2.1). Both these use cases represent the need for human–in–the–loop information flows that encourage reappraisal by data–subjects and experts (Section 4.2.1). Whom a worker considers an expert might vary from person to person. Rawls believed that impartial experts and mutual accountability could form a social contract that can legitimize social control that is being proposed by PSAI [110]. Since HR typically has a contentious reputation among IWs, organizations might need to appoint specialized officers for this role, such as the up-and-coming wellness officers, although it is yet to be seen how the impressions of these officers develop over time. Furthermore, the stakeholders in these flows need to be held accountable and build trust with the data subject [71]. The alteration of stakeholders from the flows may not completely remedy the power asymmetry. However, our findings indicate that IWs recognized that changes to the distribution of insights could protect them from further worsening their state. Therefore, these flows need to exist in plain sight for the IWs to understand, augment, and redirect.

Some IWs described that PSAI could be useful to them as complementary information in performance evaluations, to improve awareness of their role, and to justify resource requests (Section 4.1.1). Given the nature of the data it captured, PSAI was viewed as empirical evidence that could drive changes in an IW’s professional state but also in their overall organization. If an IW was overworked, they could convince their manager to rearrange work distribution or give them a day off. Even more so, PSAI insights could be used to bargain better for pay. Typically, in an asymmetrical power structure, the workers have inferior bargaining power [104]. Thus, PSAI must be conceived to maximize the bargaining power each IW has. To empower an IW with such technology, we need to design beyond purposes of nudges and reflection [18] and design for collective bargaining [99]. Data–driven bargaining has its basis in traditional methods for Human Resource Management, such as timekeeping [70]. As workers become more conscious of themselves due to increasing perceived or actual technological surveillance, they have the potential to contest claims by their employer. Ideally, such PSAI technologies must be accessible to IWs independent of their employer and independent of the PSAI their organization may have already deployed. If the outputs of PSAI technologies are only limited to actions IWs should do (e.g., “you seem very stressed, take a break”), they might not be able to accumulate enough knowledge of what they have been doing (e.g., “you have been overworked for 60 days, please consult your manager”). Future PSAI technologies need to make inferences that are reproducible and support sensemaking. Note however, isolated individual understandings can be limited in an asymmetrical power structure [18, 99]. Instead, pooling of information also fit within the norms of distribution, as cumulatively sharing PSAI can help them gain better perspective on the algorithmic estimates (Section 4.2.1). Research and activism on both crowdwork and gig–work have proposed to arm workers with data on their work to combat asymmetry [18, 48]. From our findings, the right iteration of PSAI could serve this purpose for IWs and help them build their own conceptions of these algorithmic estimates. Taking a leaf from studies in crowd–work [119], the next step would be to pursue research on collective platforms to leverage behavioral data for workplace bargaining.

The need for improved legislation on worker surveillance is not new [103], but the urgency at which it needs to be revised needs to match the rapid development (and deployment) of AI technologies [21]. Ajunwa et al. have proposed an “Employee Privacy Protection Act” (EPPA) to limit data harnessed by technologies like PSAI to the work context [6]. They have also proposed an “Employee Health Information Privacy Act” (EHIPA) to tackle unscrupulous data transactions by third–parties, which could help mitigate some of the challenges to PSAI that senses phenomena exclusive of the workplace [6]. Such propositions are certainly a step in the right direction but are centered on confining the flow of the data, i.e., limiting whom they go to, but not how they use it. The algorithmic element of PSAI makes its mechanics elusive, and therefore traditional auditing approaches will be lacking.

Assessing the impacts of PSAI despite the black box. Many PSAI technologies are shrouded as “black–boxes” and this opacity supports certain folk theories regarding what these systems are capable of [49]. It is well known that explainability of machine–learning and AI systems is a hard problem, but we believe adding regulation can motivate developers of PSAI to at least account for the information flows and describe them along the dimensions of contextual norms in information work. We can follow the idea of Model Cards proposed by Mitchell et al. [97], to document intended usage of PSAI. For instance, developers might need to expand on how the algorithmic inferences could be consequential to an IW’s employment with explicitly defined entry points for human–reappraisal and stakeholder involvement. Similarly, they could be required to disclose which aspects of wellbeing and performance are ignored by the system (e.g., “this PSAI cannot be used to infer your team management skills” or “this PSAI is not appropriate for communication–driven roles”).

Assessing PSAI within socio-economic context.Grill and Andalibi had called to increase the visibility of the social impacts of algorithmic phenotyping [49]. Contemporary research has already raised the concerns surrounding the social dynamics of emotional recognition [66], a well–documented manifestations of PSAI. Ideally, regulations must protect against foreseeable but anomalous economic scenarios that compel organizational supervision. For instance, in the future, economic downturn can be used to justify the diversion of PSAI inferences for operational decisions such as downsizing. Organizations can argue these situations are analogous to PSAI for public–health [135]. These crisis scenarios require regulation the most. Auditors should be able to protect certain jobs that are considered more precarious from PSAI, e.g., contractual positions. At the same time, certain sectors might be deemed too austere for responsible utilization of PSAI. Sectors that lack sufficient alternative job openings lead to austerity that makes it illusory for workers to improve with PSAI or even meaningfully reject any enforced PSAI. Future research can illuminate other sociological factors that inform protective regulation. In this way, thinking of more worker-centric PSAI could ensure that individual liberties are upheld despite the necessary supervision required for organizational progress.

A harder question to answer is defining changes in the research of PSAI for workers. The path forward needs insiders to embrace reflexivity on our own methods but also adopt calls for more human–centered approaches from “outsiders” who have critiqued this research. Oftentimes the scientific advancement of technology–supported HRM hinges on capturing and modeling otherwise unseen or ignored phenomena [91, 94, 136]. Sometimes this research is presented as morally indifferent to misuse. This indifference starts eroding when researchers start intersecting with more societal disciplines, such as HCI and CSCW. Yet, research projects and papers that do anticipate misuse are often limited to statements that urge for consented usage. Unfortunately, data subjects might not be able to make informed decisions without the appropriate disclosures of PSAI. Despite our worker–centric approach, in a technology–forward environment an IW’s judgments could be clouded by their personal theories of AI as well as folk theories about the inner functioning of AI-based systems [49]. It is only when we appreciate external critique can we understand the risks of perpetuating PSAI, such as the potential for self–harm [64].

Participatory contributions to development of PSAI. The bare minimum would be to include reflective discussions based on the norms of information flow among information workers (or more specific norms suited to their subpopulation). A more worker–centric approach would be to embed qualitative methods such as the scenario–based interviews we conducted as a formative evaluation. Ideally, researchers should have IWs participate in the entire research life–cycle, drawing upon principles and ideas from participatory action research [95]. Even before IRB reviews, study protocols could be informed with feedback from IWs to understand if appropriate measures or phenomena are the input for PSAI. Later, models can be validated through a participatory lens where co–researcher IWs can vet the practical value or harms or potential harms. Studies like WeBuildAI [79] already provide some framework for participatory algorithmic decision-making. Future work should expand this to algorithmic phenotyping.

External feedback for research on PSAI. Quantitative researchers also need to understand that participatory methods and qualitative evaluations will not create a universally accepted instance of PSAI. As Calacci notes that participatory algorithm design for workers might not be able to reconcile multiple conflicting stakeholders but could at least ensure that normative expectations are not breached [18]. In practice, many researchers innovating new PSAI do not work on recruiting, data acquisition, or participant communication. After all, research on PSAI is often propelled by datasets of behavioral data because these are practical and desirable to support scientific replicability and reproducibility. However, these data also distance researchers from the data–subjects, and in some cases, may lead to dehumanized conceptions of data–subjects and donors as simply “training data” or “numbers” [22]. To mitigate the impersonal relationship between researchers and data-subjects, we might consider setting up an independent advisory board formed of subjects and outsiders. Overall, increasing worker-centered research on PSAI can bridge this gap and produce more sensitive and humane systems to improve prosperity of IWs.