Monitoring Screen Time or Redesigning It?
Two Approaches to Supporting Intentional Social Media Use

Many people report frustration with their technology usage habits. They say that their phones distract them when they would prefer to focus [24], encourage compulsive habits [59], and lead them to behaviors they dislike but cannot change [33]. Some experiences produce more frustration than others, and many people report that social media and casual games are particularly likely to entice them into spending more time on task than they would like [18]. The construct of lagging resistance (in contrast to lagging adoption) refers to the common sentiment of wanting to cut back on the use of a particular app or system but not doing so just yet [1], reflecting people's interest in—and struggle to—change their usage habits.

Some users successfully make changes to their usage habits but only by taking extreme steps to cut off their access to experiences they value. Prior work documents users avoiding social media sites they enjoy [2], locking themselves out of their accounts by handing their passwords over to a friend [1], deactivating their accounts [9], or deleting apps altogether [59]. This cold-turkey approach has costs for users and industry alike. The apps and devices people use often offer enormous benefits to their users, including increases in social capital [14], interpersonal connections [50], and subjective well-being [6]. Thus, it is well worth supporting users in findings ways to use systems they value at the cadence that is best for them. Scholars label the act of achieving this balance “digital wellbeing” [60].

Although users’ desire for digital wellbeing is well documented, this end goal remains elusive. In the present study, we seek to nudge this space forward by contributing to the body of empirical evidence on design decisions that are—and are not—likely to promote digital wellbeing.

Twitter is one of many platforms that incites lagging resistance and user frustration. Twitter is a microblogging service where users, from ordinary people to celebrities, can broadcast short posts called tweets (currently limited to 280 characters each) and follow the broadcasts of individuals of their choosing. In 2006, Twitter began as a platform for sharing brief status updates with friends, but it has since expanded from this exclusively interpersonal focus to serve a wide variety of informational, cultural, and geopolitical needs [5]. Twitter is regularly used by companies to promote their brand [13], by activists to rally the public to a cause [20], by communities to share information during crises [57], by journalists to document breaking news, and by bad actors and specials interests to spread misinformation and propaganda [15].

Like other social media platforms, Twitter leaves many end-users struggling to control their usage habits. For example, people say they wish they could give up Twitter for Lent (a 40-day religious period during which people choose to refrain from certain “vices”) but fail because resisting the urge to check Twitter is too daunting  [56]. Twitter users also report that they find themselves saying “just a couple of minutes more” and lingering on the platform longer than they plan to [29]. Twitter shares many features–such as click-bait titles, infinite scrolling, and constant notifications–with other social platforms that users say undermine their intentions and lead to frustration. For example, Facebook users say they wish they could curb compulsive use [45], YouTube users describe recommendations leading them down rabbit holes [41], and Instagram users report trying and failing to cut down on their time in the app [30]. One goal of the current work is to examine which aspects of Twitter's design users see as linked to compulsive habits and usage patterns they would like to change.

Of course, people also have intentions that drive them to use social media in the first place. For Twitter, the most common gratifications that users seek are obtaining information, connecting socially, and having fun [8, 11, 22]. On YouTube, users seek entertainment, information, and, to a lesser extent, social interaction and status [23]. This raises the question of how users might be empowered to still obtain their desired gratifications from these social media without sacrificing their sense of control and lapsing into compulsive use.

A number of studies have examined how designers might support users in maintaining agency over their technology use and promoting their digital wellbeing [7]. Several investigations have demonstrated that tools to support users in curbing their use of technology can be effective. For example, Ko and colleagues successfully leveraged social support techniques to create an app for groups of college students to set limits on their collective phone use [33]. FamiLync enabled parents and children to cooperate in setting limits on usage [31] and the AppDetox research project enables users to set rules to block specific apps [39]. Other research projects have sought to support users in cutting back in targeted ways. The PomodoLock research project supports users in resisting self-interruptions while trying to focus on a task by locking them out of distraction experiences for short chunks of time [26]. Let's FOCUS reduces technology-related distractions in classroom settings [25]. And Lock ‘n’ Lol supports users in self-regulating their use of technology while spending time with others in person [32].

Other work has brought together multiple solutions or comparative different approaches. MyTime is a composite intervention that leverages multiple forms of support simultaneously, including goal setting, passive information, and timers [18]. The HabitLab platform rotates through more than 20 different intervention approaches, demonstrating that this novelty improves effectiveness [34]. Still other work examines the tradeoffs between different interventions, showing, for example that reminders about goals are more annoying but less likely to make users anxious than interventions that suppress content [45].

Despite the many innovations to design for user agency and digital wellbeing, people's struggles remain widespread. Prior work shows that users who adopt restrictive strategies for reducing their technology use gradually shift to move permissive ones  [35] and other work shows that people often find such tools intrusive and annoying [45]. In a co-design study in which participants created tools to support their own usage goals, people first designed lockout mechanisms and other tools to help them self-police their use but then said they would be unlikely to adopt their own creations [59]. Other work finds that people appreciate digital wellbeing tools and believe they have promise, yet currently find them ineffective in practice [48].

This ambivalence suggests that people do not want to have adversarial relationships with the technologies they use. Today, the primary approach to designing for digital wellbeing is to arm users with tools to shut down their own engaged habits [44, 48, 61]. But users may well prefer not to be enticed into habits they dislike in the first place. Prior work found that removing autoplay and adding parental controls were equally effective in helping children to manage their YouTube use [17]. Consistent with this finding, prior work has posited that creating internal supports for user agency (that is, designs that make the original experience easier to manage) may be a useful complement to external supports, like the current ecosystem of screen time tools, that empower the user to lock themselves out of an experience they enjoy [41]. In this work, we take up the question of what internal and external supports have to offer by conducting an extensive design process to create a set of each for the Twitter platform and using an experimental field deployment to evaluate their effects.

Participants We recruited 130 mobile Twitter users through Amazon's Mechanical Turk platform, one of whom was excluded for incomplete responses. The remaining 129 participants were all located in the U.S. and had previously completed at least 1000 tasks on the Mechanical Turk platform with a task approval rating of at least 99%. Participants were required to own a smartphone with the Twitter client installed and to spend a minimum of 10 minutes per day using Twitter, with at least 10% of their total Twitter use occurring on their phone. Median Twitter use among participants was consistent with the average Twitter user, who used the platform for an average of 10 minutes per day in 2019 [53]. Participant demographics are shown in Table 1.

Materials and Procedures We created a survey modeled on a prior HCI survey study that examined users’ sense of agency when using YouTube [41]. The survey first screened for our inclusion criteria; eligible participants then continued to the remaining questions, which asked about demographics and participants’ perspectives on Twitter. To investigate how the design of Twitter influences users’ sense of agency (RQ1), the survey asked about the aspects of Twitter that make participants feel most in control and least in control when using the platform (Table 2). The order of these survey sections (most versus least) was randomized. We used the wording “feel in control” in keeping with prior work examining the sense of agency [46, 58]. Each participant listed six aspects of Twitter that influence their sense of agency when using the platform, three in response to the prompt asking what makes them feel most in control and three for least in control. Participants were compensated $4 for answering all questions (20 minutes). The complete survey can be found in Appendix A.

Data Analysis The first and second authors reviewed all 774 responses (six per participant) and conducted weekly meetings to develop initial codes, using thematic analysis [4]. We excluded 214 responses that did not include reflections on specific features. Of these, 89 were obviously not related to the question (including some responses that seemed to be generated by bots) (e.g., “I can walk for 60 minutes in the morning, and then build on that success daily”). An additional 41 lacked substantive content (e.g., “I choose when and where I log on to Twitter”), 33 did not mention a specific feature of the Twitter app (e.g., “Ease of use for fast response time”), and 51 contained only general feelings about the app (e.g., “I'm not truly worried about being addicted to online social media browsing but if doing it too much I can always delete the app ”). To ensure that no response was mistakenly excluded, the first and third authors both independently coded the excluded responses with the four exclusion reasons. Any responses which were not excluded by both authors were discussed to reach an agreement, such that both authors agreed on all exclusions.

The coding process was conducted via a shared spreadsheet, and the initial codebook contained 33 Twitter features.² The first and second authors then applied the initial codes to 50 randomly selected responses (25 in-control responses and 25 out-of-control responses) and met to discuss. Interrater reliability was assessed using Cohen's kappa, with κ = 0.80, indicating substantial agreement [37]. After the first round of coding, the entire research team met online and: 1) combined similar codes; 2) developed hierarchical subcodes for codes that appeared in many responses (for example, “notification mechanism” contained subcodes of “skimming notifcations” and “notification settings”), and 3) merged similar codes into coherent themes. Our final codebook consisted of eight themes describing 24 different features (codes) drawn from 560 responses (336 about features that make users feel most in control, 224 for least in control). The first two authors then conducted a second coding pass of all responses together using the revised codebook. The first author wrote a summary of the coding results with examples of each code and discussed the results with team members in weekly meetings. We focus on the 16 features that were mentioned more than 10 times.³ The first and second authors also went through a subsequent coding pass to assess the inter-rater agreement on the final set of codes. The two authors each individually coded 50 randomly selected responses (25 in-control and 25 out-of-control responses) according to the final codebook (with 25 codes); Cohen's κ was 0.89, indicating almost perfect agreement. Although there were 25 codes, the inter-rater agreement was high because each code represents a well-defined feature of Twitter, and most could be directly extracted from the response without ambiguity. For example, the response “Lots of promoted tweets” was easily coded as “Ads.”

Survey Results As shown in Figure 1, the most frequently mentioned features were the ability to Follow and Mute (17.2%), Feed Organization (10.7%), and UI/Usability issues (10.2%). The features that participants were most likely to say left them feeling most in control included: Follow and Mute, Direct Messaging, Proactive Interactions, Organization Functions, Writing Tweets, Filters, and Search. In contrast, Bots and Spam, Ads, Topics of Content, Feed Organization, and UI/Usability issues were mentioned mostly as features that left participants feeling least in control. Several features including Recommendations, Privacy Concerns, Social Metrics, and Notifications received split opinions in the responses.

Table 3 provides a detailed description of the six features that were mentioned 10 times or more and made people feel least in control a majority of the time. In our subsequent design work, we used these six features as the basis of six challenge areas: promising domains for designers to target for improvement when designing internal and external supports to improve users’ sense of agency.

We next conducted a 90-minute online design workshop to brainstorm a wide variety of ways to improve users’ sense of agency when using Twitter (RQ2). We created four design prompts based on the features that survey participants said influenced their sense of agency. Specifically, we asked how designers might give Twitter users more control over how they: 1) find relevant information in their feeds, 2) express themselves by writing tweets, 3) receive recommendations, and 4) receive notifications.

Participants. Our inclusion criteria were that participants: 1) have professional experience in a design role, and 2) use Twitter regularly. We posted a recruiting announcement on email lists and slack channels for design professionals and mid-career design students. This announcement included a link to a screener survey to collect data about demographics and professional experience, and we received 30 responses. We then selected four of these respondents based on demographic diversity and depth of design experience and invited them to participate in the workshop. ⁴ Invited respondents included graduate students in design-related fields and industrial designers with an average of three years of professional experience. All had used the mobile Twitter client for over a year, with two of them using Twitter daily and two using it weekly. Participants received $40 for their time.

Materials and Procedures. During the workshop, the researchers first presented a summary of the survey results and then shared the four design prompts (with 15 minutes allocated to each prompt). After each propmpt, participants were asked to write down their design ideas and upload their sketches to a collaborative sketchboard. ⁵ The sketchboard was divided into four sections (one for each prompt). Each participant worked independently on the shared sketchboard, and then presented their designs and rationale during a collaborative share-out session. In all, 35 design ideas were generated (see Appendix C) with accompanying sketches (see Appendix E). The research team then iteratively reviewed and refined ideas from the design workshop to define a final set of six features to implement in a mobile client for Twitter.

Results. The final list of implemented designs target four of the six challenge areas raised by survey participants, including Feed Organization, Content Organization, Recommendations, and Ads. These features were categorized into two external supports, which are described in Table 4, and four internal supports, which are described in Table 5. These features allowed us to compare how internal and external mechanisms affect users’ experience.

“Chirp” is an app for the Android mobile operating system, which we built by modifying the open-source Twitter client app, Twidere.⁶ The baseline version of Chirp mirrors the official Twitter app: after logging-in with an existing Twitter account, the user sees the same feed of tweets from the people they are already following on Twitter. Because survey respondents reported that recommendations and ads influenced their sense of agency, we also wanted to include these in the baseline app for Chirp, so that we could manipulate them via internal supports. We simulated this content by sampling from trending hashtags, because the Twitter API does not provide a feed of recommendations or ads. In the baseline app, these recommended tweets were inserted into the default feed at the rate of one per every ten tweets.

When external supports are enabled, two new features are available (see Table 4 and Fig. 2). First, a Time Limit Dialog pops up to notify the user about their usage time when the user has spent over 20 minutes in the app. The user can either click “exit” to exit the app, or ignore the dialog. Second, the user can click an hourglass icon on the main interface to bring up the Usage Stats Page. The Usage Stats Page includes metrics such as app usage time, open times, the number of new tweets viewed, tweets liked, tweets replied to, retweets or quotes, tweets composed, and accounts followed/unfollowed/muted. The usage time from the past week is also plotted in a bar chart.

When internal supports are enabled, four new features are available (see Table 5 and Fig. 3). First, a Reading Progress Indicator appears in any feed of tweets indicating that the user has already seen the tweets below. The user can manually pull the feed to refresh, and Chirp will automatically refresh all feeds upon launch. New tweets after the refresh are displayed above the Reading Progress Indicator. Second, a Feed Filter allows the user to filter tweet types by clicking a button on the top of the feed to hide replies/retweets or tweets. Third, a Recommended Tweet Blocker removes the recommended tweets that appear in the baseline version of the app. Fourth, Chirp hides the main feed and prompts the user to add Custom Lists to the bottom toolbar. The user can create and add lists by clicking the add button on the toolbar and manage list members on the list configuration page.

We recruited 51 English-speaking participants from across the United States via email, online forums, and Amazon's Mechanical Turk platform. The inclusion criteria were the same as those for our formative study (see Section 3.1): owning a smartphone with the Twitter client installed and spending more than 10 minutes per day on Twitter with more than 10% of total Twitter usage occurring on the phone. Participants first completed the survey used in our formative design work and installed the Chirp client on their phones. Ultimately, after enrolling and attempting to install the app, 43 participants completed the entire four-week deployment. At the end of the deployment, we excluded participants from our analysis who used Chirp less than four days per week and less than 16 days in total. Among the removed participants, nine used Chirp less than eight days in total, three used Chirp for 10-12 days. All of the removed participants used Chirp for less than 3 days in at least one condition, and their average usage time was 5.7 minutes per day (much less than the eligible participants’ 13.4 minutes per day), which meant that they might not be able to provide meaningful feedback on time-based features such as the Time Limit Dialog. This threshold and was determined before beginning our analysis or examining the contents of the data that would be excluded.

Thus, the final sample was composed of 31 participants (15 men, 15 women, and one non-binary person) with an average age of 36 years (sd = 10.5). All but one participant had used Twitter for over one year; the remaining participant had used it for over 6 months. Twenty-six participants said that, on average, they used Twitter for 5-60 minutes per day, ten for 1-2 hours per day, and seven said they used it for more than 2 hours per day. All participants (including those whose data was not analyzed) received $10 for finishing the first week of the study, $10 for the second week, $20 for the third, and $80 for the fourth (in total $120). We also invited 11 participants to participate in a 30-minute follow-up interview, for which they received $15 Amazon shopping gift card.

We implemented four versions of Chirp to create all combinations of: 1) including or excluding internal supports, and 2) including or excluding external supports. This yielded four versions of Chirp, which each served as a separate experimental condition (Table 6).

For the purpose of deployment, we implemented data logging in Chirp, including the opened times, app use time, the view time for each feed, and the view time after the reading progress indicator is shown in each feed. We also implemented support for showing pop-up questionnaires based on the experience sampling method (ESM) [12] to collect in situ feedback from users.

To protect participants’ privacy, we transmitted the log to a Bigquery database via the Firebase platform, which supports encrypted data transmission using HTTPS. The app did not require any permissions for the data logging (i.e., all logging was performed based on in-app usage). Additionally, we deleted all logged data in Bigquery and stored it in a separate spreadsheet for analysis. The only identifiers in the spreadsheet were participant IDs.

We conducted a four-week, within-subjects deployment. All participants used each of the four versions of Chirp for one week. The order in which they viewed the four conditions was determined by a Latin square design. Each week, the version of the Chirp app on the participant's phone would automatically change to match the assigned condition. Participants were not informed specifically about the concept of external vs. internal supports, nor were they told in advance about the features they would see each week. Each time the app changed to a new condition, the participant was sent a tutorial (in both video and pdf formats) that introduced the features in the new version. For each condition, we introduced all of the features contained in that condition, even though some of the features may have appeared in a prior week, to create a consistent experience. At the start of each condition, participants were required to open the new version of the app and take a screenshot of the new features to ensure that they viewed them at least once.

Throughout the deployment, an ESM prompt would appear after the app was used for 3 minutes and then again after every 15 minutes of use. The prompt was modeled after prior work assessing sense of agency [41] and asked the participant, “For this use of Chirp, how much did you feel in or out of control?” Response options ranged from 1 (less in control) to 5 (more in control). The ESM prompt also posed additional questions as part of a second study that is outside the scope of the current investigation; that data is not reported or analyzed here.

At the end of each week (and thus, each condition), participants filled out a survey about their experience using Chirp. The survey included questions such as, “How was your experience of [sic] using Chirp last week?” For each feature of interest that had been present that week, the survey asked the participant to provide open-ended feedback and to report how the feature made them feel by selecting from the following fixed-choice options: More in control, Less in control, No difference, or Did not use it much. The order in which the survey asked about each feature was randomized to avoid ordering effects. Full survey questions are provided in Appendix F. At the conclusion of the deployment, participants were invited to participate in a follow-up debriefing interview. We conducted 11 interviews via Zoom, where participants talked about their experience using Chirp and thoughts on each feature in detail. The interview protocol is provided in Appendix G.

In our screening survey, six participants reported wanting to use Twitter more than they did currently, six reported wanting to use it less, and 19 wanted to maintain their current usage habits. We aggregated the amount of time each participant spent using the Chirp client in each of the four conditions (see Figure 4, left) and conducted a one-way, repeated measures ANOVA of condition on usage time. We found no significant differences in usage time as a function of condition (F(3, 30) =.434, p =.73).

To examine users’ sense of agency as a function of internal and external supports, we performed the nonparametric Aligned Rank Transform (ART) procedure [62] with the presence or absence of internal supports and the presence or absence of external supports as factors and participants’ ESM responses regarding sense of agency as our dependent measure. We found that internal supports had a small but significant effect on response scores (F_{(1, N = 450)} = 6.73, p <.01), indicating that users felt more in control when this particular set of internal supports was enabled. There was no significant effect of this particular set of external supports (F_{(1, N = 450)} = 1.02, n.s.) or the interaction of the two factors (F_{(1, N = 450)} = 0.12, n.s.). ESM scores by condition are shown in Figure 4, right.

At the end of each week, we asked participants to rate the features they encountered in that week's version of Chirp by responding with a scaled, fixed-choice response to the prompt, “Did this feature affect the amount you felt in control of how you used Chirp?” For each participant, we calculated an average score for each feature. Unfortunately, due to an error with data capture, we were not able to record this feedback for recommended tweets. The ratings for the remaining five features are shown in Figure 5. The Custom Lists feature was rated the most helpful feature in increasing users’ sense of agency, and the Time Limit Dialog had the lowest absolute rating.

For each participant, we averaged responses to all external supports and separately averaged responses to all internal supports, creating two measures for each participant. We ran a paired-samples t-test comparing these averages and found that participants were significantly more likely to say that this collection of four internal supports increased the extent to which they felt in control (mean =.83, sd =.25) than they were to say that this collection of two external supports did (mean =.60, sd =.40, t(29) = 2.82, p =.009, d =.51). Cohen's d indicated a medium effect size.

6.3.2 The Reading Progress Indicator Increases Sense of Agency. Similarly, interviewees reported that the Reading Progress Indicator increased their sense of agency, saying that they could more easily spend their time the way they wanted to. Eight of the 11 interviewees said that this feature consistently helped them feel more in control of their time on Chirp. For example, P18 explained that it, “definitely helped me use [the app] in a more focused manner...it improved the quality of time. It's hard for me to estimate [any effect on] quantity.” Participants explained that this change arose from the simple fact that they had no wish to inadvertently spend their time re-reading content they had seen before, and that they appreciated having the interface support them in viewing new content they valued rather than leading them down an infinite feed of yesterday's news:

“I dislike that it [Twitter] is endless and you can kind of scroll forever, so I like that little banner that [says], ‘Hey you've scrolled to the point where you've already looked at this stuff.’ I think that helps me not look at duplicate content and go, ‘oh this, like, looks familiar’ but I'm kind of wasting my time in a sense, just viewing something that I probably looked at yesterday.” (P3)

By proactively telling users when they were about to scroll into the content they had seen before, Chirp became less of an adversary and more of a partner in helping participants use the app the way they wanted to. Participants said that rereading old content is almost never how they want to spend their time, making it useful for social media platforms to warn users when they are about to do so, rather than seamlessly encouraging them to continue scrolling. They explained that they would value this kind of interface support across all of the social platforms they use:

“I think most people only want to go to consume new content. Most social media apps I don't think they bother to tell you when you've already read something before just again because they want to keep you on the app for as long as possible...I think it's really important to have that [the history label] in social media.” (P2)

Nearly three-quarters of survey participants said that the Reading Progress Indicator helped them feel more in control (see Figure 5), and interviewees repeatedly told us they experienced this feature as helpful and supportive, saying things like, “It let me know that there was nothing else to search” (P8) and “I thought it was pretty helpful just to know that I've viewed everything” (P26).

We also examined whether the Reading Progress Indicator affected the amount of time participants spent with content they had previously seen. In each condition, we logged the amount of time participants spent viewing content they had seen before; the Reading Progress Indicator was visible and marked the transition to old content only in the conditions where internal supports were enabled. The amount of time (in minutes) that participants spent with old content is shown in Fig 6.

We log-transformed the amount of time participants spent viewing old content to make it fit gamma distributions, and we performed analyses of variance using a generalized linear mixed model (GLMM) with a gamma link function. The independent variable was whether the Reading Progress Indicator was shown or not. The result showed that the appearance of the Reading Progress Indicator significantly reduced the time participants spent with content they had previously seen ($\chi ^2_{(1, N=2858)}=381.42, p <.001$).

Participants’ reactions to our internal supports were uniformly more positive than their reactions to our external supports. Although people's reactions will certainly vary as a function of the specific supports that are put in place, the external supports we added (a feature for monitoring usage and a timer asking them if they would like to close the app) reflect two of the most common design patterns across current digital wellbeing tools [44, 48]. Thus, participants’ greater appreciation of the internal supports we created–relative to these two common features–suggests there is room for designers to improve upon the status quo. It further suggests that adding internal supports that are tailored specifically to the interface of interest is a promising approach for making such improvements.

One inherent advantage of external supports is that they are agnostic to the experiences they monitor and can be applied across an entire ecosystem of content. A time-tracking external support can operate across all apps on a smartphone, for example, and does not require participation from the individual companies creating third-party experiences. As a result, external supports may remain a useful tool for designers to deploy in instances where internal supports are not an option. We found that monitoring usage information and sharing it with the user on demand (the Usage Stats Page) was more effective in increasing agency than a dialog nudging participants to close the app (the Time Limit Dialog). Participants explained that the Time Limit Dialog at times even undermined their sense of agency rather than improving it, demonstrating how external supports can backfire. Thus, our results suggest that if designers continue to create standalone external supports, they will be more likely to achieve the aim of increasing user agency if they simply provide information to the user without acting on it on the user's behalf.

Prior work points out that people find this kind of passive monitoring to be ineffective in changing their behaviors in the long run, as it lacks the restrictiveness that people say they need if they are to change their habits [48]. In theory, this might suggest designing more powerful lockout mechanisms, but participants in our study were wary of intrusive features telling them what to do. Although they agreed that more restrictive interventions might be effective, they said such interventions would be better for “other people,” a stance that is consistent with people's reluctance to adopt screen time tools [59] and tendency to abandon them [34]. This past work taken together with our findings suggests the most useful external supports may be those which passively provide information and target short-term education and reflection rather than trying to encourage users to adopt tools with heavy-handed enforcement for the long term.

By grounding our internal supports in the specific challenges posed by the current Twitter interface, we found that we were able to move the needle slightly on people's sense of agency in the context of consuming Twitter content. More usefully, we found that these internal supports made inroads in the specific usage scenarios they targeted, suggesting that if designers attend to the specific barriers to user agency that are most notable, they will be able to effect targeted change. People stopped scrolling through tweets they had already seen when the Reading Progress Indicator was visible, and they reported spending more time with the content they were currently interested in when their feed was segmented by the Custom Lists feature.

Collectively, people were less ambivalent about the specific internal supports we created, and they more consistently said that these features affected their sense of agency for the better. This stood in contrast to their mixed responses to the specific external supports we created, which at times they said undermined their agency and would be a better fit for someone else. These findings suggest promise in taking a tailored, platform-specific approach to designing for digital wellbeing. They further suggest the potential value of maintaining a catalogue mapping specific design patterns for internal supports to specific outcomes for users. Custom Lists could easily be adapted as a generic pattern for segmenting any space that aggregates disparate content with the potential to pull users in many different directions. The Reading Progress Indicator could be one instantiation of a pattern to annotate content to make its relevance or irrelevance more salient.

Many prior studies have shown that current screen time tools and other external supports do not yet meet users’ needs [48, 59]. Our findings suggest promise in exploring a design agenda that focuses on developing internal supports and abstracting these supports into patterns to be leveraged across experiences with similar designs. Participants pointed out that different forms of support can be in conflict, with some guiding users toward more targeted and meaningful patterns of engagement and others giving blanket nudges to cut back on all engagement. The former is more aligned with the perspectives of the participants in our study (who said they wanted to improve the quality of the time they spend on Twitter rather than the quantity) and with prior work reporting that users are seeking experiences that are inherently more meaningful, rather than tools they can use to cut off their access (e.g., [59]).

Participants’ descriptions of their experiences suggest that usage time has limited value as a metric for understanding their preferences. This held both with respect to creating engaging experiences and with respect to supporting agency and digital wellbeing. Many described their favorite added feature (such as the Reading Progress Indicator or Custom Lists) by saying it improved the quality of their time without changing the quantity. Others said that they could not be sure of its effect on time spent. Some people said it helped them cut back on the total time they spent on the app while others said they spent more time with Chirp once the feature was introduced, because the experience of using it became more valuable. Rather than rereading tweets they had already seen or sifting through a sea of clutter, participants said they more easily navigated directly to the content of interest. Digital wellbeing efforts often seek to cut back on engagement, but deployment participants said that what matters most is that their engagement is meaningful.

This points to at least two conclusions. First, designers who seek to create engaging experiences cannot assume that increases in time-on-task reflect increased user interest and engagement. Digging through a frustratingly irrelevant feed of content, in some cases, increased the time participants spent but not their interest in the experience. Digging through the clutter was one of the most frequent complaints in our formative survey, and the Custom Lists feature that segmented and organized this clutter was the most appreciated feature. Helping users reduce the time they spent bushwhacking increased their engagement and appreciation for the experience. Second, designers cannot assume that decreases in time-on-task reflect improved digital wellbeing. Many participants said they used Chirp more when they had supports for increasing agency, because the experience became more worthy of their time and attention. Currently, many interventions in this space emphasize reducing time-on-task through tracking interfaces and nudges.

Although usage time is one of the most common metrics in the digital wellbeing domain, our results reveal its limitations. Instead of relying upon screen time as an outcome measure, digital wellbeing designers might develop a more nuanced understanding of user goals or adopt alternative measures such as self-reported sense of agency.

This work has several limitations: 1) This work examined a small, specific bundle of internal and external supports, and there will likely be wide differences in the effectiveness of supports within either of these broad buckets. Thus, our conclusions about the comparative effectiveness of internal and external supports broadly will not apply to all possible designs or combinations of designs that fall into either category. Furthermore, we implemented four distinct internal supports in contrast to two distinct external supports, and we did not have a mechanism for predicting the magnitude of any one of these features. Thus, these results should be interpreted as providing data about our specific designs, and we cannot make strong claims about all internal supports or how they might compare to all external supports. 2) Our formative survey was conducted with participants who used Twitter more than the average use time of 10 minutes per day [53], and we may have come to different conclusions by systematically examining user groups with other habits. 3) We conducted our deployment with a small number of participants, which may have masked effects that we would see with a larger sample, potentially preventing us from seeing trends in people's experiences with these features. 4) Our Usage Stats feature presented a static visualization. Having an interactive dashboard (such as the iOS Screen Time page) which allows the user to inspect their usage in different time granularities might be more helpful. We also did not explore other external features such as time management planning tools [42], which allow the user to plan ahead and reflect on how much time they would like to spend with a particular app. 5) Separately, prior work points out that the passive usage monitoring we explore here raises privacy concerns [61], which we did not investigate in the current study. This complicates the design agenda for external supports and suggests designers to consider the tradeoffs between supporting the user's agency over their attention and agency over their data. Future work remains to more robustly model when and how internal and external supports can benefit users.