1 Introduction
0s and 1s on a screen. The Cloud. Fast moving. Clean. Efficient. Exponentially growing. Data Centers. Code on the black screen of a terminal window. Buzzing. Such images construct part of a shared sociotechnical imaginary around data. As data increasingly become part of the most intimate parts of people’s lives, they remain paradoxically a largely invisible phenomenon. In the context of home Internet of Things (IoT) devices (the focus of this paper), the operations that produce and make data legible are obscure. Data collection lacks transparency, data analysis includes opaque algorithms, and trends and insights from data often only benefit the large corporations owning the data collection devices. People living with these devices are left with the services offered (the assistance of the smart speaker, the temperature control of the smart thermostat, and so forth), and sometimes an app or dashboard that allows real-time data display and shares back an archive of the data collected over time. Because of these often restraining interfaces, imagination—and the creation of shared imaginaries—plays a central role in how people know and understand data.
Here, imagination is understood not as the opposite of reality but rather as a vital dimension of it. Building on a long philosophical tradition [
4,
9,
13,
40,
58], imagination is seen as the capacity to represent: to create images of something that exists or does not exist. Yet, our imagination around data rarely exists in a vacuum, rather it is deeply enmeshed within society. Jasanoff and Kim define sociotechnical imaginaries as “collectively held, institutionally stabilized, and publicly performed visions of desirable futures, animated by shared understandings of forms of social life and social order attainable through, and supportive of, advances in science and technology” (p. 4) [
57]. In short, views of data as seen in the media, pop culture, corporations, and legislature reproduce and solidify specific understandings of data. The challenge is that current data imaginaries are often restricted or homogenized by technocentric discourses that present data as objective, neutral and transparent [
30,
52]. In
Imaginal Politics [
13], philosopher Chiara Bottici asks: “How do we account for the paradox of a world full of images, but deprived of imagination?” Bottici’s question resonates not only with the sphere of political life and institutions (the focus of her work) but with the reality of data imaginaries as well. How is it that images, stories, and imaginaries of data are so often reproducing the same tropes [
52]? For a phenomenon as expansive and elusive as data, why don’t we have more diversity in how we portray, imagine and represent data? One dominant and homogenous vision of data leaves little space for alternative accounts, and prioritizes a very partial perspective: a view of data as a realistic representation of a phenomenon. While many may adhere to this view of data, other perspectives are needed to account for the plurality of domestic experiences people may have (or want) with data.
In an effort to diversify data imaginaries, we created the Data Epics project. We commissioned seven fiction writers to write short stories based on people’s home IoT devices’ data. Each writer was paired with a household and worked with four sets of monthly data from devices such as smart plugs, a smart bed, voice assistants, a smart camera, a garage door opener, a smart exercise bike, and motion sensors. The result is a collection of 28 short stories in which data plays a variety of roles (from narrator to main character) and where a reader might encounter the worlds and lives of data. By proposing this novel type of data representation, quite different from data visualizations or data physicalizations, we emphasize the interpretive nature of working with data and position imagination as a central pillar to understanding data.
We argue that the Data Epics are particularly well suited to challenge common assumptions about data, or in other words, that by making new data imaginaries, they
unmake [
85] existing assumptions about data. They become, in the words of Jasanoff and Kim [
57], “experiment and demonstration”: the basis for a society’s self-reproduction through “the enactment and reenactment of its imaginaries” (p. 5). By building alternative stories about data, we can start to counterbalance the commonly reproduced (technocentric) imaginaries about data. Here, we see imaginaries as encompassing the ‘images’ or affects associated with data (for instance the cloud, fast moving, ones and zeros, and so forth), but also how data behave and work (for example that data are neutral, knowable, or distinct from fiction).
If ‘experiment and demonstration’ are the basic for how data imaginaries are made and maintained, we argue that they may be unmade and pluralized in the same way. Hence, we use Unmaking as a guiding analytical lens for our analysis of the Data Epics. Within
Human–Computer Interaction (HCI) and Design literature, unmaking has emerged in recent years as an approach that values taking things apart to better understand them [
72] and one that makes space for alternatives [
39]. In the context of personal or domestic data, data is still often perceived through a techno-solutionist, capitalist, and positivist framing [
30,
96], which eclipses the realities of data capture and interpretation. Unmaking this framing is an important step toward having more open discussions about privacy, control, access, and the value of data in domestic contexts. Those discussions are central for people to reclaim their own data, back from the large corporations who currently offer services through IoT devices.
In this article, we discuss how the Data Epics and the process of data fictionalization unmake (and make new) data imaginaries in the context of home IoT data. We analyze interviews with the writers and the participants, and present seven ‘un/makings’ of data. We conclude with a reflection on the new data imaginaries made with the Data Epics, the resistance in unmaking certain imaginaries (such as seeing data as an accurate representation of self), and the work to share new imaginaries.
4 Findings: Making and Unmaking Data Imaginaries
We report on the participants’ reactions to, and experiences of, their data stories, as well as on the writers’ processes of turning data into fiction. We don’t expect the reader to know the Data Epics stories as much as we do (although they are in the supplementary files if anyone is curious) and we know it can become disorienting. So, we organized the findings below under seven un/makings of data imaginaries. Each subsection follows a particular pair of writer-participant and their four stories. We don’t discuss each story in detail, but emphasize elements relevant to the specific un/making discussed. For each key finding (or un/making), we provide an illustrated metaphor that captures the core movement or orientation of these particular encounters and processes. Given the richness and scope of the participants’ and writers’ experience, we offer these metaphors as synthesizing principles to outline and illustrate key focal points of the project. In addition, each subsection starts with an excerpt from one of these four stories as a way to display the tone, style, topic, and voice of the stories and illustrate the arguments we make. We also include a small table that presents an overview of the four stories discussed (including title, IoT device, and story plot).
Figure 3 offers an overview of the seven un/makings to illustrate how they are both juxtaposed and interwoven. All seven present ways to unmake the common technocentric view of data. As a whole, our findings challenge the idea that data may represent and be about the self, put into question the neutrality of data and interpretation, examine the links between fact, fiction and creativity, and illustrate issues in ‘knowing’ and accessing data.
4.1 The Selves: Mirroring Multiple Identities
For this participant (Stephanie) and this writer (Alex Madison), exchanges of data and stories explored the notion of the self in a parallel, or perhaps asymptotic way. Over the course of the four stories (see
Table 2), Stephanie sought to see
herself in the stories while Madison purposefully used the medium of fiction to explore the
multiplicity of selves that data capture and record (
Figure 4).
When Stephanie received her first story, she was delighted to see her exact voice commands italicized: “It was really reflective of my day-to-day thoughts and that was really cool.” The Other Fish tells the story from the data’s point of view, waiting on their human counterpart to feed them questions so that they could become a more realized version of themselves.
Stephanie reflected on The Other Fish as if it was a sort of record—a diary—of her musings and the events that had happened over the weeks, even ones that she had forgotten about. The joy that Stephanie felt reading the first story did not last long, as her relationship with the characters in the next stories felt more distant from herself. She found the second story, Stop the Music, while still told from the point of view of the data, contrasted with the first story because of its watchful and “cryptic” tone. The italics that she enjoyed in the first story were gone, making it difficult for Stephanie to associate her data with the “creepy” story she received. All of sudden, the story felt simultaneously indiscreet and removed from her:
The songs she prefers as she eats her breakfast and the songs she turns to when she can’t sleep – each one adjusts my appearance and tells them more about her. (Stop the Music)
Stephanie continued to be disappointed with the next stories, as she hoped for the stories to take the same warm and “cute” tone as The Other Fish. The third story, The Best Time to Come Back to Me, centered around a male main character with a partner, Lydia, to which Stephanie responded: “I don’t know who he is. Or who she is. Or why it relates to me.” Throughout the project, Stephanie desired relatable characters and to easily pick out the pieces of the story that were inspired by her and her data. She didn’t seem bothered that The Other Fish was not completely accurate—in a sense, she liked the imaginative spin that Madison took on the utterances Stephanie had given to the Google home. But as Madison continued to explore what data is outside of the home (how it originates, or the warehouse before being in the home, for instance), Stephanie felt a larger and larger disconnect between herself and stories. She felt as if she was intruding on someone else’s story, almost like she was reading someone else’s diary.
For Madison, this disconnect was intentional. “I didn’t want to create this ghost of a person in my head”, she explained in the third story interview. When she received the second dataset, Madison remarked that it was “an update on the character” and contributed to flesh out the outline of the person she had started to imagine behind the data. When she received the third dataset, she decided to break from this “shadow form of the person” by changing their gender and explore “different narrative possibilities.” This fictional exploration was woven with a larger reflection on data that had been ongoing since the beginning of the project for Madison: “I started to think about how data is tied to other individuals; the data we produce is only one piece of this data body, a body that is distributed across several datasets and devices.” As Madison’s understanding of the diverse body of data grew, her stories evolved to include more varied data perspectives and protagonists. Beyond this multiplicity of individuals ‘behind the data,’ the idea of data as a record of our past selves was particularly interesting and troubling for Madison: “this idea that whatever we do is stored as data” and therefore that our past selves keep existing in the form of data traces.
While the notion of self in Madison’s process took on more varied and expansive forms, Stephanie’s remained firmly attached to the orbit of her own self—which seemed unavoidable given the nature of the exchange: ‘her’ data in exchange of ‘stranger’s’ stories. Stephanie’s experience went from delight to confusion to disappointment as the expectations of a recognizable self-evaporated, while Madison sought to disrupt the ‘familiar’ outline of the person that started to form through the data and her stories. Through their parallel experiences, we begin to understand how the process of data fictionalization unmade the expectations of data in its relation to us, the producers. When data is dissociated from ‘us’ it becomes less interesting, less useful and, possibly, just ‘someone else’s.’ But this ‘stranger’s data’ can also be the starting point of a larger reflection of the diverse ‘data body’ we all contribute to and that Madison explored in her stories.
The Un/Making. Data is often imagined as a mirror, reflecting our behaviors and revealing truths about our lives. Stephanie liked the first story because it reflected her own self but felt uneasy about the others because the characters born out of her own data felt foreign, even ‘creepy.’ For Madison, this unmaking was intentional: instead of growing the shadow of the person she perceived in the data, she decided to explore the other personas that could emerge out of it. Madison used the medium of fiction to make new data imaginaries that refracted these selves into other narrative possibilities.
4.2 Expansion: Data Spillage
Similarly to Alex Madison’s desire to move beyond data as a direct mirror of self, Garrett Saleen’s stories (see
Table 3) showcase an ever-expanding portrait of data: from a single data’s life to a more systemic, global, and impactful role of data in the world. Saleen’s writing challenges the assumption that data could be private or personal. As Saleen’s explorations reach further with each story, Robert (participant) finds his understanding of data slowly unravel into a messier collection of loose threads. This progression illustrates the unmaking of the idea that home IoT data is ‘homebound’—that it only is concerned with the home, and that it can only impact domestic life.
In his first story, Lights, In White Satin, Saleen tells the story of a smart light bulb that contemplates the value of its own life. The existentialism at the core of Saleen’s first story puzzled Robert at first: “It was so abstract; I thought I had someone else’s story.” Robert later shared that after this initial experience he had to “shelf” his expectations of a story being about his home or his household.
In his second story, A Parable, Saleen moves from the use of ‘I’ as narrator to a mix of ‘I’ and ‘we,’ indicating the plural quality of data (now coming from a set of three smart bulbs). As Saleen explains, the story showcases how data:
“are learning from each other, and they are making leaps, you know, in logic and things. I think it’s a much more collective story. It’s a story of a bunch of different things interacting with each other, even though they are kind of the same thing and they’re all in kind of the same boat.”
As seen in the excerpt in
Figure 5, here we start to see how data might organize on their own, how they have a world of their own, and unique goals and desires. By directly centering data as characters, Saleen builds a surprising (in particular to Robert) world where data exists beyond the home.
Further expanding the reach of data, in story 3, Saleen experiments with, and pushes to the limits of, how much impact data might have on people. Assemblage is a story of an aging and grieving man named Frank who receives a Peloton smart stationary bicycle from his daughter and embarks on a journey of self-improvement. Saleen discusses his inspiration for the story: “I watched like 40 minutes of like Peloton reviews on YouTube. Just to get like a sense of sort of the weird subculture around it. Then I sat down the next day and like I knocked the first draft of the story out.” Data plays the crucial role in supporting “the brand,” with an ultimate goal of the (literal) transformation of bodies into machines. In the unexpected and surreal ending, Frank is injured and brought to the ‘branded’ hospital, where he undergoes a surgery that effectively turns him into an exercise bicycle:
His elbows are sewn together, the hands rubberized, heated slowly and elongated into handlebars. His legs are bent forward, welded together and run against an industrial edgemaker, repeatedly folded and heated and cooled until they thin to a razor-sharp wheel. The head is bent back and shaped into a saddle and partially filled with a high-quality memory foam, his pelvis grafted down into a stabilizing stand. (Assemblage)
Robert described this story as ‘Kafkian,’ the transformation at the end reminding him of the author’s famous story Metamorphosis. He and his partner found the story much more narrative and therefore more enjoyable. His partner Rachel was the main Peloton user, and Robert noticed an increased interest for the stories they received. “They think I am an old man!” he reports her exclaiming. This humorous remark also points at the often implicit expectations both Robert and Rachel had about the project: that it would be more explicitly about them.
The final story in Saleen’s collection, Disaster Variations, also based on Peloton stationary bike data, offers an even more impactful view of data’s reach into the world. Here, a data scientist’s stationary bike workouts have global consequences–“the extinction of species, the flooding of cities, the liquidation of economies” (Disaster Variations). When asked to summarize the story’s plot, Robert explained: “It was about a data scientist who thought, and had the data to prove I suppose, that his Peloton workouts helped stabilize society” and added “of all the stories, it was the most tangible and believable. It didn’t feel like an acid trip.” Robert later shared that his experience of the project and of the stories would have been very different if the last story had been shared first. “It would have been better to start more narrative and go more abstract” he said in the exit interview. The effect of starting with very loosely plotted stories which focused on data as characters made Robert’s expectations and experience unravel from the beginning. His imaginary of data was not captured, he felt, in the stories, which he found generally too ‘abstract.’
The Un/Making. Saleen’s process of data fictionalization led to a centering of data’s lives, worlds, and impact which unmade the data imaginary that home IoT data are private, personal, or contained within the home. The making, and depiction, of these more expansive worlds was unsettling for Robert as it moved away from his belief that data (and hence the stories) would be representative of his own household.
4.3 Introspection: Engaging the Personal in Data Interpretation
The findings so far have focused heavily on ways data may or may not be translated to represent the data producer. Here, we turn to the process of interpretation that is central to the act of translation (or in our case fictionalization). As a pair, Lahim Lamar (writer) and Oliver (participant) illustrate well how the process of data fictionalization can foreground a sense of looking inward and looking at the ‘people behind the data.’ Over the four stories (see
Table 4), this became central to Lamar’s writing and allowed Oliver to reflect on who might be behind his own data. In their exchanges, it became clear that the assumption that data is neutral was unmade (as scholars like D’Ignazio and Klein [
30] amongst others, have argued recently). Instead, human interpretation, bias, and positionality were foregrounded (
Figure 6).
Lamar began his writing process for the first story by looking for outliers in the data he received. For the first two stories, he combed through the spreadsheets and the data visualizations of a Chamberlain Garage Door (story 1) and Wyze Camera motion data (story 2). He looked for any disturbances to the typical patterns, searching for something to latch onto and make a story out of. While in the first, Owl and Daughter, Lamar told the story from the point of view of the garage door, in the second one, All Bodies Are Transition, he used the point of view of the data itself and touched on themes of embodiment, gender, and imaginaries.
For Oliver, even if Lamar aimed at staying close to the data, it was hard to see how the stories related to him or to his data. Yet, he reflected on new sides of IoT and IoT data that he hadn’t before such as how “callously” we treat devices, or how data might ponder its own body. While in the first story Oliver felt a need to see his own data, over the course of the next stories, he was able to extrapolate ideas from the stories and think about how they could be “applied to [his] data.”
As Lamar’s stories continued, they became more personal for him and less reliant on the dataset that was given. In
The Galaxy of Her (story 3), Lamar worked with a Google voice assistant dataset. While the voice commands were revealing a new side of the household, the prompt (see
Section 3.3) we gave him played a larger part in his ideation. The excerpt from
Feeling your data: Touch and making sense of personal digital data by Deborah Lupton [
66] emphasized the interwoven assemblages created of data and bodies, and led to multiple “branches of thought” for Lamar. Through this inspiration, he began to expand the role that his own point of view could play in the narrative, weaving in topics of accessibility and companionship which were not only areas of interest, but part of his career and lived experience. Lamar exemplifies this with his writing of Galaia, an AI companion who is partnered with a human to help her see and through this companionship they develop a deep bond:
The water that filled my eyes surprised me. To think she had been suffering silently without telling me. I decided to be honest. “I miss being able to see. I miss being able to go hiking and see colors like I used to. I…haven’t decided yet.” ‘But you will. And then we will not be one anymore.’ ‘I wouldn’t get rid of you Galaia. I would keep you, still.’ ‘Everything changes. Life is change. I must accept this change. I must learn to be happy for you. Even though I’m a little mad at you.’ ‘You will always be with me.’ ‘But I will grow old. I will become obsolete. I want you to have the best. I want you to upgrade.’ (Galaxy of Her)
Oliver found this story to be more intimate: “private, kind of emotional, just like the relationships and emotions that connect us.” In our interview with Oliver, he further shared with us that sometime around story 3, he decided to deactivate his Facebook account. He said he isn’t sure if that decision is related to the Data Epics project, and that he had been reading more about how Facebook “uses our data that’s not really for our own benefit. So it just was one of the things I realize that you’re just like a product and a means to an end, for them. I don’t want to be part of that anymore.”
Lamar’s final story, When You Don’t See Them Coming, is his most personal and the dataset that was, possibly, the most mundane of all—a collection of 30 Nest Camera videos recording the street outside of Oliver’s household in 30 second segments. The dataset prompted Lamar to reflect that “these videos don’t have much meaning until you apply a filter on to it,” which became the genesis of the final story. It begins in a classroom with students looking at a map of the world, where Africa is disproportionately small in comparison to Europe, prompting the two Black students (which included himself) in the class to question how it was made in the first place:
Still I wondered: how then could any map be trusted? Any chart? Any dataset? It was impossible to separate the author from the content. (When You Don’t See Them Coming)
Reflecting on his process, Lamar states: “I was thinking of a time when I was really emotionally affected by something that has to do with data… and that immediately came to mind. When you’re writing you want to sort of pinpoint things that touch emotional and it’s hard it’s sometimes hard to connect data to emotions.” Just like with story 3, this last story had a strong impact on Oliver—not because of how it might have related to his own data, but because of the human interpretation Lamar created. Oliver explained how he thought the story was about “challenging how you accept data” and “question why it is the way that it is, and so on, necessarily don’t accept the data is simply as truth.” As the project was concluding, Oliver reflected on how his understand of data changed:
“Data isn’t like math it’s more like English where words were like open for interpretation and bias. The bias of the person that’s like composing it or writing it. And I know a lot of times like the data tries to masquerade like it’s math where there’s only everything is absolute. [But] you have to look at the bias, basically who’s collecting it, and what their motivation was. You can’t trust it at face value.”
The Un/Making. As Lamar drew more and more on his personal experience to write his Data Epics stories, Oliver let go of the expectation that the stories would represent him directly and embraced how these interpretive data fictionalizations made him reflect on his relationship with data more broadly. By unmaking narratives of data as neutral, both Oliver and Lamar made new data imaginaries that foregrounded the human labor and interpretation behind the data.
4.4 Peeling Apart: Investigating Fact and Fiction
While Lamar and Oliver’s relationship highlighted the personal within processes of interpretation, Mary and Taylor’s (participants) reception of the stories hinges on the superposition of fact and fiction. Their reading process, over the course of the four stories (see
Table 5), was like peeling layers apart to discover their data amongst the fictional elements of the stories (
Figure 7). Through this process, we saw an unmaking of the common idea that things are either facts or fiction [
111]—that data are either true or false, real or invented—and that they can clearly be delineated.
Mary and Taylor started their experience with two stories where they found it hard to see themselves or their data. They looked for links and threads that would connect their data (coming from a smart thermostat and a smart bed) to the stories. After reading the second story, Taylor remarked wanting something back from the stories, especially since they had given “something of themselves” to the process. However, something changed with stories 3 and 4 (both using Alexa voice assistant data). Mary said “finally I can see myself!” The title of the third story—Mud Room, Fairy Lights—made her “laugh hysterically”, as it is a voice command their family uses to turn on the lights in their mud room. This unique command gave confidence to Mary and Taylor that this was indeed ‘their’ story, one they could start ‘reading’ in a new way. That command, along with many other ones her family regularly uses to add things to a shopping list or play music, became part of a rhythm within the story, as seen in the excerpt above. Mary and Taylor often tried to reconnect data to the events that might have produced this data, challenging and questioning the ‘truth’ status of data, as seen in the stories. For example, we look at Mary and Taylor’s conversation around a potential order of coffee beans, as presented in their fourth story, Echoes, Algorithms, by Garrett Saleen.
Working with their Amazon Echo voice assistant data, Saleen writes about the tumultuous and oppressive relationship between a device (Echo) and the algorithm ‘feeding’ on the data collected by the device. In short, Algorithm presses Echo to find out more information about its users so that the system can sell more and make more profit from them. In this excerpt, which Mary and Taylor discussed in details, Algorithm asks Echo about recent coffee purchases:
“What do you know about coffee beans?” The Echo didn’t know a thing about coffee beans but wanted to make the Algorithm sweat. “They might have said something, but I can’t recall.” “He ordered three bags of coffee yesterday. I have no data on any variety of coffee brewer. Is he aware of the fact that by linking you to any number of Echo-compatible coffee machines, he can start brewing simply by entering into a room?” (‘Echos, Algorithm’)
When we asked Mary and Taylor (interviewed together) if they could recognize their data in Echoes, Algorithms, they pointed out a few examples of commands they had said during the previous month, before spending more time discussing the coffee beans order. At first, Mary and Taylor assumed the utterances in the story were all coming from their dataset. Mary states “I remember thinking, because it said he ordered three bags of coffee and I just assumed it was you.” Since Mary usually doesn’t order coffee beans this way, logically her next guess is that Taylor (her husband) ordered. In this line of thought, she also considers others in their home, like their daughter. She jokes: “Is [Julia] ordering coffee beans when we’re not looking?”
Taylor is relatively confident he didn’t: “I sent you [the researchers] the excel spreadsheet, but I am 99% sure that there’s nothing in there about coffee beans.” While Taylor tries to remember, Mary further dives into what the data precisely contains. She asks:
“When they [the author] have access to the data all they’re hearing are those literal commands. They wouldn’t have access to what the algorithm did with the commands, right? What if the, you know, Amazon algorithm is trying to sell us beans, you know what I mean, could they see that, the author, or no?”
This conversation is making her test her mental model for how the data, the device, the service, and the algorithm work together. At this point, Taylor hypothesizes that perhaps this was an element of fiction in the story: “I was trying to think, you know, I think they just made it made it up.” Mary agrees. While they are now more certain that the coffee beans order was a fictional addition part of the story, Mary reflects on how ‘believable’ that was, saying “Well, the ordering coffee would be, it would be something I would do.” Taylor concurs “That’s so true” and Mary concludes “It was good fiction.”
The Un/Making. For Mary and Taylor, the convergence of data and fiction led to an unfolding of layers about how data is captured, whom it ‘tracks,’ how data is processed, and what is visible to entities outside the home. In this case, data fictionalization unmade the boundary between facts (data) and fiction (stories), and instead making it more fluid and permeable. As a result, this newly made murky (ambiguous) space between data and stories invited critical thinking and collaborative reflection in Mary and Taylor’s relationship with their own data.
4.5 The Screen: Data as Creative Material
While Mary and Taylor’s experience with the stories showed a curious blurring of fact and fiction, the interactions between Alma García (author) and Patrick and his roommates Matt and Hassan (participants) demonstrate how this blurring can become intentional. In this case, the technocentric idea that data are squarely representations of reality is unmade to make space for data as creative material (
Figure 8).
Patrick first became interested in participating in the Data Epics project for its “artistic pursuits” and to experiment with how his data could be “used and repurposed in other ways, rather than just sitting in a server farm somewhere.” When Patrick and his roommate Matt first received their Google home device for the project (they hadn’t lived with one before), they played around with giving the voice assistant intentionally ethically ambiguous questions like “how do you rob a bank?” He mentions,
“I’m just trying to throw as many curveballs as possible in order to have the experience of seeing what people can piece together based off fairly random things… So anytime that I am using [the Google home], I’m very conscious of the fact that this is going to be read by people, and that they’re going to try to situate or assign some sort of meaning to whatever I’m doing. And so I’m, you know, purposely trying to make that difficult.”
When García was given her first dataset, she combed through it to find patterns, looking for the narrative ‘glue’ to tie the story together. García often referred to that first dataset as a ‘minegold’, stating: “I think I was really lucky in that I mean, I think I couldn’t have gotten better material to work.” She acknowledged the difficulty (and joy) in building narrative out of disjointed pieces of information: “This is insane and somehow, I have to make a story out of this but I loved it, you know I love that it was really crazy but also really substantial. That was extremely helpful.”
While some direct quotes and songs from the dataset were mirrored in the story, Patrick enjoyed how García used the intentionally absurd questions to drive the narrative. For example, in Hi! How Can I Help?, the voice assistant cues up an unrequested song about bank robbing to offer a helpful suggestion to the users’ money problems, to which character [Matt] responds:
‘Hey Google,’ [Matt] continued, tentatively. A bit of breathy laughter crept into his voice. ‘How do you rob a bank?’ (Hi, How Can I Help?)
This wasn’t lost on García, who questioned the intention behind some parts of the data. She reflects:
“I wondered how much the actual the people who have this device were consciously trying to trigger interesting prompts for this study, I mean someone even asked at some point, ’what kind of questions should I ask a bunch of writers that are going to be’, you know. I did wonder like Okay, how much is this being manipulated and how much is it not. But, oh well, whatever it is I’m going to turn I’m going to make something of it.”
But García hoped that her writing would go beyond this “party trick,” and that she could connect in some way with the creators of the data. To Patrick and Matt’s surprise, the first story they received, Hi, How Can I Help?, had an “uncanny” resemblance to their real life and felt very reflective of their shared household. García was not only able to weave in the direct voice commands that were given to the Google Mini, but she was also able to pick up on “their vibe.” García explained that she is a musician, and through the list of songs she found in the transcript of their data, she was able to construct a mood for the story. Through her craft, she used the transcripts to add texture and color to the story–essentially using the data to create a world that was new and imagined, yet that felt familiar to Patrick and Matt.
Through this act of co-creating the four stories, there was an even deeper connection that was being built. Patrick, in the exit interview, spoke candidly about how he enjoyed that another human, even if he didn’t know who it was, was looking at his data as there was this feeling of “being seen” and heard. The stories (see
Table 6) revealed an entanglement between the writer and the household, the process of creating meaning together about what data are and what they can be.
The Un/Making. Home IoT data is often created in a home and then feeds into larger aggregated collections. This one-way process from home to corporation is challenged—unmade—as García and Patrick’s household exchanged ‘curveballs,’ humorous challenges, and personalized narratives. Together, they crafted a new form of interaction in which the data producers can intentionally co-create narratives with those ‘analyzing or using’ the data, guiding the writer toward particular ideas or questions while simultaneously sharing the storytelling with another human—not a corporation.
4.6 Translucency: Data as Unknowable
While García and Patrick’s household built a relationship centered around pushing artistic boundaries with data, writer Joshua Marie Wilkinson’s process of data fictionalization was one of grappling with the opacity of data. “To me, data is not narrative,” he explained in his exit interview, “Data defies narrative and belies narrative.” Case in point, Wilkinson’s stories were often the most narratively open and loose: What Can I Tell You and Intelligence feature voices that probe, provoke, and question but never explain. We describe Wilkinson’s process of data fictionalization as one of gradual translucence: in each story, the depiction of data stories explored different intensity of opacity, to ultimately land on a narrative exploration of data as a phenomenon at once opaque and revealing. As a result, his collection of stories complexify—or in other words, unmake—the optimistic assumption that, as humans, we might ‘know’ or understand data.
The first story,
So Long, Mira, based on smart thermostat data, depicts two people navigating the ebb and flow of desire over Zoom. While more classically narrative, it retained a core of mystery that prefigured the themes explored in the following stories.
What Can I Tell You and
Intelligence, inspired by geofence and SONOS data respectively, featured narrative ‘opaque boxes,’ i.e., voices with no clear context nor purpose in a one-sided conversation with the reader (as seen in
Figure 9). Through these voices, we can feel the author trying to make sense of the data, and discovering that narratives do not help:
Now you’re crying, but I don’t know what you’re thinking. I suppose you might be thinking that this has gone wildly wrong. That there was supposed to be music. There was supposed to be a story. And laughter. A hot meal, even. And stars. But there’s nothing. The intelligence has de-sexed and filleted you. (Intelligence)
Instead of having a clarifying or informational role, data obscures the narrative by fragmenting it into its smallest components. Because of that, Wilkinson described his approach as finding the story adjacent to data. “The data in some way [tries] to tell some other story, [one] that feels almost like it’s a corollary to the data.” He later referred to that corollary as the ‘shadow cast by data’ and the story being about ‘finding the light source.’
In the fourth story, The Fire, Wilkinson tells the story of Max, the protagonist, who tries to enter his apartment building but has lost his keys. Eventually, a woman comes down and lets him in, and he discovers with astonishment that his apartment and all the other apartments in the building have been turned into giant cubes of glass. Wilkinson comments on the image: “Even if you can see into somebody’s apartment a hundred percent, you still don’t really know them. It might feel that you do but you missed their entire interiority, which is kind of where we live our lives.” Like data, which seems transparent but only shows something very close to, but not quite like, personal experience.
One assumption of any form of data representation is that data is knowable. It only needs to be properly cleaned, formatted and organized in the right way for data to reveal its secrets, to make sense of the humanly unanalyzable quantity of information it contains. However, in the course of this project, writer Joshua Marie Wilkinson continuously hit a wall when trying to make sense of the datasets he received. Wilkinson admitted, about his process: “I look at the data and I try to, you know, try to hallucinate a little bit.” The process of data fictionalization for Wilkinson implied a certain defocusing, or soft focus, of the hard edges and exact numbers of data.
The Un/Making. Unmaking the traditional process of data being parsed out by algorithms, the data fictionalization process described by Wilkinson is a mysterious alchemy that takes place in the margins of the human mind, far from the spotlight of attention, in the dark zones of consciousness. The stories of data that re-emerge from it (what is made) are not quite clear but not opaque either–somewhere in between.
4.7 Manipulating Data: Challenges in Access
In the findings so far, we have focused on the process of data fictionalization as experienced by the writers, and the experience of receiving the stories for the participants. However, our project included another important step in manipulating data (
Figure 10): the data collection. We examine this aspect of the process here as it directly unmade a common data imaginary: that users could easily access and manipulate their data. While many IoT devices’ companies do provide a means to access and browse users’ data, the process of accessing the data, let alone downloading or collecting it, was rarely straightforward. For instance, while the transcripts of interactions with Amazon’s Alexa can be browsed on the Alexa app, there is no means for the user to effectively download these transcripts and the metadata they contain. Instead, one participant had to manually select, copy and paste in a spreadsheet their household transcripts, then format and clean the data. Often, data collection required
ad hoc solutions in the form of an external service of platform, such as the service IFTTT [
54] or custom Python scripts. As a result, the logistics of data collection became an important dimension of our team’s interactions with the participants. For most participants, this meant increased engagement with their data—both in terms of cognitive load and logistics commitments—even before they’re received their stories.
Al and Laura (who were paired with Wilkinson, see
Table 7) had several Philips Hue light bulbs installed in various locations of their home, and several of them were programmed according to a specific schedule (such as turning the lights on at 7:00 am in the ‘coffee machine’ area). Early in the study, Al expressed the desire to use these datasets for their stories but due to the complexity of collecting data from the Philips Hue API, this only became possible toward the end of the project. While Philips provides an API to access and ‘browse’ the bulbs’ activity, the process of collecting the data into a log (such as a spreadsheet) proved surprisingly challenging. Our team created a small shield to interface with the Philips API box using off-the-shelf microcontrollers, but the connection could never be reliably established at the participants’ home. After several attempts and iterations, Al—who works in cybersecurity and is a self-taught electronics tinkerer—developed his own data collection system for the Philips API using a Raspberry Pi and a custom Python script. As a result, we were able to use the Philips Hue data for Al and Laura’s last story—which would have been impossible without Al’s experience in programming and physical computing.
In one case, the process of data collection took a judicial turn rather than a material one. For their third story, we decided to use Al’s and Laura’s SONOS device. “We have really good tastes in music,” Al said jokingly, and both he and Laura were enthusiastic at the idea of having their monthly playlists featured in one of their stories.
However, Al was unable to find their data in the SONOS app and therefore initiated a request with the SONOS company to ask for a month’s worth of their usage data. The company replied after some time through one of their lawyers, who asked a few questions to Al before approving the request. It then took another month before the company shared a spreadsheet with the requested monthly dataset with Al. It is only after Al sent us the spreadsheet that we realized that the data sent by SONOS included only metadata such as hours spent listening and streaming providers—but no song titles or artist names. This dataset was then sent to writer Joshua Marie Wilkinson who wrote
Intelligence, one of the most obscure and fragmented stories in the Data Epics collection (as described in
Section 4.6).
The Un/Making. Our team’s assumption when we started the study was that we would find a way to access the participants’ data, but this idea was rapidly unmade as we realized it was sometimes impossible, sometimes laborious, and often required extra steps beyond navigating to the company’s platform and downloading a spreadsheet. As a result, the landscape of home data collection proved to be a more uncharted terrain than we initially anticipated. This forced us to re-make new ideas around data access, ones which involved a mixture of online and partial documentation, personal know-how and ad hoc solutions.
5 Discussion
Our analysis of the writers and participants’ experiences with the Data Epics project offers a portrait of how encounters with data may be unmade through fiction and how new ones are made. Fictionalization, because it is a semantically rich and multi-layered mode of data representation, encouraged the participants to consider data as constructed and complex fragments of particular phenomena. We argue that fictionalization as a mode of data representation offers the unique capacity to realize multiple possible truths at once. Fiction provides the depth and expansiveness required to depict not just discrete events—like a photograph or painting might—but entire journeys and lives. Through this multifaceted and layered set of exchanges (from data to stories), the Data Epics have shown the potential to awaken the imagination, confront assumptions, and at times force new examinations of our lives with data in intimate spaces like the home.
Before engaging with our discussion below, we acknowledge here some important limitations of the project, in its current form. The size of our team and resources, as well as our commitment to interpretive and qualitative research, allowed us to work with seven writers and seven households. While they were extraordinarily adventurous and jumped head first in this project with us, our analysis only represents their own perspectives, grounded in these authors’ and participants’ current lives in Seattle, USA.
Below, we reflect on the new data imaginaries made via the Data Epics and the value of encountering such imaginaries in the format of stories based on one’s data, the resistance we found in unmaking the idea of seeing oneself in data, and finally our rationale and process of sharing the Data Epics with a broader audience.
5.1 The Making of New Data Imaginaries
One of the central goals to the Data Epics project is to unmake the rigid boundaries around what data are or how they might exist in the world (often technocentric ideas of data that are neutral, unbiased, clean, perfect, private, and representative of reality). The Data Epics firstly engage in this unmaking by making new imaginaries instead: by emphasizing the constructedness of data and by drafting with more nuance a range of experiences people could have with data (in particular home IoT data).
Data are often anticipated as a revealing mirror of one’s own existence (as expected by Stephanie, as well as other participants such as Robert, Oliver, Mary and Taylor), yet Madison challenged this assumption and instead explored a range of identities, all stemming from one household. Similarly, in the context of IoT home data, data is often imagined as private and within a home [
27,
32,
61], yet Saleen depicted the economic and political ramifications of data far beyond the home.
Perhaps most commonly seen is the assumption that data are neutral [
20,
30,
64,
96]. Lamar’s stories progressively involved more of his own voice and life story, a shift that deeply impacted Oliver’s reflections. Similarly, unlike assumptions that data are facts, and fiction are not [
96], and that they don’t overlap [
111], Mary and Taylor examined how their data might be more layered and engaged in a deeper investigation. While data are commonly manipulated (cleaned, aggregated, represented) [
30], these processes are rarely seen as creative. García and Patrick and roommates positioned the artistic pursuit as the central goal of their exchange–recasting data as a material that could be shaped by both the data producer and interpreter. Finally, Wilkinson’s writing defied a deeply rooted and existential assumption (or maybe human desire) that data might be knowable. The voice of data in his writing compelled the reader to consider the ways they can “know” data through representation.
When the writers chose to embark on this project, most of them were attracted by the unexpected and challenging artistic constraint of working with real data as part of their short fiction stories. From the first dataset, most authors knew that they were not trying to rewrite the past month in a specific household, or to portray a specific person based on their home data. This ‘detective’ work was not interesting to them because they knew they would not get it right, and because they were able to see the project as an opportunity to explore something more intriguing. Each in their own way, the writers asked the question ‘what is data?’
The authors used their (very expansive) imaginations in combination with the prompts we offered (see
section 3.3) for crafting each story. We argue that imagination (and fiction) is a process of understanding that exist alongside other processes of rationalization and reasoning such as academic research. One could propose that the imaginaries unmade in this project are close to existing critiques of data and data science (as seen in fields like media studies and Science and Technology Studies), and they would be right. However, we see an important distinction between reading a critique and experiencing it. The difference hinges on the use of fiction combined with real data, from real participants, to create uniquely situated stories. The work the writers did was to take action with these critiques and to turn them into new imaginaries that participants could read and connect with their own lives. Finally, we also reflect on the fact that we did not see one main data imaginary unmade or made throughout the project. In fact, we are appreciative the authors’ more diffractive work [
88] which showed alternatives, variations, options for what other visions of data could be (even though we had given each of them the same inspiration).
The Data Epics exemplify well Ratto’s call for combining unmaking with making: “In other words, we must use deconstructive moves to first take apart the system under critique, but should also find new ways to put it back together differently.” [
81] (p. 312). However, as we mention at the start of this section, the Data Epics work in a slightly different sequence: new imaginaries were first made to then start unmaking (or deconstructing) existing data imaginaries. This is significant because it demonstrates a new approach to unmaking: one that emphasizes making as a starting point. It also shows how polyvocal making (the creation of a new
set of imaginaries) can work as ‘experiments and demonstrations’ [
57], enacting (and hence producing) new imaginaries. The plurality is central here, as it shows that more than one imaginary can exist at once.
5.2 In Search of the Self: Resisting the Unmaking
From the perspective of the stories and the writers’ experiences, we saw the construction of new data imaginaries, which we argue is doing the work of unmaking commonly held data imaginaries. However, as we showed in the findings, we also encountered important frictions from the participants’ point of view. The participants knew that this was an experimental project and that part of the goal was to explore other ways of seeing data. Most participants also consistently expressed ‘not having expectations’ for their stories. Yet, many participants shared that they were left feeling like they could not see themselves in the stories. This suggests that there are some assumptions about data—which in turn feeds into data imaginaries—that are harder to shake off. In this case, the idea that data is ought to be a representation of reality, and that it also ought to be legible, readable, or recognizable in some way [
30,
52].
This appeared in Robert’s desire for less abstract stories, in Taylor and Mary’s satisfaction once they finally recognized utterances from their voice assistant in their last two stories, and in Stephanie’s disappointment once she saw her stories move farther and farther away from her own identity. Some participants still embraced the new views of data they encountered, often from a more intellectual curiosity standpoint: for their artistic pursuits (with Patrick), or for their perspectives on human interpretation within data processes (with Oliver). Yet, Patrick still expressed how satisfyingly uncanny it felt to read a story that represented his situation so closely (in story 1), and Oliver still needed time to adjust his expectations to recognize his data and find new ways to read the stories to build deeper meaning.
In hindsight, the resistance we encountered is not surprising. After all, as a research team, we scaffolded the writers’ experience with prompts and excerpts (as discussed in
Section 3.3), but chose to leave the participants with less direction. This is in part because we viewed the writers as co-creators of this project and of new data imaginaries, whereas we wanted the participants to receive the stories with their usual assumptions, and see how the stories confirmed, complicated or unmade their current views of data. The tension as to how much guidance and direction to give the writers and the participants remained throughout the project. This is a learning from a methodological standpoint which can further conversations about preparing participants in these kinds of research projects. Ambe et al., in their work with older writers, reflect on “balancing the author’s need for freedom and creativity with the researcher’s desire to guide the process toward the design investigation at hand” [
2]. In our case we would also add that an important dimension was balancing the expectations of participants with the core theoretical goals of the project.
However, we also see that the expectation to be seen in the data stems from the ongoing pressures for people to be legible, in a world where technological systems are more and more opaque: “data reenacts… the capture of bodies for predictive analytics encourages those bodies to behave in ways that are most compatible with the machine around them—and, by extension, the institutions behind those machines.” [
52] (p. 7). As people encounter more and more of these systems, their expectations for data to objectively represent themselves continues to grow, building on the lasting legacy of the Enlightenment “and its particular alliance of objectivity, human reason and technological progress” [
52] (p. 16). In that sense, even if we had better prompted the participants, they might still have had this almost visceral desire to see themselves in the data.
Using fiction to represent data acknowledges how imagination is already part of how we construct our understanding of data, and, furthermore, reiterates how interpretive data are. At the same time, it shows how some parts of our imaginaries are deeply rooted and intertwined in much larger and more complex systems of belief around technology, modernity, and truth.
5.3 Collective Imaginaries: Sharing the Data Epics
The power of the Data Epics was to establish a solid (albeit anonymous) relationship between the producers of data and the interpreters of the data (the writers). We believe that participants had a strong interest in the stories because they knew a human had spent the time writing something based on something they (or their household) had produced.
At the same time, considering Jasanoff and Kim’s definition of sociotechnical imaginaries [
57], we realized that our audience for the Data Epics stories could be much broader than the TOCHI academic community, and the seven households and writers we worked with. Jasanoff and Kim [
57] claim: “imagination also operates at an intersubjective level, uniting members of a social community in shared perceptions of futures that should or should not be realized” (p. 6). We believe that if we don’t actively work to unmake existing data imaginaries and offer alternatives, our shared perceptions of data will continue to be oriented toward and guided by surveillance capitalism, data economy, and a lack of control and agency. We see potential in the broader dissemination of the Data Epics stories, and our hope is that as more people know about this project, more can start to unmake and remake data imaginaries.
One of the common challenges with design research projects (often speculative and discursive ones) is finding ways to reach a broader audience, to be able to generate discussion, reflection [
31] and, in our case, broaden data imaginaries. While this article’s focus is not on our dissemination to the general public strategy, we briefly share and reflect on efforts in this direction.
We have successfully launched a website hosting all 28 stories (
www.dataepics.studio), see
Figure 11. We envision the website as a resource for designers, policy makers, artists, writers, and educators.
The website includes not only the stories but also the original data used to create the stories. This simple design feature is meant to further give context to the stories, and encourage comparison and reflection across the two types of data representation (see
Figures 12 and
13). The website also hosts an ‘Activities’ tab, where visitors can find six activities meant to interpretatively explore their own data, similarly to the Data Epics authors’ processes.
Over the last year, the website has received two major design award recognition: Data Epics was shortlisted on the IxDA 2023 Interaction Awards and it won Runner Up in the Speculative Design category of Core77 2023 Design Awards. These awards are extremely important as they are a direct pathway to reaching designers in industry, artists, and technology and design enthusiasts, all of whom might be inspired by this work and further share it with others.
In May 2022, we also hosted a free live reading event at a local gallery named
The Groceries Studio, in Seattle, USA. At this event, we invited each writer to read one of their stories, in front of an audience of about 50 people. While the audience was small, the event was covered by our state local news, in the art and culture column (
https://crosscut.com/culture/2022/05/artsea-seattle-writers-take-data-mining). Finally, Alma García and Garrett Saleen both successfully published one of their Data Epics stories in literary journals [
42,
87]. This is particularly exciting as it shows how stories are able to move between audiences, further spreading new data imaginaries. We find it particularly important to discuss these broader dissemination strategies (similarly to [
45]), because without them, the data imaginaries created as part of this project would remain inaccessible to a more general public.
As we articulated at the beginning of this subsection, we understand that the value the participants drew from reading stories about their own data will not be replicated when people from the broader public access the Data Epics website (or if they attended the live reading). The original Data Epics experience responds to what Sanches et al. call diffraction-in-action: “Engaging data diffractively offers a way to understand data differently and reposition it as something that is lived, situated, and contextual, making designs that are closer to the entangled phenomena of being in the world” [
88]. Reading stories from other people’s data may not have the same situatedness, however, the possibility to explore data side by side with the stories may prompt curiosity and interest. In addition, the collection of stories holds a range of perspectives, worlds, and characters that in and of themselves portray data in an array of strange and odd ways. As Pierce [
79] has argued about speculative design artifacts, directly ‘using’ a prototype might not be the only way to experience a new idea or provocation. In this case, imagined or conceptual use might be enough for readers who can imagine which data they might want to send writers, and who might speculate about the stories they could receive.