How the Alt Text Gets Made: What Roles and Processes of Alt Text Creation Can Teach Us About Inclusive Imagery

As of 2014, the Census Bureau estimated that almost 30% of the U.S. population—over 85 million people—had a disability [Taylor 2018]. Yet, people with disabilities are underrepresented across many forms of mainstream media. For example, a survey of top-grossing movies found that out of over 4,000 named characters or characters with speaking roles, only 1.6% had a disability [Smith et al. 2020]. Similarly, advertisements rarely include disability, and when they do, they depict people with disabilities in highly stereotyped ways [Davis 2013]. As disabilities become more recognized by mainstream, non-disabled content creation structures, representations of disability are becoming more common. This comes with increased responsibility to people with disabilities to make sure the media is itself accessible. If disabilities are represented in images, for example, it is even more important to make sure those images have alt text. It also means the representations of the disability themselves must not be stereotypical or reductive. Thus, inclusive imagery must both depict diversity, and do so in respectful ways.

Within Media Studies and Disability Studies, there are discussions of how people with disabilities are written about in print news and depicted in advertisements [Haller 2010; Riley 2005] and how fictional characters with disabilities are portrayed in film, television, and literature [Davis 2013; Garland-Thomson 2017]. Within discussions of describing disabilities textually in the most inclusive way possible, there are a variety of resources for both general audiences (e.g., Wikipedia [2023]; APA [2022]) and HCI-specific audiences (e.g., Babinszki et al. [2019]; Hanson et al. [2015]). Existing studies of representations of disability in media largely agree on two points. First, representations of disability have historically been limited [Davis 2013]. Secondly, what representations of disability do exist in media are often one-dimensional, stereotypical, patronizing, or otherwise offensive [Briant et al. 2013; Ellis et al. 2013; Houston 2019]. Structural ableism not only causes the marginalization of disabled people in the workplace and educational sphere, but also leads to ableist depictions of disability in media. There are however increasing examples of disability activists increasing the visibility of or adding depth to conversations around depictions of disability, for example in the documentary Crip Camp [Newnham and Lebrecht 2020], videos by Haben Girma [Girma 2023], or in podcasts like Reid My Mind [Reid 2023]. The important topic of making sure media depicting disability is also itself accessible has been gaining more attention as well. Researchers have studied multimedia accessibility either in the form of professionally authored subtitles or audio descriptions for movies and TV [Duarte and Fonseca 2019], or multimedia approaches to making user generated content accessible (e.g., Kim et al. [2020]; dos Santos Marques et al. [2017]).

Screen reader users require digital images to be rendered into text so that it can be accessed via synthesized voice or Braille display. Image accessibility is therefore the process of creating and associating textual descriptions of visual information with the digital images they refer to. There are several terms used for different types of text related to image accessibility [Morris et al. 2018]—including alt text, image descriptions, long descriptions, and captions. These terms may overlap in some cases or have clear delineations based on differing definitions or familiarity with image accessibility. For the sake of this paper, we use the terms image description and alt text interchangeably.

Image accessibility is an important aspect of overall digital accessibility and is listed first in W3C's list of website accessibility requirements [W3C 2019]. The prevalence of accessible images has overall improved in the past decade, depending on sites surveyed. Image description prevalence on popular websites increased from less than 40% in 2006 to 72% in 2018 [Bigham et al. 2006; Guinness et al. 2018]. Although, some estimates put the number of images lacking alt text as of 2022 at closer to 55% [WebAIM 2022]. And in some cases prevalence is much lower, with sites primarily comprised of user generated content such as Twitter found to have alt text on only 0.1% of images [Gleason et al. 2020]. This leads to a large gap between the experiences and information available to sighted versus screen reader users [Aizpurua et al. 2016], with many screen reader users lacking important visual information on a variety of topics, while sighted internet users often are not even aware of image accessibility needs [Seixas Pereira et al. 2022]. To address this pervasive accessibility issue, the Accessible Computing field has explored many potential solutions, ranging from fully automated to highly social methods for creating and propagating image descriptions. Several approaches leverage computer vision or other automatic methods for automatically labelling images (e.g., Carvalho and Freitas [2015]; Guinness et al. [2018]; Gupta and Mannem [2012]; Wu et al. [2017]). While these deliver scalability, research shows that automatic generation methods are not as high quality as human-generated descriptions [Bennett et al. 2021; Gleason et al. 2020; MacLeod et al. 2017], making automatic approaches unideal depending on circumstances. Human-generated image description interventions have included studies of crowdsourcing [Bigham et al. 2010; Brady et al. 2013a; Salisbury et al. 2017], friendsourcing [Brady et al. 2013b; Mathur and Brady 2018], and social microvolunteering [Brady et al. 2015]. The primary concerns with human-generated methods are training writers properly to produce high quality alt text and dealing with lag times between requests for information and responses. Some researchers have attempted to combine multiple methods to achieve high quality results [Gleason et al. 2020].

To aid image description creation, there are numerous guidelines detailing how to write image descriptions. Some guidelines are based on feedback from BLV and/or screen reader users [Bennett et al. 2021; MacLeod et al. 2017; Petrie et al. 2005; Power 2012; Stangl et al. 2020]. There are also guidelines from a variety of non-computing and non-academic sources, including museums [Cooper Hewitt 2019], non-profit accessibility initiatives [DIAGRAM Center 2019], and individuals [Chen 2020; Lewis 2018]. Some of these guidelines deal with issues around describing illustrations [DIAGRAM Center 2019] or the identities of photograph subjects [Chen 2020], but none discuss artistic depictions of disability in depth [Cooper Hewitt 2019] or depictions of users in design contexts, both of which are relevant to The Avatar Project images that we conducted our user evaluation on.

Accessibility researchers have in recent years led investigations into the complexities of writing image descriptions for screen reader users in different contexts [Stangl et al. 2020] and with different marginalized identities [Bennett et al. 2021]. Stangl et al. [2020] noted that image descriptions must be tailored to different contexts of use. They also indicated that depending on the purpose behind posting, the image descriptions of subjects’ identities may need to be adjusted. Bennett et al. [2021] focused on the problem of describing marginalized identities in their study of screen reader users who also had additional marginalized identities. They investigated the ways their participants used non-visual methods for understanding identities of people online and the ways they went about describing themselves in images they posted. Our paper similarly includes interviews with people with disabilities on the topic of describing identity in images, however we define our alt text consumers as disabled users who may be represented in the images rather than only screen reader users. We also contrast the in-depth research process to create image descriptions with image accessibility processes used by practitioners in industrial contexts.

The gap between HCI researcher communities and HCI practitioners has been investigated from many different angles (e.g., Buie et al. [2010]; Colusso et al. [2017]; Dalsgaard and Dindler [2014]; Gray et al. [2014]). Within the larger body of literature on HCI practitioners, there are an increasing number of studies looking at accessibility in practice. Some of these involve evaluating existing guidelines and accessibility evaluations for the challenges they pose to designers and developers who need to put them into practice (e.g., Colwell and Petrie [2001]; Law et al. [2006]; Steen-Hansen and Fagernes [2016]). Evaluations of existing guidelines often conclude that considering designers and developers are the “users” of the guidelines—that is the people who need to understand and put into practice the information contained in the guidelines—their pragmatic needs and perspectives are not properly considered in the design of accessibility guidelines. This has led to several studies focused on developing tools to bridge the gap between accessibility guidelines or evaluations and the practitioners who have to instrumentalize them (e.g., Gaggi and Pederiva [2021]; Scott et al. [2015]). Other studies focus on pedagogical considerations for students and early career HCI professionals and how they learn or understand accessibility (e.g., Carter and Fourne [2007]; Gray [2014]). There are many studies that directly consult industrial professionals to understand their approach to or understanding of accessibility, however almost all of them are survey-based [Freire et al. 2008; Inal et al. 2020; Durdu and Yerlikaya, 2020; Porter and Kientz 2013; Putnam et al. 2012; Trewin et al. 2010; Yesilada et al. 2012]. Interview studies or qualitative investigations of accessibility in practice are few and far between (exceptions include [Azenkot et al. 2021; Giannoumis and Nordli 2020; Vanderheiden and Tobias 2000]).

Among studies of accessibility, few research papers have focused on how image accessibility is enacted as part of industrial accessibility practices. Giannoumis and Nordli briefly describe the basic knowledge of image accessibility their participants demonstrated and how it was requested and delegated in the context of their case study of a Norwegian broadcasting company [Giannoumis and Nordli 2020]. Beyond industry contexts, most studies of image accessibility focus on interventions—either tools or new processes—into alt text creation, rather than investigating the existing practices in place. Few studies have discussed the current processes that users, researchers or disabled people undergo to create alt text (exceptions include [Bennett et al. 2021; Gleason et al. 2019; Mathur and Brady 2018; Williams et al. 2022]). Notably, [Williams et al. 2022] describe the workflows that computing researchers used when creating alt text for their publications, noting issues with complicated figures, tight deadlines, and technical tools used to insert alt text. Given the small body of research discussing alt text creation processes and the even smaller overlap between studies of industrial accessibility and image accessibility in practice, our paper seeks to contribute descriptions of four initial image accessibility creation models connected to industrial accessibility practices and inclusive imagery.

The Avatar Project is a collaboration between scholars at University of California, Irvine and industrial researchers and designers at Google. The intention of the images was to function as reusable brand-compliant components that could be used by all designers (including blind designers and developers who often lack accessible tools [Storer et al. 2021]) to represent users in company contexts. They were created to help designers and engineers “think more critically about inclusivity, diversity and representation” [Mac 2020] and so were to be included in the company's design system used by designers and developers for internal documents such as mockups and personas. They would also be viewed by users in user testing environments and external advertisements. There have been four total artists commissioned to produce artwork for the Avatar Project, with the size, style, and content of the different artists’ contributed sets varying, despite all focusing on the basic goal of producing inclusive imagery.

Our collaboration began when the academic researchers were consulted for their expertise in accessibility. Prior to that, the industry authors had held a series of workshops with other designers at Google to discuss and brainstorm how to properly represent a more diverse set of users, including users with disabilities, in various design and marketing assets. The search for inclusive imagery had begun with using photographs of people with disabilities, who had given permission for their image to be used in such documents. However, concerns arose around branding and ethical implications of having photographs of real people used in contexts beyond their direct control. Thus, the three original artists were commissioned to create a diverse set of illustrations of fictional users. The fourth artist was commissioned later as the project developed.

Study 1 involved a subset of images from one of the Avatar Project artists. That artist's collection as a whole included representations of fictionalized people of many ages, races, genders, ethnicities, professions, sexualities, disabilities, and other characteristics. Across all Avatar Project sets were also non-human and abstract representations–depicting ideas, emotions, or concepts related to identity. Six of the images assessed in Study 1 included representations of visible disabilities (e.g., a skateboarder with a prosthetic arm) or assistive aids (e.g., a guide dog).

Our original interviews sought to answer the user side of the overall research question: how are accessible and inclusive images created and how are their attributes negotiated by various actors, including image creators, alt text creators, and end users? To understand this aspect of the question, we conducted nine focus groups and nineteen interviews, with 25 people who self-identified as having one or more disabilities. The study was reviewed and approved by the Institutional Review Board of the first author's university. Focus groups and interviews were conducted remotely via phone, Zoom, or Skype, and participants were compensated at a rate of $20 per hour. All sessions were video and audio recorded after obtaining written or verbal consent from participants. The first author attended all focus groups and interviews, usually with one or more additional researchers present.

Study 1 feedback from users led overall to longer alt text (as seen in the examples shown in Table 3) partially due to the nature of co-designing through consensus, but primarily because participants believed that detailed descriptions of the images were warranted given their intended uses. In order for the images to properly invoke reflections on inclusivity as well as educate primarily non-disabled designers and developers, participants argued that these image descriptions should be highly detailed. This was part of the many areas of feedback we received from the Study 1 sessions, including comments on the sub-par nature of automatic alt text, detailed feedback about clarity of wording, and discussions on the tradeoff of including many visual details for low vision users versus making the image descriptions relevant for those who had no sighted frame of reference for things like colors of skin tone. These data were important for our overall alt text recommendations, in combination with the findings related to the role of intent, interpretation, and ableism in disability representation through alt text.

First, similar to the alignment between educational potential and level of detail, participants felt that the purpose or intent behind the image was important when deciding when and how to describe identities in image descriptions. Users located the origin of an image's purpose in the goals of the artist who created the fictionalized representations. If an artist intended to depict identity in a specific way then the alt text should reflect the intended type of representation and identity. Users also felt that context, including the intended audience and the use cases for the image, should shape the final alt text. Alt text describers, they felt, should consider the imagined readers’ familiarity with the depicted identity when deciding how much detail to include in the image description. If, for example, people without disabilities are the main audience and the images are being used for an educational purpose, then explicitly describing disability may be important in a way it is not important if images are presented to a familiar audience.

Describing disability was particularly difficult for users to create solid rules around. Their personal relationships with their disabilities varied, and thus while they emphasized that accuracy in alt text was important, when alt text writers described disability it was hard for users to define how to accurately describe disability without assigning identity labels to fictionalized representations that could not claim them. A third of users we spoke to discussed difficulties labeling fictional subjects’ disabilities and approved of describing assistive aids instead, to avoid the fraught issue entirely. Alt text writers were cautioned to avoid describing identities for fictional subjects that could not claim them personally, and to balance this linguistic ambiguity with the level of conciseness needed for image descriptions.

Most importantly, users we spoke to understood image descriptions as sites of contested meaning. They argued that the subjective experience of alt text writers affected the final alt text and that if unchecked the conscious or unconscious biases of the image describers could have deep impacts on what a screen reader user understood about the final representation. Users also felt that a large part of the image description process was not just encoding certain biases or points of view into the alt text, but also the process by which alt text readers interpreted the description through their own lens. Essentially, users we spoke to described the process of creating alt text as a progressive layering of interpretations (Figure 1). First, the artist interpreted the basic identity through their perspective and goals, then the alt text writer interpreted the image with their own biases when creating the alt text, and finally the alt text reader interpreted the image description (alongside the image, if sighted) and decided on their final understanding of the identity representation present.

Additionally, users discussed the way image descriptions were part of a “very vision-centric” world (U10) where invisible disabilities were not considered and the gold standard of image descriptions were assumed to be the complete translation of the visual elements. This reinforced a type of ocularcentrism where BLV experiences were assumed to be inferior substitutes for sighted understanding.

Before we present the four different models of alt text creation, we begin by describing the general context in which the models our interviewees discussed were situated. The accessibility practitioners we spoke to had varying degrees of familiarity with organizational approaches to accessibility outside the specific projects and teams they had worked on. According to AP2, Google's approach to accessibility “[is] on multiple levels… There's a bottom-up approach; there is a bit of a top-to-bottom approach.” This top-to-bottom and bottom-up approach was demonstrated in the alt text creation processes as well. AP1 described the organization-level requirements for alt text passed down to product teams: “In any product launch, you have the accessibility review and a legal review, amongst others like engineering code reviews… [so] you have a [Quality Assurance] tester—accessibility tester—then going through the content [to assure image descriptions] make sense in the context of the task that they're trying to accomplish in order to test it.” In addition to these top-down processes, more bottom-up resources were used. For example, AP1 and AP3 both discussed being aware of or using a variety of alt text guidelines existing in different forms and formats across teams. The manifold nature of the organization–as a large tech company with many teams and divisions–required a similarly manifold approach to accessibility and alt text generation in particular, demonstrated in the differing methods that accessibility practitioners described.

The role of individual interest and advocacy as a complement to existing organizational structures was also an ongoing theme in discussions of accessibility practices with the practitioners. AP2 pointed out there were both formal and informal structures to support accessibility, with many individuals–particularly engineers–having their personal “curiosity” supported by provided training and networking opportunities: “Notwithstanding the fact that people can just connect with each other [through] bug bashes and workshops and events… there's plenty of opportunities to learn about accessibility.” As we will see in the next section, personal passion and dedication among accessibility practitioners was particularly valuable in some alt text generation methods such as the Lone Writer process (Section 5.1.2). This overarching context for the four alt text creation methods also highlights the importance of organizational support and formal policies in enabling robust alt text creation.

In this section, we describe three additional themes from the interviews with image and alt text creators that ultimately shape the overall process of inclusive imagery creation in the studied company. First, we describe the way artists and accessibility practitioners fulfilled the expectations users in Study 1 had by considering the intended use case when creating imagery and alt text. Next, we describe how the alt text creation processes we have just described were impacted by the macro pragmatic and micro personal considerations of different actors, including the organization. Lastly, we describe how inclusive imagery as it was done at Google had clear champions for the words and the images but no set role to champion the alt text, leaving a potentially fillable gap in future alt text creation processes.

Our findings from Study 1 complicated the guidance suggested by Bennett et al.’s study, which focused on self-presentation in photography and argued that the subject of a photograph should be the final authority on how to describe their identities [Bennett et al. 2021]. We found that users saw fictionalized representations as defined by the intent of the image author who, alongside the users themselves, determined what the final received interpretation of disability identity was. Our study of industrial practices and inclusive imagery production processes complicated this further by exploring the more elaborate relationship that artists played in conveying intent through images.

Our findings originally suggested at least three layers at work in the final reception of an image description. First, an artist renders a fictional interpretation of disability or any other social identity. Then, the image describer interprets the fictional subject into text. Finally, the reader of the description interprets the text. Our user interviews revealed that the identities of the “interpreters” at each stage of the translation layered on their own biases, assumptions, or perceived meanings. However, artists are not actually alone in creating a representation of disability or other social identities. In fact, at least in the case of the Avatar Project, the artists were collaborators attempting to stay faithful to the inclusive intent of the imagery project as defined by their client, Google. Character identities originated from a combination of that foundational intent and the artists’ own personal experiences and inspirations. Even artists themselves did not always intend a specific identity to be depicted, but sometimes intentionally depicted figures ambiguously. And this imagery, no matter how well-conveyed or accurate, is often running in parallel to the main written content of the webpage, product, or app. Thus, the final reader of the alt text is basing their understanding of the representation on a gestalt of main written text and the layers of interpretation and intent at play in creating the accessible imagery (Figure 4).

Beyond strengthening the model of alt text creation we proposed based only on Study 1, our initial analysis of alt text production at Google point towards some other interesting implications for image accessibility as a field of study and practice. The four creation processes we described highlight different areas where improvement in image accessibility is warranted.