A Design Space for Intelligent and Interactive Writing Assistants

To understand the current research literature on writing assistants, we perform a systematic literature review. We first identified fields closely related to intelligent and interactive writing assistants. We selected the fields of HCI and NLP as our core fields¹ and listed their relevant associated venues in Appendix D.1. From the ACM Digital Library ACL Anthology 3.3 for details). Then, three of the authors filtered candidates that fall under the scope of the project (Section 3.1). These authors first independently annotated the relevance of candidate papers, discussed disagreements, and then refined the scope. As a result, we selected 115 out of 419 papers that fulfilled the inclusion criteria for our literature review.

Note that the above sampling method targets a specific set of existing writing assistants, as opposed to all possible options of writing contexts, user groups, and technologies for future writing assistants. Concretely, papers under our review explicitly include a specific technology, a human user, and interaction between the user and the technology through an interface in the context of writing. If a paper does not present a strong connection to all of these components, we exclude it from the review. For instance, we exclude a technical paper that rewrites content in a target style even if it can be used in writing assistants (i.e., it does not explicitly consider user and interaction aspects, but studies style transfer in isolation). In the following section, we describe how we complement the strict exclusion of these works by leveraging the expertise of the authors by adding flexibility to the coding process.

With the 115 papers, the authors were split into five teams based on the five aspects (Section 3.2) to develop codes for each aspect. Then, every team sub-selected the papers that were relevant to their aspect. For instance, some papers might allude to potential users but lack descriptions of exact target users; the team focusing on the user aspect (henceforth “the user team”) excluded those papers from their consideration. At least two authors from each team read each paper to decide the relevance while resolving any disagreement through discussions.

To go beyond the understanding of existing writing assistants and think about possibilities for future writing assistants, we intentionally allowed flexibility in the code development process in the following ways. First, while some teams (task and user) derived the initial codes mostly from their selected papers (i.e., inductive coding), other teams (technology, interaction, and ecosystem) created the initial codes based on external knowledge (i.e., deductive coding). Second, to provide insights on recent advancements in technology that are not yet leveraged for writing assistants, the technology team explicitly brought in 25 papers (outside of the 115 papers) based on relevance and importance (see Appendix D.4 for more details) and referenced them while iteratively refining the codes. Third, most teams considered relevant works that are not necessarily about writing assistants (e.g., papers solely about LMs) to inform their codes. Note that we used these external references to aid our code development but did not include them as part of our coding process.

With the subset of 115 papers selected by each team, the authors in the team iteratively coded papers while refining the codes. The iterative coding process began with the distribution of papers to members of each team. We distributed papers so that each paper was read by at least two members, while maximizing the number of combinations of readers to facilitate discussion among team members. With distributed papers, each team member first independently read and coded papers with the up-to-date version of the codes. Then, team members had a discussion session to share and agree on how they coded each paper. If team members decided that a specific paper could not be adequately coded with the existing code structure, they updated the code structure either by revising, merging, splitting, adding, or removing codes. As codes were updated, team members returned to already-reviewed papers to ensure that papers were coded with the up-to-date code structure. The process of coding papers repeated until the team reviewed all papers under the final set of codes.

After each team developed their codes, all teams gathered to create the final design space by combining the dimensions and codes from all teams. During the process, we removed overlapping dimensions and codes, improved their consistency across the aspects (e.g., inclusion criteria and granularity), and revised their names and definitions for clarity. Once we finalized the design space, we repeated the iterative coding process, where each paper was read by two authors (same as the initial iterative coding in Section 3.3.2). This time, all 115 papers were coded with the full set of 35 dimensions and 143 codes from the five aspects. We briefly analyze trends and gaps based on this final coding of the papers (Section 5.2) and report two metrics for inter-coder reliability: Percent agreement (mean: 0.93, std: 0.06) and Krippendorff’s alpha (mean: 0.69, std, 0.17).² We release our coded papers as a living artifact at https://writing-assistant.github.io and encourage others to contribute by adding papers beyond what we covered in this work to track future developments in this space.

We describe dimensions and codes for the task aspect. Figure 1 (“Task”) shows task dimensions in a broad context, while Table 1 lists all dimensions, codes, and definitions.

Writing Stage:. At what point in the writing process is the task taking place? In the intricate and iterative process of writing, a writing assistant can be designed to support specific writing stages³ to provide more targeted and relevant support. For example, during the idea generation stage, writers brainstorm and develop concepts. In this stage, writing assistants can offer ideas for potential topics and content [85, 217]. In the planning phase, writers focus on organizing the structure and content outline. Writing assistants can play a crucial role in aiding writers during this phase, for instance, by reducing planning time [10, 129]. During the drafting stage, writers may require assistance with thought facilitation or initial drafts [178, 189]. The revision stage involves identifying and rectifying mistakes, making necessary corrections, and enhancing the content. Writing assistants at this stage might be used to provide feedback on various aspects of the written text [2, 249].

Writing Context:. What combination of stylistic norms, audience expectations, and domain-specific conventions characterize the approach to the task? This dimension delves into the diverse contexts of writing that shape the task’s approach based on community and domain-specific norms. The academic context is characterized by rigorous research, thorough analysis, and the formal presentation of knowledge. Within this context, writers may benefit from assistance that upholds formal practices in genres such as scientific writing [9] and theses [203]. In the creative context, the focus lies on imagination, artistic expression, and original storytelling. An intelligent writing assistant within this context can provide support for diverse creative endeavors, including writing lyrics [254], crafting metaphors [84], and offering inspiration and assistance for storytelling [185, 217]. The journalistic context centers around factual reporting, news coverage, and effective communication of information to the public. Writing assistants can play a supportive role in this process by assisting with science writing [132] and generating suggestions based on keywords [47]. In the technical context, authors convey complex information, provide instructions, and explain specialized topics, such as by writing figure captions  [186]. In the professional context, precise and formal writing is essential. Writing assistants in this context can be tailored to aid in producing reports and documents related to companies, trade, and professional work [113, 247]. In the personal context, individuals engage in private or public reflections on their thoughts, experiences, and emotions. Writing assistants tailored for this context can provide support for tasks that involve emotional expression in writing [189, 252], as well as fostering empathetic relations with readers [193].

Purpose:. What is the purpose of the written artifact? A writing task may be motivated by a broad range of writing goals that guide the writing process. One fundamental goal is to explain; expository applies to writing tasks whose purpose is to convey relevant facts and knowledge [137, 203]. Another common purpose is telling a story; narrative applies to tasks whose goal is to convey an account of real [90] and imagined experiences [88, 108, 208]. Some texts are written to be descriptive in order to convey emotion [252] and provide expressive details, particularly with evocative language [83, 84, 158]. Meanwhile, persuasive writing aims to convince or influence the audience through text [2, 120], requiring high readability [128] and concision [114]. Sometimes, the purpose of the written artifact can be to be educational (e.g., readers learning about a topic by reading the written artifact). In this context, writing assistance could assist writers in conveying educational ideas in an informative and clear manner [10, 85]. With entertainment as its purpose, a writing task can aim to engage and amuse readers through text, requiring coherent storytelling [132, 172] to engage the audience and may use the structure of the writing itself to convey surprise [80] and pleasure [144, 254]. Analytical writing provides in-depth examination, which benefits from reflection and iteration [57, 112]. Accessibility describes tasks that aim to support inclusivity; for example helping neurodivergent individuals explain and write about common social situations [133]. Likewise, second language writing can be difficult due to unfamiliar spelling and grammar rules [155] as well as vocabulary [111], and writing assistants can help by aiding in translation to ease the difficulty of writing in an unfamiliar language [111, 111, 155, 274]. Lastly, feedback describes tasks that provide evaluative comments, such as customer reviews [13, 63] and constructive responses [217].

Specificity:. How detailed are the task requirements? To answer this question, we examine design choices that add specification to the writing task. Nonspecific is applied to tasks that do not have specific objectives. Systems supporting these tasks often include generic drafting environments that bundle many functionalities [107, 144] or general-purpose systems that do not specify a particular objective [193, 208]. Tasks that provide a broad indication of the desired outcome without specifying the precise steps needed to achieve it are considered to have general direction as their specificity. These tasks may suggest the direction of an outcome through writing conventions [190], writing reflection variations [80, 133], or writing format [47, 64], but will not define how the desired outcome should be realized. On the other hand, some writing tasks have specific objectives that directly connect to the overall goal. For example, while a general direction might be a target writing format, specific objectives may suggest the inclusion of certain subsections [54, 114] or information [186] that contribute to that overall goal. Finally, tasks with detailed requirements come with detailed and measurable instructions for the task. Weber et al. [255], for example, outline a task for supporting legal writing based on the case solution’s major claim, definition, subsumption, and conclusion, as well as elements and relations in the subsumption, which are highly specific requirements about what should be done in the task.

Audience:. Who is the intended recipient of the task output? This dimension describes the audiences for whom the output of the task is intended. Specified audiences are specifically mentioned in the paper, such as the academic community [85], people on the autistic spectrum [133], people with limited vision [178], and writers themselves [12, 213]. An implied audience is inferred by the system design or characterization of the textual artifact, for example online shoppers for a tool designed to help writers review products [64], business stakeholders for a system that aids with writing introductory help requests [114], and writers themselves for a tool that assists with personal reflection [252].

Figure 1 (“User”) shows user dimensions in a broad context, while Table 2 lists all dimensions, codes, and definitions.

Demographic Profile:. What are the demographic details of the users considered? The demographic profile of users reflects a broad spectrum of their characteristics, highlighting the diversity inherent across various groups of users. Some systems ensure equal representation of minority and marginalized groups by incorporating gender and race into their design [105]. Others assist the writing process of users with diverse socioeconomic status, as it influences the availability and usage of technology, thereby impacting user experiences and needs [89]. Language and culture, reflecting users’ fluent languages and cultural backgrounds, uniquely shape their writing experiences and outcomes. A number of studies focus on non-native English writers [27, 37, 274] in English writing contexts to address challenges in non-native language writing and to enhance language learning experiences. Understanding age is important when tailoring systems to accommodate the developmental and cognitive characteristics of users across a wide age spectrum, ranging from young, pre-literate children [267] to adolescents [89, 213]. Similarly, education level and background of users, whether they are high school students  [213] or university graduates  [2, 248], indicates varying cognitive and learning competencies, which in turn may influence how users interact with and engage in systems. Finally, some studies emphasize tailoring system designs to a specific profession, such as professional creative writers [80, 172], ensuring that the systems meet the unique writing needs of different professions.

User Capabilities:. What user attributes associated with the writing process can be influenced by writing assistants? User capabilities represent a distinct category from demographic attributes, which remain largely unchanged while interacting with writing assistants; user capabilities are often targeted for improvement by writing assistants. Writing expertise captures the user’s writing proficiency or expertise (e.g., amateur vs. professional writer) in terms of writing quality [185, 186] or genre specializations (e.g., science writing vs. email writing) [85, 114]. Efficiency, or how efficiently the user can complete specific writing tasks, was also considered in many papers. Several studies have utilized metrics such as word count, time, and effort expended by the user to quantify improvements in the user’s writing efficiency before, during, and after using a writing assistant [7, 27]. Technical proficiency relates to the extent to which a user is knowledgeable about the underlying technology of a writing assistant. Understanding how a LM functions, for instance, has been shown to influence how effectively the user engages with a writing assistant [185]. Several papers in the literature focus on enhancing user capabilities related to emotional and cognitive aspects. For example, several studies capture the user’s confidence in both the writing process and the final product, possibly expressed as self-efficacy or perceived skill level [114, 259]. Creativity examines how the writing assistant can foster the user’s creative exploration or curiosity when performing a writing task, for instance, by supporting idea generation [45, 85, 265]. Emotion refers to the user’s emotional state before, during, and after interacting with the writing assistant [16, 189, 252]. Empathy focuses on the user’s ability to emotionally and cognitively empathize with others in the writing process. This empathetic focus was observed within educational contexts, where students are instructed to write more empathetic peer reviews [248, 249]. Cognition looks at cognitive aspects like the user’s focus, sense of immersion, and cognitive load, as systems can increase cognitive engagement by tackling phenomena like writer’s block [12, 45, 217]. Finally, neurodiversity encompasses considerations for users with diverse neurological profiles, such as aphasia [179] and dyslexia [71].

Relationship to System:. How do users build a mental model of the system? As users engage with a system, they gradually develop a mental model of its functioning, which subsequently shapes their interaction and engagement with the system. First, agency refers to users’ sense of control over the system or the writing process. It is typically facilitated by providing users with options to steer the model outputs, either through adjusting model parameters [47, 88] or by using customized prompts [58, 265]. The ownership or authenticity of a final product can be influenced by system design. A writer’s sense of ownership may diminish as the proportion of system-generated text increases [146], yet this issue could be mitigated by personalizing writing assistants to mimic a writer’s unique style [86], or by designing assistants with greater agency [179]. Similarly, maintaining a sense of integrity is an important factor when assisted by AI. This encompasses worries about unintentional plagiarism and the moral implications of using writing assistants [15, 85]. Trust is another critical facet of users’ perceptions, referring to their perception of the system’s capabilities and their sentiments toward the technology itself. The level of trust users hold towards AI could influence the human-AI collaborative writing experience [15, 156]. A system’s availability emerged in the context of comparing human support to computer support, where human writers (e.g., friends) are not always readily available, while computer programs are typically perceived as constantly accessible [86]. Privacy highlights users’ concerns regarding how their data is handled by a system, including a sense of surveillance over their writing process [12, 189, 259]. Lastly, users are concerned with the transparency [153] of writing assistants as they seek clarity on how systems operate [13, 193], how data is used [193], and AI’s role in these systems [13, 34, 189].

System Output Preferences:. What influences users’ perceptions of system outputs? Understanding how writers evaluate system outputs, such as writing suggestions, is crucial as it can influence their interaction and engagement with the system. One common consideration is textual coherence, which underlines the need for grammatically and contextually coherent outputs [85, 146]. Another significant dimension is textual diversity, which emphasizes the importance of offering varied system outputs to foster creativity in writing [84, 85, 226]. The explainability of the system can also influence users’ perceptions of its outputs. Providing additional information to explain the rationale behind system outputs may enhance user understanding and engagement [190]. Additionally, system-generated content may exhibit various forms of bias, ranging from skewed perspectives on topics [120, 195] to societal stereotypes [105]. Lastly, the personalization of system outputs, which involves adapting to and reflecting an individual’s unique writing style [80], may enhance the user’s writing experience.

Figure 1 (“Technology”) shows technology dimensions in a broad context, while Table 3 and 4 list all dimensions, codes, and definitions.

Data - Source:. Who is the creator of the data used to train or adapt a model? The source of the data used to develop a system or train a model can have a direct effect on the system’s overall performance and reliability. A dataset can be authored by experts who have domain knowledge of the specific downstream task [3, 128, 255, 273], or users of the system, during their interaction with the writing assistant [9, 27, 177, 252]. However, due to the difficulty of recruiting real experts and users, many researchers resort to crowdworkers to create data or annotate data entries [35, 130, 251, 272]. Sometimes, authors themselves participate in the preparation and annotation of the dataset [125, 193, 195, 269]. Recently, we see more datasets that are generated by a machine [105, 123, 196, 227], which has the advantage of being relatively cheap and fast to generate at scale compared to human-generated datasets. Finally, there are other types of creators such as non-experts, unspecified individuals, or a broad set of creators (e.g., in the case of web crawled data) [13, 225, 265, 274].

Data - Size:. What is the size of the dataset⁴ used to train or adapt a model? Depending on the size of a dataset required to train or adapt (e.g., fine-tuning or prompting) a model, there can be a huge overhead in terms of data collection. While some models can be developed using very small data (between 1 to 100 examples) [10, 85, 156], the others require much larger data. If the training needs more data (around 100 to few thousands of examples) which is often the case for fine-tuned models, we categorize them as medium [37, 97, 252, 253, 268]. For larger datasets (around tens of 1000s of examples) we denote this as large [56, 204, 254]. For models that undergo extensive large-scale pre-training, we categorized data used in this process as extremely large to indicate a dataset of millions of examples [43, 220, 225, 273] or more. We also included an unknown if the paper did not explicitly mention the dataset used for training [178, 190, 235].

Model - Type:. What is the type of the underlying model? Advancements in AI accelerators and the availability of large amounts of data have led to an evolution in model architectures,⁵ which we capture as the following four types. First, rule-based models rely on pre-defined logic, lookup tables, regular expressions, or other similar heuristic approaches that are deterministic in nature [10, 29, 90, 218]. For statistical machine learning (ML) models, we consider models that are trained from scratch on historical data, are not necessarily “deep” (as in deep neural networks), and are used to make future predictions (e.g., support vector machines and logistic regression) [111, 121, 193, 268]. Over the past decade, deep neural networks have been the popular models of choice for writing assistants, including recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) [7, 47, 208, 259]. Finally, recent works have increasingly utilized foundation models, such as BERT [61], RoBERTa [157], GPT [23, 198, 199], and T5 [200], to name a few. A foundation model is “any model that is trained on broad data that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks” [17]. These models can perform a wide range of tasks out of the box, learn from a few examples to provide tailored support to users, and be further fine-tuned for specific downstream task(s) [146, 220, 251, 255].

Model - External Resource Access:. What additional access does the model have at inference time? Recently, models have been developed with access to additional tools or data at inference time to make them capable of providing assistance beyond the knowledge encoded in their parameters. In the case of tool, a model may access external software or third-party APIs to perform tasks like search, translation, or calculator, or even setting calendar events on behalf of users [43, 178, 264, 269]. Data refers to external datasets or resources, such as information stored in a database, external knowledge repositories, or any other structured/unstructured data sources that the models might leverage to provide writing assistance [132, 220, 227, 273].

Learning - Problem:. How is the writing assistance task being formulated as a learning problem? How exactly writing assistants support their users usually varies based on the learning problem their underlying models are designed to solve. Classification refers to the class of problems that require categorizing data into predefined classes based on their attributes. It is one of the most widely formulated classes of problems in writing assistants, applicable to tasks such as detecting errors in writing [77, 243] and detecting the purpose of writing revisions [123], among others. In contrast, regression problems involve the prediction of a continuous numerical value or quantity as the output instead of categorical labels or classes. This includes problems such as the prediction of the writer’s sentiments [249], the readability [128], or the emotional intensity [237] of written text as numerical ratings or scores. Structured prediction refers to a class of learning problems that involve predicting structured outputs or sequences (e.g., sequences, trees, and graphs) rather than single, isolated labels or values. Numerous works have focused on developing these approaches to make edits to improve the quality of written text during the revision stage [136, 165, 166, 167, 230]. Rewriting problems involve sequence transduction tasks, where texts from one form are transformed to another while improving the quality by making them fluent, clear, readable, and coherent. These tasks are essential in various writing assistance applications, such as grammatical error correction [37, 43, 274], paraphrasing [264], or general-purpose text editing [66, 70, 201, 223] to name a few. Generation refers primarily to problems that involve the creation of new, contextually relevant, coherent, and readable text from relatively limited inputs, such as autocomplete, paraphrasing, and story generation [7, 45, 116, 217]. Retrieval problems take the input from a user as a query (e.g., keywords) and search in a knowledge base or dataset for relevant information. Such problems may involve ranking the available data based on its relevance and similarity to the input but do not necessarily include the generation of new text beyond what is available in the knowledge base [37, 220, 229].

Learning - Algorithm:. How is the underlying model trained? The models used as back-bones of writing assistants incorporate different training mechanisms based on the type of the available data, as well as the specific downstream tasks. In supervised learning, models are trained on a labeled dataset where each input is associated with the correct output. Some of the commonly used methods include Logistic Regression, Random Forests, and Naive Bayes [16, 26, 97, 253]. Supervised learning also includes approaches such as Transfer Learning, which involves training a model on a large dataset and then fine-tuning it for a specific task or domain using a smaller dataset [68, 274]. In unsupervised learning, models are trained on unlabeled data to learn patterns and structures within the data. This approach includes techniques such as representation learning and clustering methods, to name a few [185, 254]. Self-supervised learning approaches train models on unlabeled data with a supervisory signal [81]. These approaches leverage the benefits of both supervised and unsupervised learning, especially in scenarios where obtaining a large amount of labeled data is challenging. This includes pre-training objectives for large language models such as Causal Language Modeling [199] and Masked Language Modeling [61]. In reinforcement learning (RL), models learn by interacting with an environment and receiving feedback in the form of rewards. This approach is useful for tasks requiring action sequences, such as language generation and dialogue systems [225].

Learning - Training and Adaptation:. How is the underlying model being trained or adapted for a specific task at hand? The training and adaptation process is integral part of developing an intelligent model that can perform tasks at hand and support user needs. Many models used to be trained from scratch [83, 98, 158, 235] before foundation models. On the other hand, with the advance of foundation models (e.g., BERT and GPT-4), the common learning paradigm has been shifted to “pre-training” a large model on broad data and then “adapting” the model to a wide range of downstream tasks. One way to adapt a model is fine-tuning, where the pre-trained model is further trained on a specific dataset [13, 195, 226, 272]. Note that there are numerous variants of fine-tuning, such as transfer learning, instruction tuning, alignment tuning, prompt tuning, prefix tuning, and adapter tuning, among others. Another way to adapt a model is prompt engineering (or “prompting”), where one can simply provide a natural language description of the task (or “prompt”) [22] to guide model outputs [58, 120, 146, 172]. A prompt may include a few examples for a model to learn from (“few-shot learning” or “in-context learning”). Lastly, we can tune decoding parameters of a model to influence model outputs (e.g., changing temperature to make outputs more or less predictable, manipulating logit bias to prevent some words from being generated) [88, 146, 218].

Evaluation - Evaluator:. Who evaluates the quality of models outputs? A core aspect of model development is its evaluation. We consider four common types of evaluators who can review and evaluate various qualities of model outputs (as opposed to writing assistants or user interactions). Automatic evaluation compares machine-generated outputs with human-generated labels or texts using aggregate statistics or syntactic and semantic measures. These include metrics like precision, recall, F-measure, and accuracy, as well as ones used in generation tasks such as BLEU [188], METEOR [142], and ROUGE [263] to name a few [3, 37, 235]. Machine-learned evaluation uses automated metrics, which are themselves produced by a machine-learned system. These are typically classification or regression models that are trained to evaluate the quality of model outputs [123, 196, 219, 249, 270]. On the other hand, human evaluation corresponds to evaluating the system with human annotators either directly interacting with, or evaluating the output of a writing assistant. Some evaluations may require the judging of task-specific criteria (e.g., evaluating that certain entity names appear correctly in the text [173]), while others can be generalized for most text generation tasks (e.g., evaluating the fluency or grammar of the generated text [156, 163, 247, 265]). Human-machine evaluation captures cases where both machine-learned metrics or models and human judges are involved in the evaluation of the outputs. This hybrid evaluation is particularly relevant in co-creative, mixed-initiative writing assistance settings. Such studies often involve expert users and participatory methodologies [45, 146, 154, 172].

Evaluation - Focus:. What is the focus of evaluation when evaluating individual model outputs? Evaluating (or benchmarking) models has been a long standing challenge in NLP [152]. In particular, as we increasingly use foundation models (e.g., GPT-4) for a wide range of downstream tasks, it is difficult to evaluate the quality of model outputs across all tasks, let alone the difficulty of evaluating open-ended generation. Here, we highlight four common evaluation focus relevant to writing assistants in the literature. Linguistic quality focuses on the grammatical correctness, readability, clarity, and accuracy of the model’s outputs. This aspect ensures that the outputs are not only correct in terms of language use but also easily understandable and precise in conveying the intended message [58, 128, 251, 273]. Controllability assesses how well the model’s outputs reflect constraints (or control inputs) specified by users or designers. For instance, how effectively the model adheres to any specific level of formality or writing style [120, 195, 217, 238]. Furthermore, it is crucial that the model’s responses not only make sense in isolation, but also fit seamlessly within the broader context of the text. Style & adequacy pertains to the alignment between the model’s outputs and their surrounding texts or contexts. This includes evaluating the stylistic and semantic coherence, relevance, and consistency of the outputs with the given context [84, 160, 178, 265]. Finally, ethics encompasses a range of crucial considerations such as bias, toxicity, factuality, and transparency. Ethics focuses on the model’s outputs adherence to social norms and ethical standards, and seeks to avoid generating outputs that contain harmful biases, misinformation, and other unethical elements [15, 105, 193, 227]. This aspect of evaluation is particularly critical in maintaining the trustworthiness and societal acceptance of the model.

Scalability:. What are the economic and computational considerations for training and using models? Recent models, especially LMs, have demonstrated exceptional performance across various tasks [25, 40, 184]. However, the significantly large size of these models has substantially increased the cost of their development [127]. In this regard, directly utilizing pre-trained LMs via prompting [23, 257] or employing efficient fine-tuning methods like low-rank adaptation [109] and prefix-tuning [151] can help avoid the cost of full fine-tuning. During deployment, this affects not only the inference costs but also the latency, which often degrades user experience [30, 147]. Techniques such as quantization [87] and knowledge distillation [103] have shown promising results in addressing these issues.

Figure 1 (“Interaction”) shows interaction dimensions in a broad context, while Figure 3 shows a detailed visualization. Tables 5 and 6 list all dimensions, codes, and definitions.

UI - Interaction Metaphor:. What is the interaction metaphor for the system? Interaction metaphors shape how the user relates to the system. We identify three primary metaphors. First, systems designed as agents employ designs meant to evoke human-like interaction, including roles like “collaborator” or “co-writer” [36, 146], “dialog partner” [189, 246], “assistant” [80], and “companion” [84]. Techniques to evoke the agent metaphor include character-by-character text rendering to simulate typing [120], avatars [29], implicit and explicit conversational interaction [189], and first-person conversational styles [193]. In contrast, other systems present as tools, where there is no sense of interacting with another agent. These systems tend to avoid conversational styles and rather present feedback in imperative or factual style [113, 247, 255]. They provide traditional GUI elements (e.g., checkboxes, buttons) that spread out system capabilities rather than centralizing them into one “agent.” Hybrid systems blend “agent” and “tool” metaphors in their design or their authors’ descriptions [84, 146].

UI - Layout:. Where are the system interactions situated in the UI? Often, interaction with the underlying system takes place in the writing area where users create text. This supports the selection of the existing writing [57, 220] and/or seamless integration of output from the system [58, 146]. Alternatively, a design might choose to isolate the interaction with AI as a separated UI element, such as through a sidebar. A separated design puts clear boundaries between the users’ writing and the output of the system. For example, this design style is used to display information related to text diagnosis [243], provide inspiration [45, 132], or support language learning [37]. Interaction with the system can also be embedded with the user’s text input UI, such as text suggestions on touchscreen keyboards [7]. Finally, there are custom designs: for example, a tangible UI that triggers the system by lifting a coffee cup [12], or an exploratory visualization for referencing information which has no user text input [130].

UI - Interface Paradigm:. What is the general platform of the interface? The text editor is the prevalent interface for AI-assisted writing [58, 220]. It provides writers with a traditional blank canvas while incorporating a variety of system-driven functionalities such as feedback [113], automated checklists [80], or completion suggestions [120]. In contrast, chatbot interfaces use turn-taking interactions for motivation [189] or suggestions [35]. Conversational exchanges serve to progressively and iteratively achieve a goal, like fiction writing [15, 48, 217]. Finally, other interfaces cater to specific needs, introducing novel [130, 213], sometimes multimodal [45, 226], interactions. For example, such custom interfaces take lyrical structure into account for music generation [254], and emphasize the conceptual and figurative expressions to generate metaphors for scientific writing  [132].

UI - Visual Differentiation:. How is the system output differentiated from user output? This dimension identifies different visual designs for separating user-written text and system outputs. A system must present its outputs to the user in some way, or the interaction loop between the user and the system is incomplete. The most common mechanism is to use text formatting such as colors and underlines [58, 264, 265]. Another common mechanism is to keep system outputs in a separate location like a wizard or panel [57, 193, 249]. A system may not use any formatting when the system output has a different media type (e.g., meta-analysis [113, 255] or audio [12]) than the text written by the user. Alternatively, the differentiation is “none”: System output is presented identically to user output after it is added into the text [7, 146].

UI - Initiation:. How is system output triggered? System outputs can be triggered in two main ways: User-initiated triggers give the user control over when system output is created or presented in the UI. For example, phrase suggestions might be displayed whenever the user presses the “tab” key [146] or clicks on a “generate” button [85]. In contrast, with system-initiated triggers, the system has control over when it brings in its output. This might be a rule, heuristic, or a dedicated trigger decision model. For example, the user and system might take turns such that the system output is triggered when the user submits a chat message [189] or sentence [48], or pauses writing for a certain amount of time [13, 27, 120]. For some systems, this initiative is almost real-time, or “live.” For example, several systems update suggestions and feedback in a dedicated panel while the user is typing [7, 37, 243].

User - Steering the System:. How can the user steer the system? Users need to communicate their intentions and goals to a system to steer its behavior. In the implicit paradigm, users directly compose the artifact (typically, text in a workspace) as a way to communicate their intentions to the system which in turn provides support based on this information. Such systems often provide support in the form of text suggestions [146, 265], reflection [57, 189], or inspiration and ideation [84]. Alternatively, users have explicit controls over the behavior of the system. For example, users give thumbs up or down on shown text suggestions to steer future output towards user preferences [49], control the diversity of generated text through numeric parameters [88], or steer story arcs via sketching [45]. The two paradigms have implications for the users’ workflow [58]: implicit input can be less disruptive to the writing process, while explicit input offers more ways for users to express intentions. Finally, some systems offer no control to the user, for example, if the LM is prompted in the background to respond in a particular way regardless of the user’s prior text [120].

User - Integrating System Output:. How does the user integrate system output? After the system creates an output, the user can choose how to integrate it into the overall goal. For material intended to be included in the output, the user may take a selection action, such as accepting a suggestion using a button or key press [7, 132, 146]. When the system does not provide material for direct inclusion, the user may engage with the output as inspiration and choose to type their own text [189, 252]. If the system text is inserted into the final text with no explicit user interaction, the user must then choose whether to keep, edit, or remove the text, through editing [45, 172]. Finally, the system may provide outputs that require no integration, if they are not meant to be added to an output or to inform direct contributions. For example, analysis for future improvements [113, 255].

System - User Data Access:. What user data can the system access through the UI? Intelligent writing assistants can access different types of user data to produce the desired outputs. Typically, the user’s input text is the primary source of data; the current writing progress can be used to generate completions [7, 146] or provide feedback [29, 77]. It is also important to allow users to have additional controls over the text generations, which we characterize as additional data. For example, some systems allow users to specify the task for generation via explicit instruction [35, 265] or via sketching [45].

System - Output Type:. What type of output does the system create? Intelligent writing systems can generate different types of outputs to support users’ writing. System output might serve as an analysis of the user’s writing. For example, how clear text is in an administrative context [77], how empathetic a peer review is [249], or how structured and formal writing is in a professional context [113]. Systems can provide this analysis in the form of annotations or highlights on the text [113, 249], analytics in a separate UI [128, 132], or high-level statements about the text [29, 203]. In contrast, generation is usually realized as completions [7, 13, 254], although it can also encompass longer sections (e.g., creating a paragraph of text [45, 68]). Finally, some system output only provide proposals: Content meant to be used as reference but not directly incorporated into the document. This content can be in the form of planning and outlining support [203, 213], or questions about the user’s writing to encourage reflection [29] or summaries of the text so far [57].

System - Curation Type:. How are system outputs curated? The predominant way that intelligent writing support systems curate outputs is to generate a custom response from an NLP model, especially with the recent dramatic increase in fluency and capability of LMs  [120, 146, 208, 254]. However, providing the natural language outputs of such models requires the designer to give up a great deal of control over the possible outputs. In sensitive contexts such as topics like mental health [189, 193, 252] or to leverage the benefit of high-quality source material, designers choose to trade off the flexibility of an NLP model for the control of a pre-made, curated list of responses [229], from which an output is selected by the system [29, 37, 217]. To increase personalization, these pre-curated responses may also be dynamically customized [189]. Alternatively, the output may be generated from user input in a deterministic way, in writing systems that do not use AI and/or prefer rule-based automation [12, 137, 213].

Figure 1 (“Ecosystem”) shows ecosystem dimensions in a broad context, while Table 7 lists all dimensions, codes, and definitions.

Digital Infrastructure:. What compatibility issues are considered? A key compatibility issue to consider is usability consistency, i.e., the extent to which the writing assistant’s user experience intentionally aligns with other systems in the writer’s ecosystem. Examples in the corpus include integrating AI language technologies into everyday writing apps [96], extending Google Docs [244], and using familiar-looking BibTeX style codes to enable writers to invoke remote bibliographic searches [10]. This example also illustrates technical interoperability with external services, which designers may conceive in many ways. For instance, another project integrated tangible media with their writing assistant, such that lifting a mug on a digital coaster triggered text-to-speech replay of recent sentences to assist reflection [12].

Access Model:. How does the openness of data, models, and products influence design decisions? This dimension taps into how the openness and licensing of data, models, and products may influence design decisions and dissemination of artifacts. First, when a new writing assistant is developed to be embedded within existing commercial software, products, or platforms, the aesthetic and display space of these products exert a strong influence over the UX and UI design of the writing assistant, as we see in the examples of Airbnb [121] and Facebook [189]. This could further affect the choice of data (e.g., proprietary data) and models (e.g., closed models). On the other hand, for many researchers, it is common and often desired practice to build and work with free and open-source software (as well as open-source data and models [239, 241, 258]) given their transparency and free availability. Lastly, researchers in turn can open source their writing assistants [146, 179] as well as other associated artifacts, such as interaction traces for human-LM collaborative writing [146] and programming languages for generating text [107], to promote collaboration and reproducibility.

Social Factors:. Who affects the design and use of writing assistants? This dimension draws attention to how designers engage with stakeholders, and the social support around authors. First, designing with stakeholders concerns human-centered and participatory design methods to meaningfully involve target authors, and other groups such as educators, coaches, or subject-matter experts. This was by far the most prevalent ecosystem dimension in the corpus, attributable to design concept interviews, co-design sessions, and usability studies with target authors [172, 186, 226]. While not in the corpus, other research focuses on co-designing tools with educators to build trust in their design [212, 221]. Second, designing for social writing covers the formal and informal support network writers may call on, including co-authors, peers, mentors, friends and family. For instance, children writing at home may involve informal social support from their parents [231, 233].

Locale:. Does the writing assistant’s design take into account features of a physical locale? Digital systems are used in physical environments, whose affordances could aid writing (e.g., whiteboard used to plan a document, sticky notes on screens, and opportunistic conversations with passing colleagues) or could impede writing (e.g., a noisy environment may permit notetaking but disrupt the attention needed for detailed writing; fieldwork with limited Internet access may prevent certain writing until back in the office). This dimension draws attention to whether the writer is engaged in local writing (i.e., at what is considered to be “home base” with full social and technical networking), or remote writing (i.e., away from home base, with different affordances and constraints) such as fieldwork or commuting. Much of the research to date did not attend to this explicitly, but examples include designing an intentionally calm interface to help reduce classroom distractions for vulnerable young people [89], and using a messaging platform to encourage more reflective writing for well-being [189, 252].

Norms & Rules:. What norms and rules affect the design and use of a writing assistant? Writing is embedded in countless societal processes, therefore they will inevitably be governed by various norms and rules, both formal and informal. First, there can be alignments or misalignments with laws. While none of the corpus papers addressed legal issues, this is of course an active field of scholarship [95]. Furthermore, U.S. and E.U. legislation is changing [53, 106] with intellectual property legal cases under way [75, 206]. Consequently, this code draws designers’ attention to legal changes, which could conceivably shift market preference to LMs trained on ethically sourced data, or passing a particular algorithmic impact assessment. Second, writing assistants could account for societal conventions. User-centered design builds on concepts and practices familiar to users, such as writing conventions in job application letters [113] and a system based on an established typology of expository phrases for science writing that readers and writers would recognize [85]. Other types of conventions guiding design decisions include features of good metaphors [84], the social acceptability of automating emails, and clinical principles to support for writers with aphasia [179] or their mental health [189]. Emerging evidence [31, 60] may raise expectations around employee productivity, with the Writers Guild of America strike in the U.S. demonstrating the conflict that the proliferation of AI writing is now provoking [181]. While there were no corpus papers documenting the embedding of writing assistants into established work practices and conventions, we see this beginning to happen in K-12 [212] and higher education [52, 138].

Change Over Time:. What are the key temporal considerations when designing a writing assistant? This last dimension recognizes that people, technology, regulation, and the broader information environment are in motion, not static. We anticipate writing assistants and the ecosystem influencing each other over different timescales, from instantaneous to longer-term change. As detailed in the user and interaction dimensions, writing assistants can be shown to have demonstrable effects on authors and their writing outcomes, with empirical studies documenting immediate effects on product reviews [7, 120], stories [226], screenplays [172] and business pitches [247], to name just a few genres. However, while we found no empirical studies beyond a single writing session, some researchers anticipate the longer-term risks of homogenization in creative writing [5, 84, 187], professional deskilling in written communication [27], and loss of author agency simply through fatigue in reviewing AI suggestions [13]. Designers should consider the risk that AI-generated text is ingested as training data by other AI projects (“model collapse” [224, 240]), an example of change in information environment [7].

We can also anticipate changes in readers. As LMs can (co-)author complex writing that is hard to distinguish from human-authored text [46, 122], designers should consider different readers’ criteria for trustworthiness [121]. However, as with authors, there were no longitudinal studies of readers or reading practices with writing assistants. Given the current pace of technological development, frequent changes in technology will be the norm, much more so than what we observed in the past [94, 161]. A longer-term HCI perspective might ask how systems can be designed to gracefully adapt to the evolution of technology (e.g., assist an author who dislikes the new LM to roll back to their older, more personally-tuned version). Finally, as regulation catches up with technological advances, it could affect model performance (e.g., models trained on copyright material could be banned), procurement (e.g., corporate AI governance restricts products meeting new ethical standards), or subscription models (e.g., more “ethical” LMs might cost more).