Creating user interfaces that work like natural conversations requires a new design paradigm. Con... more Creating user interfaces that work like natural conversations requires a new design paradigm. Conversations are fundamentally series of sequences of utterances, or strings of text. As such, methods for laying out visual elements spatially do not help designers in authoring conversations, or dialog structures. Furthermore, current platforms for creating natural-language interfaces provide little guidance on how to design dialog in ways that emulate or evoke the interaction mechanics of human conversation. Fortunately, guidance can be found in the social sciences. Conversation Analysis (CA) in particular, offers decades of findings about the interactional practices of natural human conversation. The IBM Natural Conversation Framework (NCF) adapts empirical findings from CA to the practice of user experience design. It provides basic conversation theory, interaction patterns, reusable software components and more, to designers and developers. In short, it offers a novel design methodology and paradigm. In this paper, we outline the NCF methodology, focusing on its features for natural user feedback. We then introduce a new set of patterns recently added to its pattern language that enable agents to initiate next topics and thereby to guide users. Finally, we present results from a user study of both natural user feedback and next topic features.
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces, 2020
Design systems for creating user experience reduce effort, scaffold learning and increase collabo... more Design systems for creating user experience reduce effort, scaffold learning and increase collaboration [2], yet they are currently available primarily for applications with visual interfaces. Design systems for conversational UX are still few and far between. Any design system should provide a distinctive philosophy, interaction patterns and content format. The Alma Design System, from IBM Research, provides these. Its design philosophy takes human conversation as its metaphor and adapts principles from conversation science, such as mutual understanding, recipient design, minimization and repair. Its pattern language provides 100 interaction patterns and components adapted from the literature of Conversation Analysis. It includes formal patterns for conversational activities and their management. And its content format consists of six slots and can be extended to any number of parts.
With the rise in popularity of chatbot and virtual-agent platforms, from Apple, Amazon, Google, M... more With the rise in popularity of chatbot and virtual-agent platforms, from Apple, Amazon, Google, Microsoft, Facebook, IBM and more, a new design discipline is emerging: Conversational UX Design. While it is easy to create natural language interfaces with these platforms, creating an effective and engaging user experience is still a major challenge. Natural language processing (NLP) techniques have given us powerful tools for analyzing bits of language, but they do not tell us how to string those bits together to make a natural conversation. Natural conversation has a sequential organization that is independent of the organization of language itself. At IBM Research-Almaden, we are addressing this user experience (UX) design challenge by applying formal, qualitative models from the field of Conversation Analysis to the design of conversational agents. Our Natural Conversation Framework (NCF) is a design framework for conversational user experience. It provides a library of generic conversational UX patterns that are inspired by natural human conversation patterns and that are agnostic to platform and input method (text or voice). This chapter will cover the four components of our Natural Conversation Framework: (1) an interaction model, (2) common activity modules, (3) a navigation method and (4) a set of sequence metrics. In addition, it will briefly outline a general process for designing conversational UX: from mock-up to working prototype.
Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework , 2019
As voice interfaces and virtual assistants have moved out of the industry research labs and into ... more As voice interfaces and virtual assistants have moved out of the industry research labs and into the pockets, desktops and living rooms of the general public, a demand for a new kind of user experience (UX) design is emerging. Although the people are becoming familiar with Siri, Alexa, Cortana and others, their user experience is still characterized by short, command- or query-oriented exchanges, rather than longer, conversational ones. Limitations of the microphone and natural language processing technologies are only part of the problem. Current conventions of UX design apply mostly to visual user interfaces, such as web or mobile; they are less useful for deciding how to organize utterances, by the user and the virtual agent, into sequences that work like those of natural human conversation.
This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.
This article explores the potential of automated transcription technology for use in Conversation... more This article explores the potential of automated transcription technology for use in Conversation Analysis (CA). First, it applies auto-transcription to a classic CA recording and compares the output with Gail Jefferson’s original transcript. Second, it applies auto-transcription to more recent recordings to demonstrate transcript quality under ideal conditions. And third, it examines the use of auto-transcripts for navigating big conversational data sets. The article concludes that although standard automated transcription technology lacks certain critical capabilities and exhibits varying levels of accuracy, it may still be useful for (a) providing first-pass transcripts, with silences, for further manual editing; and (b) scaling up data exploration and collection building by providing time-based indices requiring no manual effort to generate. Data are in American English.
Today’s Internet search engines are highly effective in returning relevant web pages to users in ... more Today’s Internet search engines are highly effective in returning relevant web pages to users in fractions of seconds. Yet interactions with search engines are far from trouble free. When interacting with search engines, users experience a variety of troubles, which are still poorly understood. One particular kind of trouble stems from users’ prior knowledge about entities of interest, particularly regarding their names. This study examines how referential practice is organized in the context of search-engine interactions. It finds that, as in conversation, users employ naming in their queries to refer to entities if they can. However, when they do not know the name, or a name fails, they attempt a two-stage search: first they search for the entity name, using generic descriptions combined with image search , and second, if the name is found, they formulate subsequent queries using that name. Computer interaction analysis is used to reveal formal features of users’ referential practices from recordings of screen video with eye tracking and design recommendations for search engines are offered.
Internet search engines display understanding or
misunderstanding of user intent in and through ... more Internet search engines display understanding or
misunderstanding of user intent in and through the
particular batches of results they retrieve and their
perceived relevance. Yet understanding is not simply an
automatic outcome but a joint interactional achievement
between human and machine. If potential troubles with
search queries emerge, either the user or the search engine
may initiate repair on the query in ways that resemble repair
in human conversation as described in conversation
analysis. Users can repair their own queries in first or third
position, while search engines can initiate repair from
second position. However search-engine interactions
currently contain no fourth-position repair. Finally search
engines may also complete queries collaboratively with
users in ways that are similar to but distinct from repair. In
this study we examine interactions between users and
search engines using a novel approach we call "computer
interaction analysis," which utilizes eye-tracking screen
video and a novel notation scheme for transcribing it.
Today’s personal computers enable complex forms of user interaction. Unlike older mainframe compu... more Today’s personal computers enable complex forms of user interaction. Unlike older mainframe computers that required batch processing, personal computers enable real-time user control on a one-to-one basis. Such user interaction involves mixed initiative, logic, language and pointing gestures, features reminiscent of interaction with another human. Yet there are also major differences between computer interaction and human interaction, such as computers’ inability to stray from scripts or to adapt to the idiosyncrasies of particular recipients or situations. Given these similarities and differences, can we study computer interaction using methods similar to those for studying human interaction? If so, are the findings from the analysis of human interaction also useful in understanding computer interaction? In this paper, we explore these questions and outline a novel methodological approach for examining human-computer interaction, which we call “computer interaction analysis.” We build on earlier approaches to human interaction with a computer and adapt them to the latest technologies for computer screen capture and eye tracking. In doing so, we propose a new transcription notation scheme that is designed to represent the interweaving streams of input actions, display events and eye movements. Finally we demonstrate the approach with concrete examples involving the phenomena of placeholding, repair and referential practices.
Referential practice – the variety of ways in and through which speakers refer to things in socia... more Referential practice – the variety of ways in and through which speakers refer to things in social interaction – involves a range of very different methods. When referring to physical objects or processes in face-to-face interaction, people may choose from a variety of resources, including verbal categories, names, pointing, verbal descriptors, depictive gestures, and prop demonstrations. This raises the question: Under what circumstances do speakers choose particular resources over others? To address this question, this study examines referential practice in a particular kind of face-to-face workplace setting, the service counter of a quick print shop. At the service counter, not only do customers use alternative resources in referring to the document services they want, but these resources appear to be ordered relative to one another in terms of a preference for minimization. In referring to document services, customers first try the most minimal form, the official name, but if that fails or is unknown, customers fall back on more expanded forms of reference, such as pointing or depicting.
Massively multiplayer online (MMO) environments are an emerging computer technology that makes po... more Massively multiplayer online (MMO) environments are an emerging computer technology that makes possible new kinds of
distributed communities and online sociability. What distinguishes MMOs from other Internet media is that they take face-toface
conversation as their primary metaphor for user interaction, rather than, say, the page or the bulletin board. Because they
simulate 3D spaces and contain thousands of people who do not know each other, MMOs constitute public spaces, although
virtual ones. As such, they can be studied in ways analogous to those of public places in the physical world. Inspired by the
work of William H. Whyte and Ray Oldenburg on sociability in real-life public places, we take a similar approach toward the
study of MMOs. We ask the question: what makes some virtual public spaces in MMOs successful “third places” while other
similar places fail? Through our virtual ethnography of dance clubs and corner bars in three MMO environments, we find four
features of virtual public spaces that appear critical for their success: accessibility, social density, activity resources, and hosts.
We further argue that MMO sociability is just as authentic as that in “real-life” contexts while highlighting ways in which it
is distinctly different.
Georg Simmel [American Journal of Sociology 55:254–261 (1949)] is widely credited as the first sc... more Georg Simmel [American Journal of Sociology 55:254–261 (1949)] is widely credited as the first scholar to have seriously examined sociability – ‘‘the sheer pleasure of the company of others’’ and the central ingredient in many social forms of recreation and play. Later Ray Oldenburg [The Great Good Place. New York: Marlowe & Company (1989)] extended Simmel’s work by focusing on a certain class of public settings, or ‘‘third places,’’ in which sociability tends to occur, such as, bars, coffee shops, general stores, etc. But while Simmel and Oldenburg describe activities and public spaces in the physical world, their concepts may apply as well to virtual or online worlds. Today Massively Multiplayer Online Games (MMOGs) are extensive, persistent online 3D environments that are populated by hundreds of thousands of players at any given moment. The sociable nature of these online spaces is often used to explain their success: unlike previous video games, MMOGs require players to exchange information and collaborate in real-time to progress in the game. In order to shed light on this issue, we critically examine player-to-player interactions in a popular MMOG (Star Wars Galaxies). Based on several months of ethnographic observations and computerized data collection, we use Oldenburg’s notion of ‘‘third places’’ to evaluate whether or not the social spaces of this virtual world fit existing definitions of sociable environments. We discuss the role online games can play in the formation and maintenance of social capital, what they can teach us about the evolution of sociability in an increasingly digitally connected social world, and what could be done to make such games better social spaces.
Massively multiplayer online games (MMOGs) currently
represent the most widely used type of soci... more Massively multiplayer online games (MMOGs) currently
represent the most widely used type of social 3D virtual
worlds with millions of users worldwide. Although
MMOGs take face-to-face conversation as their metaphor
for user-to-user interaction, avatars currently give off much
less information about what users are doing than real
human bodies. Consequently, users routinely encounter
slippages in coordination when engaging in joint courses of
action. In this study, we analyze screen-capture video of
user-to-user interaction in the game, City of Heroes, under
two conditions: one with the game’s standard awareness
cues and the other with enhanced cues. We use
conversation analysis to demonstrate interactional slippages
caused by the absence of awareness cues, user practices that
circumvent such limitations and ways in which enhanced
cues can enable tighter coordination.
To date the most popular and sophisticated types of virtual worlds can be found in the area of vi... more To date the most popular and sophisticated types of virtual worlds can be found in the area of video gaming, especially in the genre of Massively Multiplayer Online Role Playing Games (MMORPG). Game developers have made great strides in achieving game worlds that look and feel increasingly realistic. However, despite these achievements in the visual realism of virtual game worlds, they are much less sophisticated when it comes to modeling face-to-face interaction. In face-to-face, ordinary social activities are ‘‘accountable,’’ that is, people use a variety of kinds of observational information about what others are doing in order to make sense of others’ actions and to tightly coordinate their own actions with others. Such information includes: (1) the real-time unfolding of turns-at-talk; (2) the observability of embodied activities; and (3) the direction of eye gaze for the purpose of gesturing. But despite the fact that today’s games provide virtual bodies, or ‘‘avatars,’’ for players to control, these avatars display much less information about players’ current state than real bodies do. In this paper, we discuss the impact of the lack of each type of information on players’ ability to tightly coordinate their activities and offer guidelines for improving coordination and, ultimately, the players’ social experience.
Media research has shown that people enjoy
watching television as a part of socializing in
grou... more Media research has shown that people enjoy
watching television as a part of socializing in
groups. However, many constraints in daily life
limit the opportunities for doing so. The Social TV
project builds on the increasing integration of
television and computer technology to support
sociable, computer-mediated group viewing
experiences. In this paper, we describe the initial
results from a series of studies illustrating how
people interact in front of a television set. Based on
these results, we propose guidelines as well as
specific features to inform the design of future
“social television” prototypes.
Effective group meetings are important for the productivity of corporations. But many meetings do... more Effective group meetings are important for the productivity of corporations. But many meetings do not achieve their goals because some people are too shy to speak while others are too dominant. To avoid the cost and intrusiveness of human meeting facilitation and to increase self-awareness of conversation behaviors, various types of meeting facilitators have been developed over the past couple of years. We present a prototype that is unique because it captures both individual and group behaviors and provides real time playful feedback. The portable prototype includes a set of table-top microphones with an audio interface to a laptop PC, where audio data are processed and an avatar-based UI displays the shared state of individual and group behaviors during a meeting. The interface reveals not only level of participation, but also several other meaningful but harder to detect behaviors such as turn taking, interruptions, and group laughter. The presentation’s design is deliberately playful to keep participants monitor, self-estimate and improve their meeting behavior.
Standardized survey interviewing is a form of institutional talk whose special constraints often ... more Standardized survey interviewing is a form of institutional talk whose special constraints often conflict with ordinary conversational practices.Although survey manuals tend to recognize the conflict between standardization and respondents’ requests for clarification, an alternative strategy for managing troubles in answering—“projective reporting”—is largely missing from the literature. Respondents may report circumstances in response to fixed-choice questions rather than providing an answer option. Such reports invite the interviewer to infer its relevance for an answer. Respondents use projective reporting in two different ways: to account for particular answers and to defer judgments in answering. Speakers and recipients in ordinary conversation have a variety of options for dealing with the upshots of projective reportings, but some of these options are prohibited in standardized interviews.When respondents use reporting to defer judgment, ordinary conversation and standardized interviewing can conflict.
Creating user interfaces that work like natural conversations requires a new design paradigm. Con... more Creating user interfaces that work like natural conversations requires a new design paradigm. Conversations are fundamentally series of sequences of utterances, or strings of text. As such, methods for laying out visual elements spatially do not help designers in authoring conversations, or dialog structures. Furthermore, current platforms for creating natural-language interfaces provide little guidance on how to design dialog in ways that emulate or evoke the interaction mechanics of human conversation. Fortunately, guidance can be found in the social sciences. Conversation Analysis (CA) in particular, offers decades of findings about the interactional practices of natural human conversation. The IBM Natural Conversation Framework (NCF) adapts empirical findings from CA to the practice of user experience design. It provides basic conversation theory, interaction patterns, reusable software components and more, to designers and developers. In short, it offers a novel design methodology and paradigm. In this paper, we outline the NCF methodology, focusing on its features for natural user feedback. We then introduce a new set of patterns recently added to its pattern language that enable agents to initiate next topics and thereby to guide users. Finally, we present results from a user study of both natural user feedback and next topic features.
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces, 2020
Design systems for creating user experience reduce effort, scaffold learning and increase collabo... more Design systems for creating user experience reduce effort, scaffold learning and increase collaboration [2], yet they are currently available primarily for applications with visual interfaces. Design systems for conversational UX are still few and far between. Any design system should provide a distinctive philosophy, interaction patterns and content format. The Alma Design System, from IBM Research, provides these. Its design philosophy takes human conversation as its metaphor and adapts principles from conversation science, such as mutual understanding, recipient design, minimization and repair. Its pattern language provides 100 interaction patterns and components adapted from the literature of Conversation Analysis. It includes formal patterns for conversational activities and their management. And its content format consists of six slots and can be extended to any number of parts.
With the rise in popularity of chatbot and virtual-agent platforms, from Apple, Amazon, Google, M... more With the rise in popularity of chatbot and virtual-agent platforms, from Apple, Amazon, Google, Microsoft, Facebook, IBM and more, a new design discipline is emerging: Conversational UX Design. While it is easy to create natural language interfaces with these platforms, creating an effective and engaging user experience is still a major challenge. Natural language processing (NLP) techniques have given us powerful tools for analyzing bits of language, but they do not tell us how to string those bits together to make a natural conversation. Natural conversation has a sequential organization that is independent of the organization of language itself. At IBM Research-Almaden, we are addressing this user experience (UX) design challenge by applying formal, qualitative models from the field of Conversation Analysis to the design of conversational agents. Our Natural Conversation Framework (NCF) is a design framework for conversational user experience. It provides a library of generic conversational UX patterns that are inspired by natural human conversation patterns and that are agnostic to platform and input method (text or voice). This chapter will cover the four components of our Natural Conversation Framework: (1) an interaction model, (2) common activity modules, (3) a navigation method and (4) a set of sequence metrics. In addition, it will briefly outline a general process for designing conversational UX: from mock-up to working prototype.
Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework , 2019
As voice interfaces and virtual assistants have moved out of the industry research labs and into ... more As voice interfaces and virtual assistants have moved out of the industry research labs and into the pockets, desktops and living rooms of the general public, a demand for a new kind of user experience (UX) design is emerging. Although the people are becoming familiar with Siri, Alexa, Cortana and others, their user experience is still characterized by short, command- or query-oriented exchanges, rather than longer, conversational ones. Limitations of the microphone and natural language processing technologies are only part of the problem. Current conventions of UX design apply mostly to visual user interfaces, such as web or mobile; they are less useful for deciding how to organize utterances, by the user and the virtual agent, into sequences that work like those of natural human conversation.
This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.
This article explores the potential of automated transcription technology for use in Conversation... more This article explores the potential of automated transcription technology for use in Conversation Analysis (CA). First, it applies auto-transcription to a classic CA recording and compares the output with Gail Jefferson’s original transcript. Second, it applies auto-transcription to more recent recordings to demonstrate transcript quality under ideal conditions. And third, it examines the use of auto-transcripts for navigating big conversational data sets. The article concludes that although standard automated transcription technology lacks certain critical capabilities and exhibits varying levels of accuracy, it may still be useful for (a) providing first-pass transcripts, with silences, for further manual editing; and (b) scaling up data exploration and collection building by providing time-based indices requiring no manual effort to generate. Data are in American English.
Today’s Internet search engines are highly effective in returning relevant web pages to users in ... more Today’s Internet search engines are highly effective in returning relevant web pages to users in fractions of seconds. Yet interactions with search engines are far from trouble free. When interacting with search engines, users experience a variety of troubles, which are still poorly understood. One particular kind of trouble stems from users’ prior knowledge about entities of interest, particularly regarding their names. This study examines how referential practice is organized in the context of search-engine interactions. It finds that, as in conversation, users employ naming in their queries to refer to entities if they can. However, when they do not know the name, or a name fails, they attempt a two-stage search: first they search for the entity name, using generic descriptions combined with image search , and second, if the name is found, they formulate subsequent queries using that name. Computer interaction analysis is used to reveal formal features of users’ referential practices from recordings of screen video with eye tracking and design recommendations for search engines are offered.
Internet search engines display understanding or
misunderstanding of user intent in and through ... more Internet search engines display understanding or
misunderstanding of user intent in and through the
particular batches of results they retrieve and their
perceived relevance. Yet understanding is not simply an
automatic outcome but a joint interactional achievement
between human and machine. If potential troubles with
search queries emerge, either the user or the search engine
may initiate repair on the query in ways that resemble repair
in human conversation as described in conversation
analysis. Users can repair their own queries in first or third
position, while search engines can initiate repair from
second position. However search-engine interactions
currently contain no fourth-position repair. Finally search
engines may also complete queries collaboratively with
users in ways that are similar to but distinct from repair. In
this study we examine interactions between users and
search engines using a novel approach we call "computer
interaction analysis," which utilizes eye-tracking screen
video and a novel notation scheme for transcribing it.
Today’s personal computers enable complex forms of user interaction. Unlike older mainframe compu... more Today’s personal computers enable complex forms of user interaction. Unlike older mainframe computers that required batch processing, personal computers enable real-time user control on a one-to-one basis. Such user interaction involves mixed initiative, logic, language and pointing gestures, features reminiscent of interaction with another human. Yet there are also major differences between computer interaction and human interaction, such as computers’ inability to stray from scripts or to adapt to the idiosyncrasies of particular recipients or situations. Given these similarities and differences, can we study computer interaction using methods similar to those for studying human interaction? If so, are the findings from the analysis of human interaction also useful in understanding computer interaction? In this paper, we explore these questions and outline a novel methodological approach for examining human-computer interaction, which we call “computer interaction analysis.” We build on earlier approaches to human interaction with a computer and adapt them to the latest technologies for computer screen capture and eye tracking. In doing so, we propose a new transcription notation scheme that is designed to represent the interweaving streams of input actions, display events and eye movements. Finally we demonstrate the approach with concrete examples involving the phenomena of placeholding, repair and referential practices.
Referential practice – the variety of ways in and through which speakers refer to things in socia... more Referential practice – the variety of ways in and through which speakers refer to things in social interaction – involves a range of very different methods. When referring to physical objects or processes in face-to-face interaction, people may choose from a variety of resources, including verbal categories, names, pointing, verbal descriptors, depictive gestures, and prop demonstrations. This raises the question: Under what circumstances do speakers choose particular resources over others? To address this question, this study examines referential practice in a particular kind of face-to-face workplace setting, the service counter of a quick print shop. At the service counter, not only do customers use alternative resources in referring to the document services they want, but these resources appear to be ordered relative to one another in terms of a preference for minimization. In referring to document services, customers first try the most minimal form, the official name, but if that fails or is unknown, customers fall back on more expanded forms of reference, such as pointing or depicting.
Massively multiplayer online (MMO) environments are an emerging computer technology that makes po... more Massively multiplayer online (MMO) environments are an emerging computer technology that makes possible new kinds of
distributed communities and online sociability. What distinguishes MMOs from other Internet media is that they take face-toface
conversation as their primary metaphor for user interaction, rather than, say, the page or the bulletin board. Because they
simulate 3D spaces and contain thousands of people who do not know each other, MMOs constitute public spaces, although
virtual ones. As such, they can be studied in ways analogous to those of public places in the physical world. Inspired by the
work of William H. Whyte and Ray Oldenburg on sociability in real-life public places, we take a similar approach toward the
study of MMOs. We ask the question: what makes some virtual public spaces in MMOs successful “third places” while other
similar places fail? Through our virtual ethnography of dance clubs and corner bars in three MMO environments, we find four
features of virtual public spaces that appear critical for their success: accessibility, social density, activity resources, and hosts.
We further argue that MMO sociability is just as authentic as that in “real-life” contexts while highlighting ways in which it
is distinctly different.
Georg Simmel [American Journal of Sociology 55:254–261 (1949)] is widely credited as the first sc... more Georg Simmel [American Journal of Sociology 55:254–261 (1949)] is widely credited as the first scholar to have seriously examined sociability – ‘‘the sheer pleasure of the company of others’’ and the central ingredient in many social forms of recreation and play. Later Ray Oldenburg [The Great Good Place. New York: Marlowe & Company (1989)] extended Simmel’s work by focusing on a certain class of public settings, or ‘‘third places,’’ in which sociability tends to occur, such as, bars, coffee shops, general stores, etc. But while Simmel and Oldenburg describe activities and public spaces in the physical world, their concepts may apply as well to virtual or online worlds. Today Massively Multiplayer Online Games (MMOGs) are extensive, persistent online 3D environments that are populated by hundreds of thousands of players at any given moment. The sociable nature of these online spaces is often used to explain their success: unlike previous video games, MMOGs require players to exchange information and collaborate in real-time to progress in the game. In order to shed light on this issue, we critically examine player-to-player interactions in a popular MMOG (Star Wars Galaxies). Based on several months of ethnographic observations and computerized data collection, we use Oldenburg’s notion of ‘‘third places’’ to evaluate whether or not the social spaces of this virtual world fit existing definitions of sociable environments. We discuss the role online games can play in the formation and maintenance of social capital, what they can teach us about the evolution of sociability in an increasingly digitally connected social world, and what could be done to make such games better social spaces.
Massively multiplayer online games (MMOGs) currently
represent the most widely used type of soci... more Massively multiplayer online games (MMOGs) currently
represent the most widely used type of social 3D virtual
worlds with millions of users worldwide. Although
MMOGs take face-to-face conversation as their metaphor
for user-to-user interaction, avatars currently give off much
less information about what users are doing than real
human bodies. Consequently, users routinely encounter
slippages in coordination when engaging in joint courses of
action. In this study, we analyze screen-capture video of
user-to-user interaction in the game, City of Heroes, under
two conditions: one with the game’s standard awareness
cues and the other with enhanced cues. We use
conversation analysis to demonstrate interactional slippages
caused by the absence of awareness cues, user practices that
circumvent such limitations and ways in which enhanced
cues can enable tighter coordination.
To date the most popular and sophisticated types of virtual worlds can be found in the area of vi... more To date the most popular and sophisticated types of virtual worlds can be found in the area of video gaming, especially in the genre of Massively Multiplayer Online Role Playing Games (MMORPG). Game developers have made great strides in achieving game worlds that look and feel increasingly realistic. However, despite these achievements in the visual realism of virtual game worlds, they are much less sophisticated when it comes to modeling face-to-face interaction. In face-to-face, ordinary social activities are ‘‘accountable,’’ that is, people use a variety of kinds of observational information about what others are doing in order to make sense of others’ actions and to tightly coordinate their own actions with others. Such information includes: (1) the real-time unfolding of turns-at-talk; (2) the observability of embodied activities; and (3) the direction of eye gaze for the purpose of gesturing. But despite the fact that today’s games provide virtual bodies, or ‘‘avatars,’’ for players to control, these avatars display much less information about players’ current state than real bodies do. In this paper, we discuss the impact of the lack of each type of information on players’ ability to tightly coordinate their activities and offer guidelines for improving coordination and, ultimately, the players’ social experience.
Media research has shown that people enjoy
watching television as a part of socializing in
grou... more Media research has shown that people enjoy
watching television as a part of socializing in
groups. However, many constraints in daily life
limit the opportunities for doing so. The Social TV
project builds on the increasing integration of
television and computer technology to support
sociable, computer-mediated group viewing
experiences. In this paper, we describe the initial
results from a series of studies illustrating how
people interact in front of a television set. Based on
these results, we propose guidelines as well as
specific features to inform the design of future
“social television” prototypes.
Effective group meetings are important for the productivity of corporations. But many meetings do... more Effective group meetings are important for the productivity of corporations. But many meetings do not achieve their goals because some people are too shy to speak while others are too dominant. To avoid the cost and intrusiveness of human meeting facilitation and to increase self-awareness of conversation behaviors, various types of meeting facilitators have been developed over the past couple of years. We present a prototype that is unique because it captures both individual and group behaviors and provides real time playful feedback. The portable prototype includes a set of table-top microphones with an audio interface to a laptop PC, where audio data are processed and an avatar-based UI displays the shared state of individual and group behaviors during a meeting. The interface reveals not only level of participation, but also several other meaningful but harder to detect behaviors such as turn taking, interruptions, and group laughter. The presentation’s design is deliberately playful to keep participants monitor, self-estimate and improve their meeting behavior.
Standardized survey interviewing is a form of institutional talk whose special constraints often ... more Standardized survey interviewing is a form of institutional talk whose special constraints often conflict with ordinary conversational practices.Although survey manuals tend to recognize the conflict between standardization and respondents’ requests for clarification, an alternative strategy for managing troubles in answering—“projective reporting”—is largely missing from the literature. Respondents may report circumstances in response to fixed-choice questions rather than providing an answer option. Such reports invite the interviewer to infer its relevance for an answer. Respondents use projective reporting in two different ways: to account for particular answers and to defer judgments in answering. Speakers and recipients in ordinary conversation have a variety of options for dealing with the upshots of projective reportings, but some of these options are prohibited in standardized interviews.When respondents use reporting to defer judgment, ordinary conversation and standardized interviewing can conflict.
With recent advances in natural language understanding techniques and farfield microphone arrays,... more With recent advances in natural language understanding techniques and farfield microphone arrays, natural language interfaces, such as voice assistants and chatbots, are emerging as a popular new way to interact with computers. They have made their way out of the industry research labs and into the pockets, desktops, cars and living rooms of the general public. But although such interfaces recognize bits of natural language, and even voice input, they generally lack conversational competence, or the ability to engage in natural conversation. Today's platforms provide sophisticated tools for analyzing language and retrieving knowledge, but they fail to provide adequate support for modeling interaction. The user experience (UX) designer or software developer must figure out how a human conversation is organized, usually relying on commonsense rather than on formal knowledge. Fortunately, practitioners can rely on conversation science.
This book adapts formal knowledge from the field of Conversation Analysis (CA) to the design of natural language interfaces. It outlines the Natural Conversation Framework (NCF), developed at IBM Research, a systematic framework for designing interfaces that work like natural conversation. The NCF consists of four main components: 1) an interaction model of "expandable sequences," 2) a corresponding content format, 3) a pattern language with 100 generic UX patterns and 4) a navigation method of six basic user actions. The authors introduce UX designers to a new way of thinking about user experience design in the context of conversational interfaces, including a new vocabulary, new principles and new interaction patterns. User experience designers and graduate students in the HCI field as well as developers and conversation analysis students should find this book of interest.
As voice interfaces and virtual assistants have moved out of the industry research labs and into ... more As voice interfaces and virtual assistants have moved out of the industry research labs and into the pockets, desktops and living rooms of the general public, a demand for a new kind of user experience (UX) design is emerging. Although the people are becoming familiar with Siri, Alexa, Cortana and others, their user experience is still characterized by short, command- or query-oriented exchanges, rather than longer, conversational ones. Limitations of the microphone and natural language processing technologies are only part of the problem. Current conventions of UX design apply mostly to visual user interfaces, such as web or mobile; they are less useful for deciding how to organize utterances, by the user and the virtual agent, into sequences that work like those of natural human conversation. This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.
Uploads
This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.
misunderstanding of user intent in and through the
particular batches of results they retrieve and their
perceived relevance. Yet understanding is not simply an
automatic outcome but a joint interactional achievement
between human and machine. If potential troubles with
search queries emerge, either the user or the search engine
may initiate repair on the query in ways that resemble repair
in human conversation as described in conversation
analysis. Users can repair their own queries in first or third
position, while search engines can initiate repair from
second position. However search-engine interactions
currently contain no fourth-position repair. Finally search
engines may also complete queries collaboratively with
users in ways that are similar to but distinct from repair. In
this study we examine interactions between users and
search engines using a novel approach we call "computer
interaction analysis," which utilizes eye-tracking screen
video and a novel notation scheme for transcribing it.
distributed communities and online sociability. What distinguishes MMOs from other Internet media is that they take face-toface
conversation as their primary metaphor for user interaction, rather than, say, the page or the bulletin board. Because they
simulate 3D spaces and contain thousands of people who do not know each other, MMOs constitute public spaces, although
virtual ones. As such, they can be studied in ways analogous to those of public places in the physical world. Inspired by the
work of William H. Whyte and Ray Oldenburg on sociability in real-life public places, we take a similar approach toward the
study of MMOs. We ask the question: what makes some virtual public spaces in MMOs successful “third places” while other
similar places fail? Through our virtual ethnography of dance clubs and corner bars in three MMO environments, we find four
features of virtual public spaces that appear critical for their success: accessibility, social density, activity resources, and hosts.
We further argue that MMO sociability is just as authentic as that in “real-life” contexts while highlighting ways in which it
is distinctly different.
represent the most widely used type of social 3D virtual
worlds with millions of users worldwide. Although
MMOGs take face-to-face conversation as their metaphor
for user-to-user interaction, avatars currently give off much
less information about what users are doing than real
human bodies. Consequently, users routinely encounter
slippages in coordination when engaging in joint courses of
action. In this study, we analyze screen-capture video of
user-to-user interaction in the game, City of Heroes, under
two conditions: one with the game’s standard awareness
cues and the other with enhanced cues. We use
conversation analysis to demonstrate interactional slippages
caused by the absence of awareness cues, user practices that
circumvent such limitations and ways in which enhanced
cues can enable tighter coordination.
watching television as a part of socializing in
groups. However, many constraints in daily life
limit the opportunities for doing so. The Social TV
project builds on the increasing integration of
television and computer technology to support
sociable, computer-mediated group viewing
experiences. In this paper, we describe the initial
results from a series of studies illustrating how
people interact in front of a television set. Based on
these results, we propose guidelines as well as
specific features to inform the design of future
“social television” prototypes.
This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.
misunderstanding of user intent in and through the
particular batches of results they retrieve and their
perceived relevance. Yet understanding is not simply an
automatic outcome but a joint interactional achievement
between human and machine. If potential troubles with
search queries emerge, either the user or the search engine
may initiate repair on the query in ways that resemble repair
in human conversation as described in conversation
analysis. Users can repair their own queries in first or third
position, while search engines can initiate repair from
second position. However search-engine interactions
currently contain no fourth-position repair. Finally search
engines may also complete queries collaboratively with
users in ways that are similar to but distinct from repair. In
this study we examine interactions between users and
search engines using a novel approach we call "computer
interaction analysis," which utilizes eye-tracking screen
video and a novel notation scheme for transcribing it.
distributed communities and online sociability. What distinguishes MMOs from other Internet media is that they take face-toface
conversation as their primary metaphor for user interaction, rather than, say, the page or the bulletin board. Because they
simulate 3D spaces and contain thousands of people who do not know each other, MMOs constitute public spaces, although
virtual ones. As such, they can be studied in ways analogous to those of public places in the physical world. Inspired by the
work of William H. Whyte and Ray Oldenburg on sociability in real-life public places, we take a similar approach toward the
study of MMOs. We ask the question: what makes some virtual public spaces in MMOs successful “third places” while other
similar places fail? Through our virtual ethnography of dance clubs and corner bars in three MMO environments, we find four
features of virtual public spaces that appear critical for their success: accessibility, social density, activity resources, and hosts.
We further argue that MMO sociability is just as authentic as that in “real-life” contexts while highlighting ways in which it
is distinctly different.
represent the most widely used type of social 3D virtual
worlds with millions of users worldwide. Although
MMOGs take face-to-face conversation as their metaphor
for user-to-user interaction, avatars currently give off much
less information about what users are doing than real
human bodies. Consequently, users routinely encounter
slippages in coordination when engaging in joint courses of
action. In this study, we analyze screen-capture video of
user-to-user interaction in the game, City of Heroes, under
two conditions: one with the game’s standard awareness
cues and the other with enhanced cues. We use
conversation analysis to demonstrate interactional slippages
caused by the absence of awareness cues, user practices that
circumvent such limitations and ways in which enhanced
cues can enable tighter coordination.
watching television as a part of socializing in
groups. However, many constraints in daily life
limit the opportunities for doing so. The Social TV
project builds on the increasing integration of
television and computer technology to support
sociable, computer-mediated group viewing
experiences. In this paper, we describe the initial
results from a series of studies illustrating how
people interact in front of a television set. Based on
these results, we propose guidelines as well as
specific features to inform the design of future
“social television” prototypes.
This book adapts formal knowledge from the field of Conversation Analysis (CA) to the design of natural language interfaces. It outlines the Natural Conversation Framework (NCF), developed at IBM Research, a systematic framework for designing interfaces that work like natural conversation. The NCF consists of four main components: 1) an interaction model of "expandable sequences," 2) a corresponding content format, 3) a pattern language with 100 generic UX patterns and 4) a navigation method of six basic user actions. The authors introduce UX designers to a new way of thinking about user experience design in the context of conversational interfaces, including a new vocabulary, new principles and new interaction patterns. User experience designers and graduate students in the HCI field as well as developers and conversation analysis students should find this book of interest.
This edited book explores the intersection of UX design, of both text- or voice-based virtual agents, and the analysis of naturally occurring human conversation (e.g., the Conversation Analysis, Discourse Analysis and Interactional Sociolinguistics literatures). It contains contributions from researchers, from academia and industry, with varied backgrounds working in the area of human-computer interaction. Each chapter explores some aspect of conversational UX design. Some describe the design challenges faced in creating a particular virtual agent. Others discuss how the findings from the literatures of the social sciences can inform a new kind of UX design that starts with conversation.