chapter

ECA as user interface paradigm

Authors:

Richard Catrambone,

Jun XiaoAuthors Info & Claims

From brows to trust: evaluating embodied conversational agents

January 2004

Pages 239 - 267

Published: 01 January 2004 Publication History

Abstract

A strong debate has ensued in the computing community about whether Embodied Conversational Agents (ECAs) are beneficial and whether we should pursue this direction in interface design. Proponents cite tile naturalness and power of ECAs as strengths, and detractors feel that ECAs disempower, mislead, and confuse users. As this debate rages on, relatively little systematic empirical evaluation on ECAs is actually being performed, and the results from this research have been contradictory or equivocal. We propose a framework for evaluating ECAs that can systematize the research. The framework emphasizes features of the agent, the user, and the task the user is performing. Our goal is to be able to make informed, scientific judgments about the utility of ECAs in user interfaces. If intelligent agents can be built, are there tasks or applications for which an ECA is appropriate? Are there characteristics (in appearance, in personality, etc.) the ECA should have? What types of users will be more productive and happy by interacting with an ECA? Our initial experiment within this framework manipulated the ECA's appearance (realistic human versus iconic object) and the objectiviy of the user's task (editing a document versus deciding what to pack on a trip). We found that the perception of the ECA was strongly influenced by the task while features of the ECA that we manipulated had little effect.

References

[1]

Barker, T. (2003). The Illusion of Life Revisited. In Proceedings of AA-MAS 2003 Workshop on Embodied Conversational Characters as Individuals , Melbourne, Australia.

[2]

Bradshaw, J.M. (1997). An introduction to software agents. In Bradshaw J.M., editor, Software Agents, pp. 3-46, AAAI Press/MIT Press.

Digital Library

[3]

Cassell, J. (2000). Embodied conversational interface agents. Communications of the ACM, 43(4): 70-78

Digital Library

[4]

Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors (2000). Embodied Conversational Agents, MIT Press, Cambridge.

[5]

Cassell, J., and Bickmore, T. (2001). A Relational Agent: A Model and Implementation of Building User Trust. In Proceedings of the ACM CHI 2001 Conference, pp. 396-403, Seattle, Washington.

Digital Library

[6]

Collier, G. (1985). Emotional Expression, Lawrence Erlbaum Assoc., Hillsdale, NJ.

[7]

Dahlback, N., Jonsson, A., and Ahrenberg, L. (1993). Wizard of Oz studies - why and how. In Proceedings of the 1993 International Workshop on Intelligent User Interfaces, pp. 193-200, Orlando, FL.

Digital Library

[8]

Dehn, D.M. and Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52(1): 1-22.

Digital Library

[9]

Erickson, T. (1997). Designing agents as if people mattered. In Bradshaw J.M., editor, Software Agents, pp. 79-96, AAAI Press/MIT Press.

Digital Library

[10]

Griffin, P., Hodgson, P., and Prevost, S. (2003). Character User Interfaces for Commercial Applications. In Proceedings of AAMAS 2003 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.

[11]

Haddah, H. and Klobas, J. (2003). The Relationship between Visual Abstraction and the Effectiveness of a Pedagogical Character-Agent. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents - Let's Specify and Evaluate Them!, Bologna, Italy.

[12]

Isbister, K. and Doyle, P. (2002). Design and Evaluation of Embodied Conversational Agents: A Proposed Taxonomy. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents -- Let's Specify and Evaluate Them!, Bologna, Italy.

[13]

King, W. J. and Ohya, J. (1996). The representation of agents: Anthropomorphism, agency and intelligence. In Proceedings of CHI 1996 Conference Companion, pp. 289-290, Vancouver, B.C.

Digital Library

[14]

Koda, T. (1996). Agents with faces: A study on the effect of personification of software agents. MIT Media Lab, MS thesis.

[15]

Lai, J., Wood, D., and Considine, M. (2000). The effect of task conditions on the comprehensibility of synthetic speech. In Proceedings of the ACM CHI 2000 Conference, pp. 321-328, The Hague, Netherlands.

Digital Library

[16]

Lanier, J. (1995). Agents of alienation. Interactions 2(3): 66-72.

Digital Library

[17]

Laurel, B. (1990). Interface agents: Metaphors with character. In Laurel B., editor, The art of human-computer interface design, pp. 355-365, Addison-Wesley, New York.

[18]

Lyman, P. and Varian, H. (2002). How Much Information?, available at http://www.sims.berkeley.edu/how-much-info/.

[19]

Martin, J.C., Grimard, S., and Alexandri, K. (2001). On the annotation of nmltimodal behavior and the computation of cooperation between modalities. In Proceedings of AAMAS 2001 Worksoh on Representing, Annotating, and Evaluating Nonverbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents, pp. 1-7, Montreal, Canada, 2001.

[20]

Mayer, R.E., Moreno, R., Boire, M., and Vagge, S. (1999). Maximizing constructivist learning from multimedia communications by minimizing cognitive load. Journal of Educational Psychology, 91(4): 638-643.

[21]

McBreen, H.M., Shade, P. Jack, M.A., and Wyard, P.J. (2000). Experimental Assessment of the Effectiveness of Synthetic Personae for Multi-Modal E-Retail Applications. Proceedings of Fourth International Conference on Autonomous Agents, pp. 39-45, Barcelona, Spain.

Digital Library

[22]

McCrae, R. and Costa P. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52(1): 81-90.

[23]

Moon. Y. and Nass, C. (1996). How 'real' are computer personalities? Psychological responses to personality types in human-computer interaction. Communication Research, 23(6): 651-674.

[24]

Nass, C., Steuer, J., and Tauber, E. (1994). Computers are social actors. In Proceedings of the ACM CHI 1994 Conference, pp. 72-78, Boston, MA.

Digital Library

[25]

Nass, C., Lombard, M., Henriksen. L., and Steuer J. (1995). Anthropocentrism and computers. Behaviour and Information Technology, 14(4): 229-238.

[26]

Nass, C. and Lee, K.M. (2000). Does computer-generated speech manifest personality? An experimental test of similarity-attraction. In Proceedings of the ACM CHI 2000 Conference, pp. 329-336, The Hague, Netherlands.

Digital Library

[27]

Shneiderman, B. and Maes, P. (1997). Direct manipulation vs. interface agents, Interactions, 4(6): 42-61.

Digital Library

[28]

Shneiderman, B. (1997). Direct manipulation versus agents: Paths to predictable, controllable, and comprehensible interfaces. In Bradshaw J.M., editor, Software Agents, pp. 97-106, AAAI Press/The MIT Press, Menlo Park, CA.

Digital Library

[29]

Rickenberg, R. and Reeves, B. (2000). The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. In Proceedings of the ACM CHI 2000 Conference, pp. 329-336, The Hague, Netherlands.

Digital Library

[30]

Ruttkay, Z., Dormann, C., and Noot, H. (2002). Evaluating ECAs - What and How?. In Proceedings of AAMAS 2002 Workshop on Embodied Conversational Agents -- Let's Specify and Evaluate Them!, Bologna, Italy.

[31]

Sanders, G. and Scholtz, J. (2000). Measurement and Evaluation of Conversational Agents. In Cassell J., Sullivan J., Prevost S., Churchill E., editors, Embodied Conversational Agents, pp. 346-373, MIT Press, Cambridge, MA.

Digital Library

[32]

Sculley, J. (1989). The Relationship Between Business and Higher Education: a Perspective on the 21st Century. Communications of the ACM, 32(9): 1056-1061.

Digital Library

[33]

Sweller, J., van Merrienboer, J.J.G., and Paas, F.G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3): 251-296.

[34]

Takeuchi A. and Naito, T. (1995). Situated facial displays: Towards social interaction. In Proceedings of the ACM CHI 1995 Conference, pp. 450-455, Denver, CO.

Digital Library

[35]

Van Mulken, S., Andre, E., and Muller, J. (1998). The Persona Effect: How substantial is it?. In Proceedings of the ACM CHI 1998 Conference , pp. 53-66. Los Angeles, CA.

Digital Library

[36]

Walker, J.H., Sproull, L., and Subramani, R. (1994). Using a human face in an interface. In Proceedings of the ACM CHI 1994 Conference, pp. 85-91, Boston, MA.

Digital Library

[37]

Xiao, J., Catrambone, R., and Stasko, J., (2003). Be Quiet? Evaluating Proactive and Reactive User Interface Assistants. In Proceedings of INTERACT'03, pp. 383-390, Zurich, Switzerland.

Cited By

Jensen TKhan M(2022)I’m Only Human: The Effects of Trust Dampening by Anthropomorphic AgentsHCI International 2022 – Late Breaking Papers: Interacting with eXtended Reality and Artificial Intelligence10.1007/978-3-031-21707-4_21(285-306)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-21707-4_21
Chin HYi MBrewster SFitzpatrick GCox AKostakos V(2019)Should an Agent Be Ignoring It?Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290607.3312826(1-6)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290607.3312826
Lisetti CAmini RYasavur URishe N(2013)I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health InterventionsACM Transactions on Management Information Systems10.1145/25441034:4(1-28)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1145/2544103
Show More Cited By

Index Terms

ECA as user interface paradigm
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
      1. Intelligent agents
2. Human-centered computing
  1. Collaborative and social computing
  2. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

The oz of wizard: simulating the human for interaction research
HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction

The Wizard of Oz experiment method has a long tradition of acceptance and use within the field of human-robot interaction. The community has traditionally downplayed the importance of interaction evaluations run with the inverse model: the human ...
Evaluation of multimodal behaviour of embodied agents
From brows to trust

Individuality of Embodied Conversational Agents (ECAs) may depend on both the look of the agent and the way it combines different modalities such as speech and gesture. In this chapter, we describe a study in which male and female users had to listen to ...
The design of a generic framework for integrating ECA components
AAMAS '08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1

Embodied Conversational Agents (ECAs) are life-like computer generated characters that interact with human users in face-to-face multi-modal conversations. ECA systems are generally complex and difficult for individual research groups to develop. ...

Comments

Information & Contributors

Information

Published In

cover image Guide books

From brows to trust: evaluating embodied conversational agents

January 2004

352 pages

ISBN:140202729X

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 January 2004

Author Tags

Qualifiers

Chapter

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jensen TKhan M(2022)I’m Only Human: The Effects of Trust Dampening by Anthropomorphic AgentsHCI International 2022 – Late Breaking Papers: Interacting with eXtended Reality and Artificial Intelligence10.1007/978-3-031-21707-4_21(285-306)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-21707-4_21
Chin HYi MBrewster SFitzpatrick GCox AKostakos V(2019)Should an Agent Be Ignoring It?Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290607.3312826(1-6)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290607.3312826
Lisetti CAmini RYasavur URishe N(2013)I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health InterventionsACM Transactions on Management Information Systems10.1145/25441034:4(1-28)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1145/2544103
Van den Bosch KBrandenburgh AMuller THeuvelink A(2012)Characters with personality!Proceedings of the 12th international conference on Intelligent Virtual Agents10.1007/978-3-642-33197-8_44(426-439)Online publication date: 12-Sep-2012
https://dl.acm.org/doi/10.1007/978-3-642-33197-8_44
Stevens CGibert GLeung YZhang Z(2011)A flexible dual task paradigm for evaluating an embodied conversational agentProceedings of the 10th international conference on Intelligent virtual agents10.5555/2041666.2041708(331-337)Online publication date: 15-Sep-2011
https://dl.acm.org/doi/10.5555/2041666.2041708
Smyth TEtherton JBest MMynatt EFitzpatrick GHudson SEdwards KRodden T(2010)MOSESProceedings of the SIGCHI Conference on Human Factors in Computing Systems10.1145/1753326.1753484(1059-1068)Online publication date: 10-Apr-2010
https://dl.acm.org/doi/10.1145/1753326.1753484
Vugt HBailenson JHoorn JKonijn E(2008)Effects of facial similarity on user responses to embodied agentsACM Transactions on Computer-Human Interaction10.1145/1746259.174626117:2(1-27)Online publication date: 21-May-2008
https://dl.acm.org/doi/10.1145/1746259.1746261
Park SCatrambone R(2007)Understanding the social relationship between humans and virtual humansProceedings of the 12th international conference on Human-computer interaction: intelligent multimodal interaction environments10.5555/1769590.1769641(459-464)Online publication date: 22-Jul-2007
https://dl.acm.org/doi/10.5555/1769590.1769641
van Vugt HKonijn EHoorn JKeur IEliëns A(2007)Realism is not all! User engagement with task-related interface charactersInteracting with Computers10.1016/j.intcom.2006.08.00519:2(267-280)Online publication date: 1-Mar-2007
https://dl.acm.org/doi/10.1016/j.intcom.2006.08.005

View Options

View options

Media

Figures

Other

Tables

View Table of Contents