Article

Lessons learned in modeling schizophrenic and depressed responsive virtual humans for training

Authors:

Robert C. Hubal,

Geoffrey A. Frank,

Curry I. GuinnAuthors Info & Claims

IUI '03: Proceedings of the 8th international conference on Intelligent user interfaces

Pages 85 - 92

https://doi.org/10.1145/604045.604062

Published: 12 January 2003 Publication History

Abstract

This paper describes lessons learned in developing the linguistic, cognitive, emotional, and gestural models underlying virtual human behavior in a training application designed to train civilian police officers how to recognize gestures and verbal cues indicating different forms of mental illness and how to verbally interact with the mentally ill. Schizophrenia, paranoia, and depression were all modeled for the application. For linguistics, the application has quite complex language grammars that captured a range of syntactic structures and semantic categories. For cognition, there is a great deal of augmentation to a plan-based transition network needed to model the virtual humans knowledge. For emotions and gestures, virtual human behavior is based on expert-validated mapping tables specific to each mental illness. The paper presents five areas demanding continued research to improve virtual human behavior for use in training applications

References

[1]

André, E., Klesen, M., Gebhard, P., Allen, S., & Rist, T. (2000). Exploiting Models of Personality and Emotions to Control the Behavior of Animated Interface Agents. In J. Rickel et al. (Eds.), Proceedings of the Fourth International Conference on Autonomous Agents (pp. 3--7), Barcelona, June 2000.

[2]

André, E., Rist, T., & Müüller, J. (1999). Employing AI Methods to Control the Behavior of Animated Interface Agents. International Journal of Applied Artificial Intelligence, 13(4-5), 415--448.

[3]

Bécheiraz, P., & Thalmann, D. (1998). A Behavioral Animation System for Autonomous Actors Personified by Emotions. Proceedings of the First Workshop on Embodied Conversational Characters, Lake Tahoe, CA.

[4]

Berenbaum, H., & Oltmanns, T.F. (1992). Emotional Experience and Expression in Schizophrenia and Depression. Journal of Abnormal Psychology, 101(1), 37--44.

[5]

Cassell, J., Bickmore, T., Vilhjálmsson, H., & Yan, H. (2000). More Than Just a Pretty Face: Affordances of Embodiment. Proceedings of 2000 International Conference on Intelligent User Interfaces, New Orleans, LA.

Digital Library

[6]

Cassell, J., Pelachaud, C., Badler, N.I., Steedman, M., Achorn, B., Beckett, T., Douville, B., Prevost, S., & Stone, M. (1994). Animated Conversation: Rule-based Generation of Facial Display, Gesture and Spoken Intonation for Multiple Conversational Agents. Computer Graphics, 28(4), 413-420.

Digital Library

[7]

Cassell, J., & Thóórisson, K.R. (1999). The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents. International Journal of Applied Artificial Intelligence, 13(4-5), 519--538.

[8]

Cassell, J., & Vilhjáálmsson, H.H. (1999). Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous. Autonomous Agents and Multi-Agent Systems, 2, 45--64.

Digital Library

[9]

Chovil, N. (1992). Discourse-Oriented Facial Displays in Conversation. Research on Language and Social Interaction, 25, 163--194.

[10]

Elliott, C. (1993). Using the Affective Reasoner to Support Social Simulations. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (pp. 194--200), Chambery, France, August 1993. Morgan Kaufmann.

[11]

Fehr, B., & Russell, J.A. (1984). Concept of Emotion Viewed From a Prototype Perspective. Journal of Experimental Psychology: General, 113(3), 464-486.

[12]

Frank, G.A., Guinn, C.I., Hubal, R.C., Stanford, M.A., Pope, P., & Lamm-Weisel, D. (2002). JUST-TALK: An Application of Responsive Virtual Human Technology. In Proceedings of I/ITSEC, Orlando, FL.

[13]

Funge, J., Tu, X., & Terzopoulos, D. (1999). Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters. Computer Graphics Proceedings, ACM SIGGRAPH: 29--38.

Digital Library

[14]

Groves, R., & Couper, M. (1998). Nonresponse in Household Interview Surveys. New York: John Wiley & Sons, Inc.

[15]

Gunther-Mohr, C. (1998). Virtual Reality Training-Is It For You? Talk presented at the American Society for Training and Development International Conference, San Francisco, June 1998.

[16]

Guinn, C.I. (1996). Mechanisms for Mixed-Initiative Human-Computer Collaborative Discourse. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics.

Digital Library

[17]

Guinn, C.I., & Montoya, R.J. (1998). Natural Language Processing in Virtual Reality. Modern Simulation & Training, 6, 44--45.

[18]

Guye-Vuillième, A., Capin, T.K., Pandzic, I.S., Magnenat Thalmann, N., & Thalmann, D. (1999) Non-verbal Communication Interface for Collaborative Virtual Environments. The Virtual Reality Journal, Springer.

[19]

Hubal, R.C., & Frank, G.A. (2001). Interactive Training Applications using Responsive Virtual Human Technology. Proceedings of I/ITSEC, Orlando, FL, November 2001.

[20]

Hubal, R.C., Frank, G.A., & Guinn, C.I. (2000). AVATALK Virtual Humans for Training with Computer Generated Forces. Proceedings of CGF-BR. Institute for Simulation & Training: Orlando FL.

[21]

Hubal, R.C., Kizakevich, P.N., Guinn, C.I., Merino, K.D., & West, S.L. (2000). The Virtual Standardized Patient-Simulated Patient-Practitioner Dialogue for Patient Interview Training. In J.D. Westwood, H.M. Hoffman, G.T. Mogel, R. A. Robb, & D. Stredney (Eds.), Medicine Meets Virtual Reality. IOS Press and Ohmsha, Amsterdam, 133--138.

[22]

Johnson, W.L., Rickel, J.W., & Lester, J.C. (2000). Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments. International Journal of Artificial Intelligence in Education, 11, 47--78.

[23]

Kizakevich, P.N., McCartney, M.L., Nissman, D.B., Starko, K., & Smith, N.T. (1998). Virtual Medical Trainer: Patient Assessment and Trauma Care Simulator. Medicine Meets Virtual Reality-Art, Science, Technology: Healthcare (R)evolution, J.D. Westwood, H.M. Hoffman, D. Stredney, & S.J. Weghorst (Eds.), 309--315. IOS Press and Ohmsha: Amsterdam.

[24]

Klein, G. (1998). Sources of Power. Cambridge, MA: MIT Press.

[25]

Kline, C., & Blumberg, B. (1999). The Art and Science of Synthetic Character Design. Proceedings of the Symposium on AI and Creativity in Entertainment and Visual Art, Edinburgh.

[26]

Knapp, M.L., & Hall, J.A. (1997). Nonverbal Communication in Human Interaction (3rd ed.). Fort Worth: Harcourt Brace.

[27]

Krause, R., Steimer, E., Sänger-Alt, C., & Wagner, G. (1989). Facial Expression of Schizophrenic Patients and Their Interactions Partners. Psychiatry, 52, 1--11.

[28]

Lester, J.C., Stone, B.A., & Stelling, G.D. (1999). Lifelike Pedagogical Agents for Mixed-Initiative Problem Solving in Constructivist Learning Environments. User Modeling and User-Adapted Interaction, 9(1-2), 1--44.

Digital Library

[29]

Loyall, A.B., & Bates, J. (1997). Personality–Rich Believable Agents That Use Language. Proceedings of the First International Conference on Autonomous Agents, Marina del Rey CA.

Digital Library

[30]

Lundeberg, M., & Beskow, J. (1999). Developing a 3D-Agent for the August Dialogue System. AVSP Proceedings, Santa Cruz CA.

[31]

Moore, G. (2001). Talking Heads: Facial Animation in The Getaway. Gamasutra.

[32]

Olsen, D.E. (2001). The Simulation of a Human for Interpersonal Skill Training. In Proceedings of the 2001 Office of National Drug Control Policy (ONDCP) International Technology Symposium, June 25-28, 2001, San Diego CA.

[33]

Ortony, A., Clore, G. L., & Collins, A. (1988). The Cognitive Structure of Emotions. Cambridge: Cambridge University Press.

[34]

Plutchik, R. (1997). The Circumplex as a General Model of the Structure of Emotions and Personality. In R. Plutchik & H.R. Conte (Eds.) Circumplex Models of Personality and Emotions (pp. 17-45). American Psychological Association.

[35]

Pollard, S., & Biermann, A. (2000). A Measure of Semantic Complexity for Natural Language Systems. Workshop on Syntactic and Semantic Complexity in Natural Language Processing Systems. ANLP-NAACL2000.

Digital Library

[36]

Prendinger, H., & Ishizuka, M. (2001). Social Role Awareness in Animated Agents. Fifth International Conference on Autonomous Agents (pp. 270--277), Montreal.

Digital Library

[37]

Ross, K.G., Pierce, L.G., Haltermann, J.A., & Ross, W.A. (1998). Preparing for the Instructional Technology Gap-A Constructivist Approach. Proceedings of I/ITSEC, Orlando FL.

[38]

Rousseau, D., & Hayes-Roth, B. (1997). Improvisational Synthetic Actors with Flexible Personalities. KSL Report #97-10, Stanford University.

[39]

Russell, J.A. (1997). How Shall an Emotion Be Called? In R. Plutchik & H.R. Conte (Eds.), Circumplex Models of Personality and Emotions (pp. 205--220). American Psychological Assn.

[40]

Shalif, I. (1991). The emotions and the dimensions of discrimination among them in daily life. Bar-Ilan University Ramat-Gan, Israel, unpublished dissertation.

[41]

Smith, R.W., & Gordon, S.A. (1997). Effects of Variable Initiative on Linguistic Behavior in Human-Computer Spoken Natural Language Dialog. Computational Linguistics, 23(1).

Digital Library

[42]

Wiggins, J.S., (Ed.) (1996). The Five Factor Model of Personality: Theoretical Perspectives. New York: The Guilford Press.

Cited By

Pence TDukes LHodges L(2016)Animation Validation of Obese Virtual Pediatric Patients Using a FLACC Pain ScaleVirtual, Augmented and Mixed Reality10.1007/978-3-319-39907-2_53(552-564)Online publication date: 19-Jun-2016
https://doi.org/10.1007/978-3-319-39907-2_53
Gupta MJin SSanders GSherman BSimha A(2014)Virtual WorldsInventive Approaches for Technology Integration and Information Resources Management10.4018/978-1-4666-6256-8.ch008(162-212)Online publication date: 2014
https://doi.org/10.4018/978-1-4666-6256-8.ch008
Pence TDukes LHodges LMeehan NJohnson A(2014)An Eye Tracking Evaluation of a Virtual Pediatric Patient Training System for NursesIntelligent Virtual Agents10.1007/978-3-319-09767-1_43(329-338)Online publication date: 2014
https://doi.org/10.1007/978-3-319-09767-1_43
Show More Cited By

Index Terms

Lessons learned in modeling schizophrenic and depressed responsive virtual humans for training

Recommendations

Investigating Social Distances between Humans, Virtual Humans and Virtual Robots in Mixed Reality
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems

Mixed reality environments offer new potentials for the design of compelling social interaction experiences with virtual characters. In this paper, we summarise initial experiments we are conducting in which we measure comfortable social distances ...
Virtual humans: back to the future
GI '12: Proceedings of Graphics Interface 2012

This paper essentially tries to examine all the roles that Virtual Humans can play in empowering human expression, and the research challenges we have to face to make this possible. It starts with a short history of Virtual Humans and how we contribute ...
Extending virtual humans to support team training in virtual reality
Exploring artificial intelligence in the new millennium

This chapter describes the use of virtual humans and distributed virtual reality to support team training, where students must learn their individual role in the team as well as how to coordinate their actions with their teammates. Students, instructors,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '03: Proceedings of the 8th international conference on Intelligent user interfaces

January 2003

344 pages

ISBN:1581135866

DOI:10.1145/604045

Conference Chair:
David Leake
Indiana University
,
Program Chairs:
Lewis Johnson
USC/Information Sciences Institute
,
Elisabeth Andre
University of Augsburg

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 January 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IUI03

Sponsor:

IUI03: Eighth International Conference on Intelligent User Interfaces

January 12 - 15, 2003

Florida, Miami, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,045
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pence TDukes LHodges L(2016)Animation Validation of Obese Virtual Pediatric Patients Using a FLACC Pain ScaleVirtual, Augmented and Mixed Reality10.1007/978-3-319-39907-2_53(552-564)Online publication date: 19-Jun-2016
https://doi.org/10.1007/978-3-319-39907-2_53
Gupta MJin SSanders GSherman BSimha A(2014)Virtual WorldsInventive Approaches for Technology Integration and Information Resources Management10.4018/978-1-4666-6256-8.ch008(162-212)Online publication date: 2014
https://doi.org/10.4018/978-1-4666-6256-8.ch008
Pence TDukes LHodges LMeehan NJohnson A(2014)An Eye Tracking Evaluation of a Virtual Pediatric Patient Training System for NursesIntelligent Virtual Agents10.1007/978-3-319-09767-1_43(329-338)Online publication date: 2014
https://doi.org/10.1007/978-3-319-09767-1_43
Mykoniatis KAngelopoulou AProctor MKarwowski W(2014)Virtual Humans for Interpersonal and Communication Skills' Training in Crime InvestigationsProceedings, Part I, of the 6th International Conference on Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments - Volume 852510.1007/978-3-319-07458-0_27(282-292)Online publication date: 22-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07458-0_27
Gupta MJin SSanders GSherman BSimha A(2012)Getting Real About Virtual WorldsInternational Journal of Virtual Communities and Social Networking10.4018/jvcsn.20120701014:3(1-46)Online publication date: 1-Jul-2012
https://dl.acm.org/doi/10.4018/jvcsn.2012070101
Lane HWray R(2012)Individualized Cultural and Social Skills Learning with Virtual HumansAdaptive Technologies for Training and Education10.1017/CBO9781139049580.015(204-221)Online publication date: 5-Mar-2012
https://doi.org/10.1017/CBO9781139049580.015
Kenny PParsons T(2011)Embodied Conversational Virtual PatientsConversational Agents and Natural Language Interaction10.4018/978-1-60960-617-6.ch011(254-281)Online publication date: 2011
https://doi.org/10.4018/978-1-60960-617-6.ch011
Hubal RMitroff SCain M(2010)Simulating a vigilance task: Extensible technology for baggage security assessment and training2010 IEEE International Conference on Technologies for Homeland Security (HST)10.1109/THS.2010.5654982(543-548)Online publication date: Nov-2010
https://doi.org/10.1109/THS.2010.5654982
Lane HSchneider MMichael SAlbrechtsen JMeissner C(2010)Virtual humans with secretsProceedings of the 10th international conference on Intelligent Tutoring Systems - Volume Part II10.1007/978-3-642-13437-1_15(144-154)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13437-1_15
Hubal RFishbein DSheppard MPaschall MEldreth DHyde C(2008)How do varied populations interact with embodied conversational agents? Findings from inner-city adolescents and prisonersComputers in Human Behavior10.1016/j.chb.2007.03.01024:3(1104-1138)Online publication date: 1-May-2008
https://dl.acm.org/doi/10.1016/j.chb.2007.03.010
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents