Cited By
View all- Kim TMejia S(2019)From Artificial Intelligence to Artificial Wisdom: What Socrates Teaches UsComputer10.1109/MC.2019.292972352:10(70-74)Online publication date: 24-Sep-2019
NADiA - Towards Neural Network Driven Virtual Human Conversation Agents
Authors: 
Jason Wu, 
Sayan Ghosh, Mathieu Chollet, Steven Ly, Sharon Mozgai, Stefan SchererAuthors Info & Claims
Jason Wu, Sayan Ghosh, Mathieu Chollet, Steven Ly, Sharon Mozgai, Stefan SchererAuthors Info & ClaimsPages 2262 - 2264
Abstract
Advances in artificial intelligence and machine learning - in particular neural networks - have given rise to a new generation of virtual assistants and chatbots. Within this work, we describe the motivation and architecture of NADiA - Neurally Animated Dialog Agent - which leverages both the user's verbal input and facial expressions for multi-modal conversation. NADiA combines a neural language model that generates conversational responses, a convolutional neural network for facial expression analysis, and virtual human technology that is deployed on a mobile phone.
References
[1]
Timothy W Bickmore and Rosalind W Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction (TOCHI), Vol. 12, 2 (2005), 293--327.
[2]
Justine Cassell. 2000. Embodied conversational agents. MIT press.
[3]
Jonathan Chang and Stefan Scherer. 2017. Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks. arXiv preprint arXiv:1705.02394 (2017).
[4]
Robert Dale. 2016. The return of the chatbots. Natural Language Engineering Vol. 22, 5 (2016), 811--817.
[5]
Asbjørn Følstad and Petter Bae Brandtzæg. 2017. Chatbots and the new world of HCI. interactions, Vol. 24, 4 (2017), 38--42.
[6]
Sayan Ghosh, Mathieu Chollet, Eugene Laksana, Louis-Philippe Morency, and Stefan Scherer. 2017. Affect-LM: A Neural Language Model for Customizable Affective Text Generation. arXiv preprint arXiv:1704.06851 (2017).
[7]
Sayan Ghosh, Eugene Laksana, Louis-Philippe Morency, and Stefan Scherer. 2016. Representation Learning for Speech Emotion Recognition. INTERSPEECH. 3603--3607.
[8]
Sayan Ghosh, Eugene Laksana, Stefan Scherer, and Louis-Philippe Morency. 2015. A multi-label convolutional neural network approach to cross-domain action unit detection Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on. IEEE, 609--615.
[9]
Mark L Knapp, Judith A Hall, and Terrence G Horgan. 2013. Nonverbal communication in human interaction. Cengage Learning.
[10]
Stefan Kopp, Brigitte Krenn, Stacy Marsella, Andrew N Marshall, Catherine Pelachaud, Hannes Pirker, Kristinn R Thórisson, and Hannes Vilhjálmsson. 2006. Towards a common framework for multimodal generation: The behavior markup language International workshop on intelligent virtual agents. Springer, 205--217.
[11]
Gale M Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior Vol. 37 (2014), 94--100.
[12]
Albert Mehrabian. 1972. Nonverbal communication. Transaction Publishers.
[13]
Sharon Mozgai, Gale Lucas, and Jonathan Gratch. 2017. To Tell the Truth: Virtual Agents and Morning Morality International Conference on Intelligent Virtual Agents. Springer, 283--286.
[14]
Najmeh Sadoughi and Carlos Busso. 2017. Joint learning of speech-driven facial motion with bidirectional long-short term memory International Conference on Intelligent Virtual Agents. Springer, 389--402.
[15]
Ari Shapiro. 2011. Building a character animation system. Motion in Games (2011), 98--109.
[16]
Joseph Weizenbaum. 1966. ELIZA - a computer program for the study of natural language communication between man and machine. Commun. ACM Vol. 9, 1 (1966), 36--45.
[17]
Joseph Weizenbaum. 1976. Computer power and human reason: From judgment to calculation. (1976).
[18]
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014).
[19]
Geoffrey Zweig, Chengzhu Yu, Jasha Droppo, and Andreas Stolcke. 2017. Advances in all-neural speech recognition. In Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on. IEEE, 4805--4809.
Index Terms
- NADiA - Towards Neural Network Driven Virtual Human Conversation Agents
Recommendations
NADiA: Neural Network Driven Virtual Human Conversation Agents
IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual AgentsAdvances in artificial intelligence and in particular machine learning and neural networks have given rise to a new generation of virtual assistants and chatbots. Within this work, we present NADiA - Neurally Animated Dialog Agent - that leverages both ...
Toward RNN Based Micro Non-verbal Behavior Generation for Virtual Listener Agents
Social Computing and Social Media. Design, Human Behavior and AnalyticsAbstractThis work aims to develop a model to generate fine grained and reactive non-verbal idling behaviors of a virtual listener agent when a human user is talking to it. The target micro behaviors are facial expressions, head movements, and postures. ...
Single or Multiple Conversational Agents?: An Interactional Coherence Comparison
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsChatbots focusing on a narrow domain of expertise are in great rise. As several tasks require multiple expertise, a designer may integrate multiple chatbots in the background or include them as interlocutors in a conversation. We investigated both ...
Comments
Information & Contributors
Information
Published In
July 2018
2312 pages
- General Chairs:
- Elisabeth Andre,
- Sven Koenig,
- Program Chairs:
- Mehdi Dastani,
- Gita Sukthankar
Sponsors
In-Cooperation
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Richland, SC
Publication History
Published: 09 July 2018
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- U.S. Army Research Laboratory
Conference
AAMAS '18
Sponsor:
Acceptance Rates
AAMAS '18 Paper Acceptance Rate 149 of 607 submissions, 25%;
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 256Total Downloads
- Downloads (Last 12 months)7
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Oct 2024
Other Metrics
Citations
Cited By
View all- Kim TMejia S(2019)From Artificial Intelligence to Artificial Wisdom: What Socrates Teaches UsComputer10.1109/MC.2019.292972352:10(70-74)Online publication date: 24-Sep-2019
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in