Abstract
A key aspect when trying to achieve natural interaction in machines is multimodality. Besides verbal communication, in fact, humans interact also through many other channels, e.g., facial expressions, gestures, eye contact, posture, and voice tone. Such channels convey not only semantics, but also emotional cues that are essential for interpreting the message transmitted. The importance of the affective information and the capability of properly managing it, in fact, has been more and more understood as fundamental for the development of a new generation of emotion-aware applications for several scenarios like e-learning, e-health, and human-computer interaction. To this end, this work investigates the adoption of different paradigms in the fields of text, vocal, and video analysis, in order to lay the basis for the development of an intelligent multimodal affective conversational agent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Cifani, S., Abel, A., Hussain, A., Squartini, S., Piazza, F.: An Investigation into Audiovisual Speech Correlation in Reverberant Noisy Environments. In: Esposito, A., Vích, R. (eds.) Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions. LNCS, vol. 5641, pp. 331–343. Springer, Heidelberg (2009)
Kapoor, A., Burleson, W., Picard, R.: Automatic prediction of frustration. International Journal of Human-Computer Studies 65, 724–736 (2007)
Shan, C., Gong, S., McOwan, P.: Beyond facial expressions: Learning human emotion from body gestures. In: BMVC, Warwick (2007)
Pun, T., Alecu, T., Chanel, G., Kronegg, J., Voloshynovskiy, S.: Brain-computer interaction research at the computer vision and multimedia laboratory. IEEE Trans. on Neural Systems and Rehabilitation Engineering 14(2), 210–213 (2006)
Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley & Sons (2004)
Zeng, Z., Tu, J., Liu, M., Huang, T., Pianfetti, B., Roth, D., Levinson, S.: Audio-visual affect recognition. IEEE Trans. Multimedia 9(2), 424–428 (2007)
Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. Network and Computer Applications 30(4), 1334–1345 (2007)
Pal, P., Iyer, A., Yantorno, R.: Emotion detection from infant facial expressions and cries. In: International Conference on Acoustics, Speech and Signal Processing, Dallas (2006)
Cambria, E., Hussain, A.: Sentic Computing: Techniques, Tools, and Applications. Springer, Dordrecht (2012)
Cambria, E., Benson, T., Eckl, C., Hussain, A.: Sentic PROMs: Application of sentic computing to the development of a novel unified framework for measuring health-care quality. Expert Systems with Applications 39(12), 10533–10543 (2012)
Cambria, E., Song, Y., Wang, H., Hussain, A.: Isanette: A common and common sense knowledge base for opinion mining. In: ICDM, Vancouver, pp. 315–322 (2011)
Cambria, E., Olsher, D., Kwok, K.: Sentic activation: A two-level affective common sense reasoning framework. In: AAAI, Toronto, pp. 186–192 (2012)
Cambria, E., Livingstone, A., Hussain, A.: The hourglass of emotions. In: Esposito, A., et al. (eds.) Cognitive Behavioural Systems. LNCS, vol. 7403, pp. 144–157. Springer, Heidelberg (2012)
Alm, C., Roth, D., Sproat, R.: Emotions from text: Machine learning for text-based emotion prediction. In: HLT/EMNLP, pp. 347–354 (2005)
Lin, W., Wilson, T., Wiebe, J., Hauptmann, A.: Which side are you on? identifying perspectives at the document and sentence levels. In: Conference on Natural Language Learning, pp. 109–116 (2006)
Danisman, T., Alpkocak, A.: Feeler: Emotion classification of text using vector space model. In: AISB (2008)
D’Mello, S., Dowell, N., Graesser, A.: Cohesion relationships in tutorial dialogue as predictors of affective states. In: Conf. Artificial Intelligence in Education, pp. 9–16 (2009)
Cambria, E., Mazzocco, T., Hussain, A., Eckl, C.: Sentic medoids: Organizing affective common sense knowledge in a multi-dimensional vector space. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds.) ISNN 2011, Part III. LNCS, vol. 6677, pp. 601–610. Springer, Heidelberg (2011)
Christian, J., Deeming, A.: Affective human-robotic interaction. In: Affect and Emotion in Human-Computer Interaction: From Theory to Applications (2008)
Petrushin, V.: Emotion in speech: Recognition and application to call centers. In: Conference on Artificial Neural Networks in Engineering, p. 710 (1999)
Navas, E., Hernez, L.: An objective and subjective study of the role of semantics and prosodic features in building corpora for emotional TTS. IEEE Transactions on Audio, Speech, and Language Processing 14, 1117–1127 (2006)
Atassi, H., Esposito, A.: A speaker independent approach to the classification of emotional vocal expressions, pp. 147-152 (2008)
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A database of german emotional speech. In: Interspeech, pp. 1517–1520 (2005)
Pudil, P., Ferri, F., Novovicova, J., Kittler, J.: Floating search method for feature selection with non monotonic criterion functions. Pattern Recognition 2, 279–283 (1994)
Ekman, P., Dalgleish, T., Power, M.: Handbook of Cognition and Emotion. Wiley, Chichester (1999)
Abel, A., Hussain, A., Nguyen, Q.-D., Ringeval, F., Chetouani, M., Milgram, M.: Maximising Audiovisual Correlation with Automatic Lip Tracking and Vowel Based Segmentation. In: Fierrez, J., Ortega-Garcia, J., Esposito, A., Drygajlo, A., Faundez-Zanuy, M. (eds.) BioID MultiComm 2009. LNCS, vol. 5707, pp. 65–72. Springer, Heidelberg (2009)
Whissell, C.: The dictionary of affect in language. Emotion: Theory, Research, and Experience 4, 113–131 (1989)
Grassi, M., Cambria, E., Hussain, A., Piazza, F.: Sentic web: A new paradigm for managing social media affective information. Cognitive Computation 3(3), 480–489 (2011)
Grassi, M.: Developing HEO Human Emotions Ontology. In: Fierrez, J., Ortega-Garcia, J., Esposito, A., Drygajlo, A., Faundez-Zanuy, M. (eds.) BioID MultiComm 2009. LNCS, vol. 5707, pp. 244–251. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hussain, A., Cambria, E., Mazzocco, T., Grassi, M., Wang, QF., Durrani, T. (2012). Towards IMACA: Intelligent Multimodal Affective Conversational Agent. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7663. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34475-6_79
Download citation
DOI: https://doi.org/10.1007/978-3-642-34475-6_79
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34474-9
Online ISBN: 978-3-642-34475-6
eBook Packages: Computer ScienceComputer Science (R0)