Abstract
INTERFACE is an integrated software implemented in Matlab© and created to speed-up the procedure for building an emotive/expressive talking head. Various processing tools, working on dynamic articulatory data physically extracted by an optotracking 3D movement analyzer called ELITE, were implemented to build the animation engine and also to create the correct WAV and FAP files needed for the animation. By the use of INTERFACE, LUCIA, our animated MPEG-4 talking face, can copy a real human by reproducing the movements of passive markers positioned on his face and recorded by an optoelectronic device, or can be directly driven by an emotional XML tagged input text, thus realizing a true audio/visual emotive/expressive synthesis. LUCIA’s voice is based on an Italian version of FESTIVAL – MBROLA packages, modified for expressive/emotive synthesis by means of an appropriate APML/VSML tagged language.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Cosi, P., Tesser, F., Gretter, R., Avesani, C.: Festival Speaks Italian! In: Proc. Eurospeech 2001, Aalborg, Denmark, September 3-7, pp. 509–512 (2001)
Ferrigno, G., Pedotti, A.: ELITE: A Digital Dedicated Hardware System for Movement Analysis via Real-Time TV Signal Processing. IEEE Trans. on Biomedical Engineering BME-32, 943–950 (1985)
Boersma, P.: PRAAT, a system for doing phonetics by computer. Glot International 5(9/10), 341–345 (1996)
Magno Caldognetto, E., Cosi, P., Drioli, C., Tisato, G., Cavicchio, F.: Coproduction of Speech and Emotions: Visual and Acoustic Modifications of Some Phonetic Labial Targets. In: Proc. AVSP 2003, Audio Visual Speech Processing, ISCA Workshop, St Jorioz, France, September 4-7, pp. 209–214 (2003)
Drioli, C., Tisato, G., Cosi, P., Tesser, F.: Emotions and Voice Quality: Experiments with Sinusoidal Modeling. In: Proceedings of Voqual 2003, Voice Quality: Functions, Analysis and Synthesis, ISCA Workshop, Geneva, Switzerland, August 27-29, pp. 127–132 (2003)
Cosi, P., Fusaro, A., Tisato, G.: LUCIA a New Italian Talking-Head Based on a Modified Cohen-Massaro’s Labial Coarticulation Model. In: Proc. Eurospeech 2003,, Geneva, Switzerland, pp. 127–132 (2003)
MPEG-4 standard. Home page, http://www.chiariglione.org/mpeg/index.htm
Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologist Press Inc., Palo Alto (1978)
Cohen, M., Massaro, D.: Modeling Coarticulation in Synthetic Visual Speech. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Models and Techniques in Computer Animation, pp. 139–156. Springer, Tokyo (1993)
Tiede, M.K., Vatikiotis-Bateson, E., Hoole, P., Yehia, H.: Magnetometer data acquisition and analysis software for speech production research. ATR Technical Report TRH 1999, ATR Human Information Processing Labs, Japan (1999)
Cosi, P., Hosom, J.P.: High Performance ‘General Purpose’ Phonetic Recognition for Italian. In: Proc. of ICSLP 2000, Beijing, China, vol. II, pp. 527–530 (2000)
Soderkvist, I., Wedin, P.: Determining the movements of the skeleton using well-configured markers. Journal of Biomechanics 26, 1473–1477 (1993)
Pelachaud, C., Magno Caldognetto, E., Zmarich, C., Cosi, P.: Modelling an Italian Talking Head. In: Proc. AVSP 2001, Aalborg, Denmark, September 7-9, pp. 72–77 (2001)
Cosi, P., Magno Caldognetto, E., Perin, G., Zmarich, C.: Labial Coarticulation Modeling for Realistic Facial Animation. In: Proc. 4th IEEE International Conference on Multimodal Interfaces ICMI 2002, Pittsburgh, PA, USA, pp. 505–510 (2000)
Perin, G.: Facce parlanti: sviluppo di un modello coarticolatorio labiale per un sistema di sintesi bimodale, MThesis, Univ. of Padova, Italy (2000-1)
De Carolis, B., Pelachaud, C., Poggi, I., Steedman, M.: APML, a Mark-up Language for Believable Behavior Generation. In: Prendinger, H., Ishizuka, M. (eds.) Life-Like Characters, pp. 65–85. Springer, Heidelberg (2004)
Pasquariello, S., Pelachaud, C.: Greta: A Simple Facial Animation Engine. In: 6th Online World Conference on Soft Computing in Industrial Appications, Session on Soft Computing for Intelligent 3D Agents (September 2001)
Ruttkay, Z., Noot, H., ten Hagen, P.: Emotion Disc and Emotion Squares: tools to explore the facial expression space. Computer Graphics Forum 22(1), 49–53 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cosi, P., Drioli, C., Tesser, F., Tisato, G. (2005). INTERFACE Toolkit: A New Tool for Building IVAs. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T. (eds) Intelligent Virtual Agents. IVA 2005. Lecture Notes in Computer Science(), vol 3661. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550617_7
Download citation
DOI: https://doi.org/10.1007/11550617_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28738-4
Online ISBN: 978-3-540-28739-1
eBook Packages: Computer ScienceComputer Science (R0)