research-article

Laughter animation synthesis

Authors:

Catherine Pelachaud,

Thierry ArtièresAuthors Info & Claims

AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems

Pages 773 - 780

Published: 05 May 2014 Publication History

Abstract

Laughter is an important communicative signal in human-human communication. However, very few attempts have been made to model laughter animation synthesis for virtual characters. This paper reports our work to model hilarious laughter. We have developed a generator for face and body motions that takes as input the sequence of pseudo-phonemes of laughter and each pseudo-phoneme's duration time. Lip and jaw movements are further driven by laughter prosodic features. The proposed generator first learns the relationship between input signals (pseudo-phoneme and acoustic features) and human motions; then the learnt generator can be used to produce automatically laughter animation in real time. Lip and jaw motion synthesis is based on an extension of Gaussian Models, the contextual Gaussian Model. Head and eyebrow motion synthesis is based on selecting and concatenating motion segments from motion capture data of human laughter while torso and shoulder movements are driven from head motion by a PD controller. Our multimodal behaviors generator of laughter has been evaluated through perceptive study involving the interaction of a human and an agent telling jokes to each other.

References

[1]

V. Adelsward. Laughter and dialogue: The social significance of laughter in institutional discourse. Nordic Journal of Linguistics, 102(12):107--136, 1989.

[2]

P. Boersma and D. Weeninck. Praat, a system for doing phonetics by computer. Glot International, 5(9/10):341--345, 2001.

[3]

C. Busso, Z. Deng, U. Neumann, and S. Narayanan. Natural head motion synthesis driven by acoustic prosodic features. Journal of Visualization and Computer Animation, 16(3--4):283--290, 2005.

Digital Library

[4]

D. Cosker and J. Edge. Laughing, crying, sneezing and yawning: Automatic voice driven animation of non-speech articulations. Proceedings of Computer Animation and Social Agents, pages 21--24, 2009.

[5]

P. C. DiLorenzo, V. B. Zordan, and B. L. Sanders. Laughing out loud: control for modeling anatomically inspired laughter using audio. ACM Trans. Graph., 27(5):125, 2008.

Digital Library

[6]

Y. Ding, C. Pelachaud, and T. Artières. Modeling multimodal behaviors from speech prosody. In IVA, pages 217--228. 2013.

[7]

Y. Ding, M. Radenen, T. Artières, and C. Pelachaud. Speech-driven eyebrow motion synthesis with contextual markovian models. In ICASSP, pages 3756--3760, 2013.

[8]

S. Mariooryad and C. Busso. Generating human-like behaviors using joint, speech-driven models for conversational agents. IEEE Trans. on Audio, Speech & Language Processing, 20(8):2329--2340, 2012.

Digital Library

[9]

R. Martin. Is laughter the best medicine? humor, laughter, and physical health. Current Directions in Psychological Science, 11(6):216--220, 2002.

[10]

M. Neff and E. Fiume. Modeling tension and relaxation for computer animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA '02.

Digital Library

[11]

R. Niewiadomski, J. Hofmann, J. Urbain, T. Platt, J. Wagner, B. PIOT, H. Cakmak, S. Pammi, T. Baur, S. Dupont, M. Geist, F. Lingenfelser, G. McKeown, O. Pietquin, and W. Ruch. Laugh-aware virtual agent and its impact on user amusement . In AAMAS, pages 619--626, 2013.

Digital Library

[12]

R. Niewiadomski and C. Pelachaud. Towards multimodal expression of laughter. In IVA, pages 231--244, 2012.

Digital Library

[13]

R. Niewiadomski, J. Urbain, C. Pelachaud, and T. Dutoit. Finding out the audio and visual features that influence the perception of laughter intensity and differ in inhalation and exhalation phases. In International Workshop on Corpora for Research on EMOTION SENTIMENT and SOCIAL SIGNALS, LREC 2012.

[14]

M. Ochs and C. Pelachaud. Model of the perception of smiling virtual character. In AAMAS, pages 87--94, 2012.

Digital Library

[15]

I. Pandzic and R. Forcheimer. MPEG4 Facial Animation - The standard, implementations and applications. John Wiley & Sons, 2002.

Digital Library

[16]

K. Perlin. Improving noise. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, SIGGRAPH '02, pages 681--682.

[17]

R. Provine. Laughter. American Scientist, 84(1):38--47, 1996.

[18]

W. Ruch and P. Ekman. The Expressive Pattern of Laughter. Emotion qualia, and consciousness, pages 426--443, 2001.

[19]

J. Saragih, S. Lucey, and J. Cohn. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2):200--215, 2011.

Digital Library

[20]

K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura. Speech parameter generation algorithms for hmm-based speech synthesis. In ICASSP, pages 1315--1318, 2000.

[21]

J. Urbain, E. Bevacqua, T. Dutoit, A. Moinet, R. Niewiadomski, C. Pelachaud, B. Picart, J. Tilmanne, and J. Wagner. The avlaughtercycle database. In LREC, 2010.

[22]

J. Urbain, H. Çakmak, and T. Dutoit. Automatic phonetic transcription of laughter and its application to laughter synthesis. In biannual Humaine Association Conference on Affective Computing and Intelligent Interaction, pages 153--158, 2013.

Digital Library

Cited By

Oralbayeva NAly ASandygulova ABelpaeme T(2023)Data-Driven Communicative Behaviour Generation: A SurveyACM Transactions on Human-Robot Interaction10.1145/3609235Online publication date: 16-Aug-2023
https://dl.acm.org/doi/10.1145/3609235
Pelachaud CBusso CHeylen D(2021)Multimodal Behavior Modeling for Socially Interactive AgentsThe Handbook on Socially Interactive Agents10.1145/3477322.3477331(259-310)Online publication date: 10-Sep-2021
https://dl.acm.org/doi/10.1145/3477322.3477331
Cafaro APelachaud CMarsella S(2019)Nonverbal behavior in multimodal performancesThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233803(219-262)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3233795.3233803
Show More Cited By

Index Terms

Laughter animation synthesis
1. Computing methodologies
  1. Computer graphics
    1. Animation
    2. Graphics systems and interfaces
      1. Virtual reality
  2. Modeling and simulation
    1. Simulation types and techniques
      1. Simulation by animation
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Recommendations

Towards multimodal expression of laughter
IVA'12: Proceedings of the 12th international conference on Intelligent Virtual Agents

Laughter is a strong social signal in human-human and human-machine communication. However, very few attempts to model it exist. In this paper we discuss several challenges regarding the generation of laughs. We focus, more particularly, on two aspects ...
Conversational speech synthesis and the need for some laughter

This paper reports progress in the synthesis of conversational speech, from the viewpoint of work carried out on the analysis of a very large corpus of expressive speech in normal everyday situations. With recent developments in concatenative techniques,...
Laughter entrainment in dyadic interactions: Temporal distribution and form
Abstract
It has been established across a wide range of communicative behaviours that conversational partners tend to become more similar during their interaction. This phenomenon, often called entrainment, has been shown to take place not only ...
Highlights
- Laughter entrainment examined across languages: French, German and Mandarin Chinese.

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems

May 2014

1774 pages

ISBN:9781450327381

General Chairs:
Ana Bazzan
UFRGS, Brazil
,
Michael Huhns
University of South Carolina, USA
,
Program Chairs:
Alessio Lomuscio
Imperial College London, UK
,
Paul Scerri
Carnegie Mellon University, USA

Sponsors

IFAAMAS

In-Cooperation

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 05 May 2014

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AAMAS '14

Sponsor:

AAMAS '14: International conference on Autonomous Agents and Multi-Agent Systems

May 5 - 9, 2014

Paris, France

Acceptance Rates

AAMAS '14 Paper Acceptance Rate 169 of 709 submissions, 24%;

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
143
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Oralbayeva NAly ASandygulova ABelpaeme T(2023)Data-Driven Communicative Behaviour Generation: A SurveyACM Transactions on Human-Robot Interaction10.1145/3609235Online publication date: 16-Aug-2023
https://dl.acm.org/doi/10.1145/3609235
Pelachaud CBusso CHeylen D(2021)Multimodal Behavior Modeling for Socially Interactive AgentsThe Handbook on Socially Interactive Agents10.1145/3477322.3477331(259-310)Online publication date: 10-Sep-2021
https://dl.acm.org/doi/10.1145/3477322.3477331
Cafaro APelachaud CMarsella S(2019)Nonverbal behavior in multimodal performancesThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233803(219-262)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3233795.3233803
akmak HDutoit T(2018)HMM-based generation of laughter facial expressionSpeech Communication10.1016/j.specom.2017.12.00698:C(28-41)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1016/j.specom.2017.12.006
Mancini MBiancardi BPecune FVarni GDing YPelachaud CVolpe GCamurri A(2017)Implementing and Evaluating a Laughing Virtual CharacterACM Transactions on Internet Technology10.1145/299857117:1(1-22)Online publication date: 25-Feb-2017
https://dl.acm.org/doi/10.1145/2998571
El Haddad KÇakmak HGilmartin EDupont SDutoit TNakano YAndré ENishida TMorency LBusso CPelachaud C(2016)Towards a listening agent: a system generating audiovisual laughs and smiles to show interestProceedings of the 18th ACM International Conference on Multimodal Interaction10.1145/2993148.2993182(248-255)Online publication date: 31-Oct-2016
https://dl.acm.org/doi/10.1145/2993148.2993182
Wang QArtières TDing YManitsaris S(2016)Learning Activity Patterns Performed With EmotionProceedings of the 3rd International Symposium on Movement and Computing10.1145/2948910.2948958(1-4)Online publication date: 5-Jul-2016
https://dl.acm.org/doi/10.1145/2948910.2948958
Pecune FMancini MBiancardi BVarni GDing YPelachaud CVolpe GCamurri AWeiss GYolum PBordini RElkind E(2015)Laughing with a Virtual AgentProceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems10.5555/2772879.2773452(1817-1818)Online publication date: 4-May-2015
https://dl.acm.org/doi/10.5555/2772879.2773452
Niewiadomski RMancini MDing YPelachaud CVolpe GAli Salah ACohn JSchuller BAran OMorency LCohen P(2014)Rhythmic Body Movements of LaughterProceedings of the 16th International Conference on Multimodal Interaction10.1145/2663204.2663240(299-306)Online publication date: 12-Nov-2014
https://dl.acm.org/doi/10.1145/2663204.2663240

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents