Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3319502.3374807acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
research-article

MoveAE: Modifying Affective Robot Movements Using Classifying Variational Autoencoders

Published: 09 March 2020 Publication History

Abstract

We propose a method for modifying affective robot movements using neural networks. Social robots use gestures and other movements to express their internal states. However, a robot's interactive capabilities are hindered by the predominant use of a limited set of preprogrammed or hand-animated behaviors, which can be repetitive and predictable, making sustained human-robot interactions difficult to maintain. To address this, we developed a method for modifying existing emotive robot movements by using neural networks. We use hand-crafted movement samples and a classifying variational autoencoder trained on these samples. Our method then allows for adjustment of affective movement features by using simple arithmetic in the network's latent embedding space. We present the implementation and evaluation of this approach and show that editing in the latent space can modify the emotive quality of the movements while preserving recognizability and legibility in many cases. This supports neural networks as viable tools for creating and modifying expressive robot behaviors.

Supplementary Material

MP4 File (fp1225vf.mp4)
Supplemental video
MP4 File (p481-suguitan.mp4)

References

[1]
Aris Alissandrakis, Chrystopher L Nehaniv, and Kerstin Dautenhahn. 2007. Correspondence Mapping Induced State and Action Metrics for Robotic Imitation. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37, 2 (April 2007), 299--307. https://doi.org/10.1109/TSMCB.2006.886947
[2]
Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems 57, 5 (2009), 469--483. https://doi.org/10.1016/j.robot.2008.10.024
[3]
François Chollet. 2015. Keras. https://github.com/fchollet/keras.
[4]
Ruta Desai, Fraser Anderson, Justin Matejka, Stelian Coros, James McCann, George Fitzmaurice, and Tovi Grossman. 2019. Geppetto: Enabling Semantic Design of Expressive Robot Behaviors. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 369, 14 pages. https://doi.org/10.1145/3290605.3300599
[5]
Dongkeon Lee, Kyo-Joong Oh, and Ho-Jin Choi. 2017. The chatbot feels you - a counseling service using emotional response generation. In 2017 IEEE International Conference on Big Data and Smart Computing (BigComp). 437--440. https://doi.org/10.1109/BIGCOMP.2017.7881752
[6]
Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion 6, 3--4 (1992), 169--200. https://doi.org/10.1080/02699939208411068
[7]
Rachel Gockley, Allison Bruce, Jodi Forlizzi, Marek Michalowski, Anne Mundell, Stephanie Rosenthal, Brennan Sellner, Reid Simmons, Kevin Snipes, Alan C Schultz, et al. 2005. Designing robots for long-term social interaction. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems. 1338--1343. https://doi.org/10.1109/IROS.2005.1545303
[8]
Minoru Hashimoto, Hiromi Kondo, and Yukimasa Tamatsu. 2008. Effect of emotional expression to gaze guidance using a face robot. In RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication. 95--100. https://doi.org/10.1109/ROMAN.2008.4600649
[9]
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR 2, 5 (2017), 6.
[10]
Geoffrey E. Hinton and Richard S. Zemel. 1993. Autoencoders, Minimum Description Length and Helmholtz Free Energy. In Proceedings of the 6th International Conference on Neural Information Processing Systems (NIPS'93). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 3--10. http://dl.acm.org/citation.cfm? id=2987189.2987190
[11]
Guy Hoffman and Wendy Ju. 2014. Designing Robots with Movement in Mind. J. Hum.-Robot Interact. 3, 1 (Feb. 2014), 91--122. https://doi.org/10.5898/JHRI.3.1. Hoffman
[12]
Samira Ebrahimi Kahou, Christopher Pal, Xavier Bouthillier, Pierre Froumenty, Çaglar Gülçehre, Roland Memisevic, Pascal Vincent, Aaron Courville, Yoshua Bengio, Raul Chandias Ferrari, Mehdi Mirza, Sébastien Jean, Pierre-Luc Carrier, Yann Dauphin, Nicolas Boulanger-Lewandowski, Abhishek Aggarwal, Jeremie Zumer, Pascal Lamblin, Jean-Philippe Raymond, Guillaume Desjardins, Razvan Pascanu, David Warde-Farley, Atousa Torabi, Arjun Sharma, Emmanuel Bengio, Myriam Côté, Kishore Reddy Konda, and Zhenzhou Wu. 2013. Combining Modality Specific Deep Neural Networks for Emotion Recognition in Video. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI '13). ACM, New York, NY, USA, 543--550. https://doi.org/10.1145/2522848. 2531745
[13]
Michelle Karg, Ali-Akbar Samadani, Rob Gorbet, Kolja Kühnlenz, Jesse Hoey, and Dana Kuli?. 2013. Body Movements for Affective Expression: A Survey of Automatic Recognition and Generation. IEEE Transactions on Affective Computing 4, 4 (Oct 2013), 341--359. https://doi.org/10.1109/T-AFFC.2013.29
[14]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:cs.LG/1412.6980
[15]
Andrea Kleinsmith and Nadia Bianchi-Berthouze. 2013. Affective Body Expression Perception and Recognition: A Survey. IEEE Transactions on Affective Computing 4, 1 (Jan 2013), 15--33. https://doi.org/10.1109/T-AFFC.2012.16
[16]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q.Weinberger (Eds.). Curran Associates, Inc., 1097--1105. http://papers.nips.cc/paper/ 4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
[17]
Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2015. Autoencoding beyond pixels using a learned similarity metric. arXiv:cs.LG/1512.09300
[18]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579--2605.
[19]
Tomá Mikolov, Martin Karafiát, Luká Burget, Jan ?ernock
[20]
y, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In 11th Annual Conference of the International Speech Communication Association.
[21]
Hong-Wei Ng, Viet Dung Nguyen, Vassilios Vonikakis, and Stefan Winkler. 2015. Deep Learning for Emotion Recognition on Small Datasets Using Transfer Learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 443--449. https: //doi.org/10.1145/2818346.2830593
[22]
Mannes Poel, Dirk Heylen, Anton Nijholt, M Meulemans, and A Van Breemen. 2009. Gaze behaviour, believability, likability and the iCat. AI & SOCIETY 24, 1 (01 Aug 2009), 61--73. https://doi.org/10.1007/s00146-009-0198--1
[23]
Jonathan Posner, James A. Russell, and Bradley S. Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology 17, 3 (2005), 715--734. https://doi.org/10.1017/S0954579405050340
[24]
Helge Rhodin, James Tompkin, Kwang In Kim, Kiran Varanasi, Hans-Peter Seidel, and Christian Theobalt. 2014. Interactive motion mapping for real-time character control. Computer Graphics Forum 33, 2 (2014), 273--282. https://doi.org/10.1111/ cgf.12325 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12325
[25]
Adam Roberts, Jesse H. Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck. 2018. A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music. CoRR abs/1803.05428 (2018). arXiv:1803.05428 http://arxiv.org/abs/1803. 05428
[26]
Igor Rodriguez, José María Martínez-Otzeta, Itziar Irigoien, and Elena Lazkano. 2019. Spontaneous talking gestures using Generative Adversarial Networks. Robotics and Autonomous Systems 114 (2019), 57--65. https://doi.org/10.1016/j. robot.2018.11.024
[27]
Frank Rosenblatt. 1958. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65, 6 (1958), 386--408.
[28]
Martin Saerbeck and Christoph Bartneck. 2010. Perception of Affect Elicited by Robot Motion. In Proceedings of the 5th ACM/IEEE International Conference on Human-robot Interaction (HRI '10). IEEE Press, Piscataway, NJ, USA, 53--60. http://dl.acm.org/citation.cfm?id=1734454.1734473
[29]
Tamie Salter, Kerstin Dautenhahn, and R Bockhorst. 2004. Robots moving out of the laboratory - detecting interaction levels and human contact in noisy school environments. In RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759). 563-- 568. https://doi.org/10.1109/ROMAN.2004.1374822
[30]
Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural Networks 61 (2015), 85--117. https://doi.org/10.1016/j.neunet.2014.09.003
[31]
Yeongho Seol, Carol O'Sullivan, and Jehee Lee. 2013. Creature Features: Online Motion Puppetry for Non-human Characters. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '13). ACM, New York, NY, USA, 213--221. https://doi.org/10.1145/2485895.2485903
[32]
Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B. ViÃgas, and MartinWattenberg. 2016. Embedding Projector: Interactive Visualization and Interpretation of Embeddings. arXiv:stat.ML/1611.05469
[33]
Michael Suguitan and Guy Hoffman. 2019. Blossom: A Handcrafted Open-Source Robot. ACM Trans. Hum.-Robot Interact. 8, 1, Article 2 (March 2019), 27 pages. https://doi.org/10.1145/3310356
[34]
Joshua M Susskind, Geoffrey E Hinton, Javier R Movellan, and Adam K Anderson. 2008. Generating facial expressions with deep belief nets. In Affective Computing. InTech.
[35]
Aäron Van Den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. CoRR abs/1609.03499 (2016).
[36]
Katsu Yamane, Yuka Ariki, and Jessica Hodgins. 2010. Animating Non-humanoid Characters with Human Motion Data. In Proceedings of the 2010 ACM SIGGRAPH/ Eurographics Symposium on Computer Animation (SCA '10). Eurographics Association, Goslar Germany, Germany, 169--178. http://dl.acm.org/citation.cfm? id=1921427.1921453
[37]
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2017. mixup: Beyond Empirical Risk Minimization. arXiv:cs.LG/1710.09412
[38]
Shengjia Zhao, Jiaming Song, and Stefano Ermon. 2017. Towards deeper understanding of variational autoencoding models. arXiv preprint arXiv:1702.08658 (2017).
[39]
Allan Zhou and Anca D Dragan. 2018. Cost Functions for Robot Motion Style. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 3632--3639. https://doi.org/10.1109/IROS.2018.8594433

Cited By

View all
  • (2024)Creating Expressive Social Robots That Convey Symbolic and Spontaneous CommunicationSensors10.3390/s2411367124:11(3671)Online publication date: 5-Jun-2024
  • (2024)Generative Expressive Robot Behaviors using Large Language ModelsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634999(482-491)Online publication date: 11-Mar-2024
  • (2024)Propensity to trust shapes perceptions of comforting touch between trustworthy human and robot partnersScientific Reports10.1038/s41598-024-57582-114:1Online publication date: 21-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HRI '20: Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
March 2020
690 pages
ISBN:9781450367462
DOI:10.1145/3319502
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. affective generation
  2. deep learning
  3. neural networks
  4. social robots

Qualifiers

  • Research-article

Conference

HRI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)48
  • Downloads (Last 6 weeks)7
Reflects downloads up to 30 Aug 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Creating Expressive Social Robots That Convey Symbolic and Spontaneous CommunicationSensors10.3390/s2411367124:11(3671)Online publication date: 5-Jun-2024
  • (2024)Generative Expressive Robot Behaviors using Large Language ModelsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634999(482-491)Online publication date: 11-Mar-2024
  • (2024)Propensity to trust shapes perceptions of comforting touch between trustworthy human and robot partnersScientific Reports10.1038/s41598-024-57582-114:1Online publication date: 21-Mar-2024
  • (2023)Face2Gesture: Translating Facial Expressions into Robot Movements through Shared Latent Space Neural NetworksACM Transactions on Human-Robot Interaction10.1145/362338613:3(1-18)Online publication date: 4-Oct-2023
  • (2023)Illustrating Robot MovementsProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3576956(231-242)Online publication date: 13-Mar-2023
  • (2022)Robotic Arm Trajectory Generation Based on Emotion and Kinematic Feature2022 International Power Electronics Conference (IPEC-Himeji 2022- ECCE Asia)10.23919/IPEC-Himeji2022-ECCE53331.2022.9807205(1332-1336)Online publication date: 15-May-2022
  • (2022)An Overview of Emotion in Artificial IntelligenceIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.31596143:6(867-886)Online publication date: Dec-2022
  • (2022)Emotion and Mood Blending in Embodied Artificial Agents: Expressing Affective States in the Mini Social RobotInternational Journal of Social Robotics10.1007/s12369-022-00915-914:8(1841-1864)Online publication date: 2-Sep-2022
  • (2022)What is it like to be a bot? Variable perspective embodied telepresence for crowdsourcing robot movementsPersonal and Ubiquitous Computing10.1007/s00779-022-01684-y27:2(299-315)Online publication date: 4-May-2022
  • (2021)You Are (Not) The Robot: Variable Perspective Motion Control of a Social Telepresence RobotExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3451571(1-4)Online publication date: 8-May-2021
  • Show More Cited By

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media