Effects of Computerized Emotional Training on Children with High Functioning Autism
Pages 1045 - 1054
Abstract
An evaluation study of a serious game and a system for the automatic emotion recognition designed for helping autistic children to learn to recognize and express emotions by means of their full-body movement is presented. Three-dimensional motion data of full-body movements are obtained from RGB-D sensors and used to recognize emotions by means of linear SVMs. Ten children diagnosed with High Functioning Autism or Asperger Syndrome were involved in the evaluation phase, consisting of repeated sessions to play a specifically designed serious game. Results from the evaluation study show an increase of tasks accuracy from the beginning to the end of training sessions in the trained group. In particular, while the increase of recognition accuracy was concentrated in the first sessions of the game, the increase for expression accuracy is more gradual throughout all sessions. Moreover, the training seems to produce a transfer effect on facial expression recognition.
References
[1]
American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Assoc., 2013.
[2]
L. Kanner, et al., “Autistic disturbances of affective contact,” Nervous Child, vol. 2, no. 3, pp. 217–250, 1943.
[3]
P. Hobson, “Understanding persons: The role of affect,” Understanding other minds, 1993, pp. 204–227.
[4]
C. Ashwin, E. Chapman, L. Colle, and S. Baron-Cohen, “Impaired recognition of negative basic emotions in autism: A test of the amygdala theory,” Soc. Neurosci., vol. 1, no. 3/4, pp. 349–363, 2006.
[5]
A. Klin, S. S. Sparrow, A. De Bildt, D. V. Cicchetti, D. J. Cohen, and F. R. Volkmar, “A normed study of face recognition in autism and related disorders,” J. Autism Develop. Disorders, vol. 29, no. 6, pp. 499–508, 1999.
[6]
K. M. Rump, J. L. Giovannelli, N. J. Minshew, and M. S. Strauss, “The development of emotion recognition in individuals with autism,” Child Develop., vol. 80, no. 5, pp. 1434–1447, 2009.
[7]
J. L. Tracy, R. W. Robins, R. A. Schriber, and M. Solomon, “Is emotion recognition impaired in individuals with autism spectrum disorders?” J. Autism Develop. Disorders, vol. 41, no. 1, pp. 102–109, 2011.
[8]
S. J. Weeks and R. P. Hobson, “The salience of facial expression for autistic children,” J. Child Psychology Psychiatry, vol. 28, no. 1, pp. 137–152, 1987.
[9]
G. Young-Browne, H. M. Rosenfeld, and F. D. Horowitz, “Infant discrimination of facial expressions,” Child Develop., vol. 48, pp. 555–562, 1977.
[10]
N. H. Soken and A. D. Pick, “Intermodal perception of happy and angry expressive behaviors by seven-month-old infants,” Child Develop., vol. 63, no. 4, pp. 787–795, 1992.
[11]
K. Durand, M. Gallay, A. Seigneuric, F. Robichon, and J.-Y. Baudouin, “The development of facial emotion recognition: The role of configural information,” J. Exp. Child Psychology, vol. 97, no. 1, pp. 14–27, 2007.
[12]
C. J. Mondloch, S. Geldart, D. Maurer, and R. Le Grand, “Developmental changes in face processing skills,” J. Experimental Child Psychology, vol. 86, no. 1, pp. 67–84, 2003.
[13]
C. M. Herba, S. Landau, T. Russell, C. Ecker, and M. L. Phillips, “The development of emotion-processing in children: Effects of age, emotion, and intensity,” J. Child Psychology Psychiatry, vol. 47, no. 11, pp. 1098–1106, 2006.
[14]
L. A. Thomas, M. D. De Bellis, R. Graham, and K. S. LaBar, “Development of emotional facial recognition in late childhood and adolescence,” Develop. Sci., vol. 10, no. 5, pp. 547–558, 2007.
[15]
S. Begeer, H. M. Koot, C. Rieffe, M. M. Terwogt, and H. Stegge, “Emotional competence in children with autism: Diagnostic criteria and empirical evidence,” Develop. Rev., vol. 28, no. 3, pp. 342–369, 2008.
[16]
H. J. Nuske, G. Vivanti, and C. Dissanayake, “Are emotion impairments unique to, universal, or specific in autism spectrum disorder? A comprehensive review,” Cogn. Emotion, vol. 27, no. 6, pp. 1042–1061, 2013.
[17]
M. Uljarevic and A. Hamilton, “Recognition of emotions in autism: A formal meta-analysis,” J. Autism Develop. Disorders, vol. 43, no. 7, pp. 1517–1526, 2013.
[18]
C. C. Peterson, V. Slaughter, and C. Brownell, “Children with autism spectrum disorder are skilled at reading emotion body language,” J. Experimental Child Psychology, vol. 139, pp. 35–50, 2015.
[19]
B. De Gelder, “Why bodies? Twelve reasons for including bodily expressions in affective neuroscience,” Philosophical Trans. Roy. Soc. London B: Biol. Sci., vol. 364, no. 1535, pp. 3475–3484, 2009.
[20]
B. De Gelder, R. Hortensius, and M. Tamietto, “Attention and awareness each influence amygdala activity for dynamic bodily expressions? a short review,” Frontiers Integrative Neurosci., vol. 6, 2012, Art. no.
[21]
B. de Gelder and R. Hortensius, “The many faces of the emotional body,” in New Frontiers in Social Neuroscience. Berlin, Germany: Springer, 2014, pp. 153–164.
[22]
B. App, D. N. McIntosh, C. L. Reed, and M. J. Hertenstein, “Nonverbal channel use in communication of emotion: How may depend on why,” Emotion, vol. 11, no. 3, 2011, Art. no.
[23]
A. P. Atkinson, W. H. Dittrich, A. J. Gemmell, A. W. Young, et al., “Emotion perception from dynamic and static body expressions in point-light and full-light displays,” Perception, vol. 33, pp. 717–746, 2004.
[24]
B. De Gelder, “Towards the neurobiology of emotional body language,” Nature Rev. Neurosci., vol. 7, no. 3, pp. 242–249, 2006.
[25]
B. De Gelder and J. Van den Stock, “The bodily expressive action stimulus test (BEAST). Construction and validation of a stimulus basis for measuring perception of whole body expression of emotions,” Frontiers Psychology, vol. 2, 2011, Art. no.
[26]
F. E. Pollick, V. Lestou, J. Ryu, and S.-B. Cho, “Estimating the efficiency of recognizing gender and affect from biological motion,” Vis. Res., vol. 42, no. 20, pp. 2345–2355, 2002.
[27]
T. J. Clarke, M. F. Bradshaw, D. T. Field, S. E. Hampson, and D. Rose, “The perception of emotion from body movement in point-light displays of interpersonal dialogue,” Perception, vol. 34, no. 10, pp. 1171–1180, 2005.
[28]
K. Schindler, L. Van Gool, and B. de Gelder, “Recognizing emotions expressed by body pose: A biologically inspired neural model,” Neural Netw., vol. 21, no. 9, pp. 1238–1246, 2008.
[29]
J. W. Tanaka and A. Sung, “The eye avoidance hypothesis of autism face processing,” J. Autism Develop. Disorders, vol. 46, no. 5, pp. 1538–1552, 2016.
[30]
C. L. Reed, P. M. Beall, V. E. Stone, L. Kopelioff, D. J. Pulham, and S. L. Hepburn, “Brief report: Perception of body posture? What individuals with autism spectrum disorder might be missing,” J. Autism Develop. Disorders, vol. 37, no. 8, pp. 1576–1584, 2007.
[31]
S. Piana, A. Staglian, A. Camurri, and F. Odone, “A set of full-body movement features for emotion recognition to help children affected by autism spectrum condition,” in Proc. IDGEI Int. Workshop, 2013, pp. 1–7.
[32]
S. Piana, A. Staglian, F. Odone, A. Verri, and A. Camurri, “Real-time automatic emotion recognition from body gestures,” arXiv preprint arXiv:1402.5047, 2014.
[33]
S. Piana, A. Staglianò, F. Odone, and A. Camurri, “Adaptive body gesture representation for automatic emotion recognition,” ACM Trans. Interactive Intell. Syst., vol. 6, no. 1, 2016, Art. no.
[34]
R. T. Boone and J. G. Cunningham, “Children's decoding of emotion in expressive body movement: The development of cue attunement,” Develop. Psychology, vol. 34, pp. 1007–1016, 1998.
[35]
A. Camurri, I. Lagerlöf, and G. Volpe, “Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques,” Int. J. Human-Comput. Stud., vol. 59, no. 1, pp. 213–225, 2003.
[36]
D. Glowinski, N. Dael, A. Camurri, G. Volpe, M. Mortillaro, and K. Scherer, “Toward a minimal representation of affective gestures,” IEEE Trans. Affect. Comput., vol. 2, no. 2, pp. 106–118, Apr.–Jun. 2011.
[37]
A. Kapur, A. Kapur, N. Virji-Babul, G. Tzanetakis, and P. F. Driessen, Gesture-Based Affective Computing on Motion Capture Data. Berlin, Germany: Springer, 2005, pp. 1–7.
[38]
F. Pons, P. L. Harris, and M. de Rosnay, “Emotion comprehension between 3 and 11 years: Developmental periods and hierarchical organization,” Eur. J. Develop. Psychology, vol. 1, no. 2, pp. 127–152, 2004.
[39]
R. Laban and F. C. Lawrence, Effort. U.K.: Macdonald & Evans, 1947.
[40]
R. von Laban, Modern Educational Dance. Buchanan, NY, USA: Princeton Book Co Pub, 1975.
[41]
S. Ramdoss, W. Machalicek, M. Rispoli, A. Mulloy, R. Lang, and M. O'Reilly, “Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review,” Develop. Neurorehabilitation, vol. 15, no. 2, pp. 119–135, 2012.
[42]
T. Balomenos, A. Raouzaiou, S. Ioannou, A. Drosopoulos, K. Karpouzis, and S. Kollias, Emotion Analysis in Man-Machine Interaction Systems. Berlin, Germany: Springer, 2005, pp. 318–328.
[43]
E. Barakova, G. Van Wanrooij, R. Van Limpt, and M. Menting, “Using an emergent system concept in designing interactive games for autistic children,” in Proc. 6th Int. Conf. Interaction Des. Children, 2007, pp. 73–76.
[44]
A. Battocchi, F. Pianesi, P. Venuti, A. Ben-Sasson, E. Gal, and P. L. Weiss, “Collaborative puzzle game: Fostering collaboration in children with autistic spectrum disorder (ASD) and with typical development,” in Proc. Virtual Rehabil. Int. Conf., 2009, pp. 204–204.
[45]
J. P. Hourcade, N. E. Bullock-Rest, and T. E. Hansen, “Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders,” Pers. Ubiquitous Comput., vol. 16, no. 2, pp. 157–168, 2012.
[46]
M. E. Hoque, J. K. Lane, R. El Kaliouby, M. Goodwin, and R. W. Picard, “Exploring speech therapy games with children on the autism spectrum,” in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2009, pp. 1455–1458.
[47]
M. A. Sharmin, M. M. Rahman, S. I. Ahmed, M. M. Rahman, and S. Ferdous, “Teaching intelligible speech to the autistic children by interactive computer games,” in Proc. ACM Symp. Appl. Comput., 2011, pp. 1208–1209.
[48]
C. S. Lányi and Á. Tilinger, “Multimedia and virtual reality in the rehabilitation of autistic children,” in Proc. Int. Conf. Comput. Handicapped Persons, 2004, pp. 22–28.
[49]
N. Parés, A. Carreras, J. Durany, J. Ferrer, P. Freixa, D. Gómez, O. Kruglanski, R. Parés, J. I. Ribas, M. Soler, et al., “Promotion of creative activity in children with severe autism through visuals in an interactive multisensory environment,” in Proc. Conf. Interaction Des. Children, 2005, pp. 110–116.
[50]
S. Bereznak, K. M. Ayres, L. C. Mechling, and J. L. Alexander, “Video self-prompting and mobile technology to increase daily living and vocational independence for students with autism spectrum disorders,” J. Develop. Phys. Disabilities, vol. 24, no. 3, pp. 269–285, 2012.
[51]
T.-W. Tsai and M.-Y. Lin, “An application of interactive game for facial expression of the autisms,” in Proc. Int. Conf. Technol. E-Learning Digital Entertainment, 2011, pp. 204–211.
[52]
S. Jain, B. Tamersoy, Y. Zhang, J. K. Aggarwal, and V. Orvalho, “An interactive game for teaching facial expressions to children with autism spectrum disorders,” in Proc. 5th Int. Symp. Commun. Control Signal Process., 2012, pp. 1–4.
[53]
B. Robins, K. Dautenhahn, R. Te Boekhorst, and A. Billard, “Robotic assistants in therapy and education of children with autism: Can a small humanoid robot help encourage social interaction skills?” Universal Access Inf. Soc., vol. 4, no. 2, pp. 105–120, 2005.
[54]
J. Wainer, B. Robins, F. Amirabdollahian, and K. Dautenhahn, “Using the humanoid robot KASPAR to autonomously play triadic games and facilitate collaborative play among children with autism,” IEEE Trans. Auton. Mental Develop., vol. 6, no. 3, pp. 183–199, Sep. 2014.
[55]
K. Dautenhahn and A. Billard, “Games children with autism can play with robota, a humanoid robotic doll,” in Proc. Universal Access Assistive Technol., 2002, pp. 179–190.
[56]
L. Cruz, D. Lucio, and L. Velho, “Kinect and RGBD images: Challenges and applications,” in Proc. 25th SIBGRAPI Conf. Graph. Patterns Images Tut., 2012, pp. 36–49.
[57]
B. Schuller, E. Marchi, S. Baron-Cohen, H. O'Reilly, P. Robinson, I. Davies, O. Golan, S. Friedenson, S. Tal, S. Newman, et al., “ASC-inclusion: Interactive emotion games for social inclusion of children with autism spectrum conditions,” in Proc. 1st Int. Workshop Intell. Digital Games Empowerment Inclusion Held Conjunction 8th Found. Digital Games, 2013, pp. 1–8.
[58]
E. Marchi, B. Schuller, A. Baird, S. Baron-Cohen, A. Lassalle, H. OReilly, D. Pigat, P. Robinson, I. Davies, T. Baltrušaitis, et al., “The ASC-inclusion perceptual serious gaming platform for autistic children,” IEEE Trans. Games, 2018. doi: 10.1109/TG.2018.2864640
[59]
P. Ekman, “Differential communication of affect by head and body cues,” J. Personality Social Psychology, vol. 2, no. 5, 1965, Art. no.
[60]
A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, and G. Volpe, “Multimodal analysis of expressive gesture in music and dance performances,” in Proc. Gesture-Based Commun. Human-Comput. Interaction, 2004, pp. 20–39.
[61]
A. Camurri, G. Volpe, S. Piana, M. Mancini, R. Niewiadomski, N. Ferrari, and C. Canepa, “The dancer in the eye: Towards a multi-layered computational framework of qualities in movement,” in Proc. 3rd Int. Symp. Movement Comput., 2016, Art. no.
[62]
S. Piana, P. Alborno, R. Niewiadomski, M. Mancini, G. Volpe, and A. Camurri, “Movement fluidity analysis based on performance and perception,” in Proc. CHI Conf. Extended Abstracts Human Factors Comput. Syst., 2016, pp. 1629–1636. [Online]. Available: http://doi.acm.org/10.1145/2851581.2892478
[63]
A. Camurri, S. Hashimoto, M. Ricchetti, A. Ricci, K. Suzuki, R. Trocca, and G. Volpe, “EyesWeb: Toward gesture and affect recognition in interactive dance and music systems,” Comput. Music J., vol. 24, no. 1, pp. 57–69, 2000.
[64]
A. Camurri, B. Mazzarino, and G. Volpe, “Analysis of expressive gesture: The EyesWeb expressive gesture processing library,” in Proc. Int. Gesture Workshop, 2003, pp. 460–467.
[65]
A. Camurri, P. Coletta, A. Massari, B. Mazzarino, M. Peri, M. Ricchetti, A. Ricci, and G. Volpe, “Toward real-time multimodal processing: EyesWeb 4.0,” in Proc. Artif. Intell. Simul. Behaviour, Convention: Motion. Emotion Cogn., 2004, pp. 22–26.
[66]
C. Basso, M. Santoro, A. Verri, and S. Villa, “PADDLE: Proximal algorithm for dual dictionaries learning,” in Proc. Int. Conf. Artif. Neural Netw. Mach. Learn., 2011, pp. 379–386.
[67]
Istituto Chiossone, 2017. [Online]. Available: http://www.chiossone.it/
[68]
http://centromaisoli.it/, 2017. [Online]. Available:http://centromaisoli.it/
[69]
Philos, 2017. [Online]. Available: http://www.associazionephilos.net/
[70]
G. H. Roid, L. J. Miller, and C. Koch, Leiter International Performance Scale. Wood Dale, IL, USA: Stoelting, 2013.
[71]
L. B. Jahromi, S. E. Meek, and S. Ober-Reynolds, “Emotion regulation in the context of frustration in children with high functioning autism and their typical peers,” J. Child Psychology Psychiatry, vol. 53, no. 12, pp. 1250–1258, 2012.
[72]
J. Legiša, D. S. Messinger, E. Kermol, and L. Marlier, “Emotional responses to odors in children with high-functioning autism: Autonomic arousal, facial behavior and self-report,” J. Autism Develop. Disorders, vol. 43, no. 4, pp. 869–879, 2013.
[73]
L. A. Edwards, “A meta-analysis of imitation abilities in individuals with autism spectrum disorders,” Autism Res., vol. 7, no. 3, pp. 363–380, 2014.
[74]
J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Routledge, 2013.
[75]
C. Spearman, “Correlation calculated from faulty data,” Brit. J. Psychology, vol. 3, no. 3, pp. 271–295, 1910.
[76]
J. L. Wilbarger, C. L. Reed, and D. N. McIntosh, “Implicit influence of affective postures on the perception of others: You can't show me how I feel,” Emotion, vol. 11, no. 3, 2011, Art. no.
Index Terms
- Effects of Computerized Emotional Training on Children with High Functioning Autism
Index terms have been assigned to the content through auto-classification.
Recommendations
Lexical Representation of Emotions for High Functioning Autism(HFA) via Emotional Story Intervention using Smart Media
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsChildren with High-Functioning Autism (HFA) have difficulty understanding and using emotional expressions. A variety of smart media, such as tablet PCs and robots, are being used to train children with autism spectrum disorders on emotional vocabulary ...
Video games in adolescence and emotional functioning: Emotion regulation, emotion intensity, emotion expression, and alexithymia
AbstractVideo-game users represent 40% of the French population and adolescents are the primary users. Yet excessive playing of video games has become a problem in modern society and is manifesting itself in treatment centers for adolescents. ...
Highlights- An understanding of video-game playing based on dimensions of emotional functioning.
Motor, emotional and cognitive empathic abilities in children with autism and conduct disorder
MB '10: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral ResearchThis paper gives an overview of the studies that investigated motor, emotional and cognitive empathy in juveniles with autism or conduct disorder. Studies that measured response to emotional faces with use of facial EMG, ECG, skin conductance, eye-...
Comments
Information & Contributors
Information
Published In
1949-3045 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Computer Society Press
Washington, DC, United States
Publication History
Published: 01 October 2021
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025