Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Psychophysiological Feedback for Adaptive Human–Robot Interaction (HRI)

  • Chapter
  • First Online:
Advances in Physiological Computing

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Recent advances in robotics and sensing have given rise to a diverse set of robots and their applications. In recent years robots have increasingly applied in the service industry, search and rescue operations and therapeutic applications. The introduction of robots to interact with humans resulted in a dedicated field called human–robot interaction (HRI). Social HRI is of particular importance as it is the main focus of this chapter. This chapter presents an affect-inspired approach for social HRI. Physiological processing together with machine learning was employed to model affective states for an adaptive social HRI and its application in social interaction in the context of autism therapy was investigated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Agrawal P, Liu C, Sarkar N (2008) Interaction between human and robot an affect-inspired approach. Interact Stud 9(2):230–257

    Article  Google Scholar 

  • Bethel CL, Burke JL, Murphy RR, Salomon K (2007) Psychophysiological experimental design for use in human-robot interaction studies. In: International Symposium on Collaborative Technologies and Systems, 2007. CTS 2007. IEEE, pp 99–105

    Google Scholar 

  • Bethel CL, Salomon K, Murphy RR (2009) Preliminary results: humans find emotive non-anthropomorphic robots more calming. In: 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2009. IEEE, pp 291–292

    Google Scholar 

  • Bien ZZ, Lee HE (2007) Effective learning system techniques for human–robot interaction in service environment. Knowl-Based Syst 20(5):439–456

    Article  Google Scholar 

  • Cacioppo JT, Tassinary LG, Berntson G (2007) Handbook of psychophysiology. Cambridge University Press, Cambridge

    Google Scholar 

  • Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput 1(1):18–37

    Article  Google Scholar 

  • Conn K, Liu C, Sarkar N, Stone W, Warren Z (2010) Towards affect-sensitive assistive intervention technologies for children with autism. In Jimmy OR (ed) Affective computing: focus on emotion expression, synthesis and recognition. ARS/I-Tech Education and Publishing, Austria, pp 365–390

    Google Scholar 

  • Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor JG (2001) Emotion recognition in human-computer interaction. Sig Process Mag IEEE 18(1):32–80

    Article  Google Scholar 

  • Croft DKEA (2003) Estimating intent for human-robot interaction. In: IEEE International Conference on Advanced Robotics, 2003. pp 810–815

    Google Scholar 

  • D’Mello SK, Graesser A (2010) Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model User-Adap Inter 20(2):147–187

    Article  Google Scholar 

  • Dawson ME, Schell AM, Filion DL (2007) The Electrodermal System. In: Cacioppo JT, Tassinary LG, Berntson GG (eds) Handbook of psychophysiology. Cambridge University Press, New York, p 159

    Google Scholar 

  • Diehl JJ, Schmitt LM, Villano M, Crowell CR (2012) The clinical use of robots for individuals with autism spectrum disorders: A critical review. Res Autism Spectrum Disord 6(1):249–262

    Article  Google Scholar 

  • Dunn L, Williams KT, Wang JJ, Booklets N (1997) Peabody picture vocabulary test, (PPVT-III): Form IIA. American Guidance Service Inc, Circle Pines

    Google Scholar 

  • Fairclough SH (2009) Fundamentals of physiological computing. Interact Comput 21(1):133–145

    Article  Google Scholar 

  • Feil-Seifer D, Mataric M (2011) Automated detection and classification of positive vs. negative robot interactions with children with autism using distance-based features. In: Proceedings of the 6th international conference on Human-robot interaction, 2011. ACM, pp 323–330

    Google Scholar 

  • Feil-Seifer D, Mataric MJ (2005) A multi-modal approach to selective interaction in assistive domains. In: IEEE International Workshop on Robot and Human Interactive Communication, 2005. ROMAN 2005. IEEE, pp 416–421

    Google Scholar 

  • Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robot Auton Syst 42(3):143–166

    Article  MATH  Google Scholar 

  • Goodrich MA, Schultz AC (2007) Human-robot interaction: a survey. Found Trends Hum Comput Interact 1(3):203–275

    Article  MATH  Google Scholar 

  • Hussain M, AlZoubi O, Calvo R, D’Mello S (2011) Affect detection from multichannel physiology during learning sessions with AutoTutor. In: Artificial intelligence in education. Springer, Berlin, pp 131–138

    Google Scholar 

  • Hussain M, Monkaresi H, Calvo R (2012) Categorical vs. dimensional representations in multimodal affect detection during learning. In: Intelligent tutoring systems. Springer, Berlin, pp 78–83

    Google Scholar 

  • Iani C, Gopher D, Lavie P (2004) Effects of task difficulty and invested mental effort on peripheral vasoconstriction. Psychophysiology 41(5):789–798

    Article  Google Scholar 

  • Jerritta S, Murugappan M, Nagarajan R, Wan K (2011) Physiological signals based human emotion recognition: a review. In: IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA), 2011. IEEE, pp 410–415

    Google Scholar 

  • Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30(12):2067–2083

    Article  Google Scholar 

  • Koenig A, Novak D, Omlin X, Pulfer M, Perreault E, Zimmerli L, Mihelj M, Riener R (2011) Real-time closed-loop control of cognitive load in neurological patients during robot-assisted gait training. IEEE Trans Neural Syst Rehabil Eng 19(4):453–464

    Article  Google Scholar 

  • Kubicek W, Karnegis J, Patterson R, Witsoe D, Mattson R (1966) Development and evaluation of an impedance cardiac output system. Aerosp Med 37(12):1208

    Google Scholar 

  • Kulic D, Croft E (2005) Anxiety detection during human-robot interaction. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005.(IROS 2005). IEEE, pp 616–621

    Google Scholar 

  • Kulić D, Croft E (2007a) Pre-collision safety strategies for human-robot interaction. Auton Robots 22(2):149–164

    Google Scholar 

  • Kulic D, Croft EA (2007b) Affective state estimation for human–robot interaction. IEEE Trans Robot 23(5):991–1000

    Google Scholar 

  • Kwakkel G, Kollen BJ, Krebs HI (2008) Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair 22(2):111–121

    Article  Google Scholar 

  • Lacey JI, Lacey BC (1958) Verification and extension of the principle of autonomic response-stereotypy. The Am J Psychol 71(1):50–73

    Article  MathSciNet  Google Scholar 

  • Lahiri U, Welch KC, Sarkar M (2012) Psychophysiological response in virtual reality based human-computer interaction in adolescents with ASD. In: Imaging and signal processing in health care and technology/772: human-computer interaction/773: communication, internet and information technology. ACTA Press, USA

    Google Scholar 

  • Leon E, Clarke G, Callaghan V, Sepulveda F (2007) A user-independent real-time emotion recognition system for software agents in domestic environments. Eng Appl Artif Intell 20(3):337–345

    Article  Google Scholar 

  • Liu C, Conn K, Sarkar N, Stone W (2007) Affect recognition in robot assisted rehabilitation of children with autism spectrum disorder. In: IEEE International Conference on Robotics and Automation, 2007. IEEE, pp 1755–1760

    Google Scholar 

  • Liu C, Conn K, Sarkar N, Stone W (2008a) Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Trans Robot 24(4):883–896

    Article  Google Scholar 

  • Liu C, Conn K, Sarkar N, Stone W (2008b) Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder. Int J Hum Comput Stud 66(9):662–677

    Article  Google Scholar 

  • Liu C, Rani P, Sarkar N (2005) An empirical study of machine learning techniques for affect recognition in human-robot interaction. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS 2005). IEEE, pp 2662–2667

    Google Scholar 

  • Liu C, Rani P, Sarkar N (2006) Affective state recognition and adaptation in human-robot interaction: A design approach. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems 2006. IEEE, pp 3099–3106

    Google Scholar 

  • Murphy RR (2004) Human-robot interaction in rescue robotics. IEEE Trans Syst Man Cybern Part C Appl Rev 34(2):138–153

    Article  Google Scholar 

  • Nourbakhsh IR, Sycara K, Koes M, Yong M, Lewis M, Burion S (2005) Human-robot teaming for search and rescue. IEEE Pervasive Comput 4(1):72–79

    Article  Google Scholar 

  • Pantic M, Pentland A, Nijholt A, Huang T (2007) Human computing and machine understanding of human behavior: A survey. Artifical intelligence for human computing. Springer, Berlin, pp 47–71

    Google Scholar 

  • Papillo J, Shapiro D (1990) The cardiovascular system. In: Principles of psychophysiology: physical, social and inferential elements. Cambridge University Press, Cambridge

    Google Scholar 

  • Pecchinenda A (1996) The affective significance of skin conductance activity during a difficult problem-solving task. Cogn Emot 10(5):481–504

    Article  Google Scholar 

  • Picard R (1997) Affective computing. MIT Press, Cambridge

    Google Scholar 

  • Picard RW (1999) Affective computing for HCI. In: Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces, 1999. pp 829–833

    Google Scholar 

  • Rani P, Liu C, Sarkar N, Vanman E (2006) An empirical study of machine learning techniques for affect recognition in human–robot interaction. Pattern Anal Appl 9(1):58–69

    Article  Google Scholar 

  • Rani P, Sarkar N (2005) Making robots emotion-sensitive-preliminary experiments and results. In: IEEE International Workshop on Robot and Human Interactive Communication, 2005. ROMAN 2005. IEEE, pp 1–6

    Google Scholar 

  • Rani P, Sarkar N, Smith CA, Kirby LD (2004) Anxiety detecting robotic system-towards implicit human-robot collaboration. Robotica 22(1):85–95

    Article  Google Scholar 

  • Rani P, Sims J, Brackin R, Sarkar N (2002) Online stress detection using psychophysiological signals for implicit human-robot cooperation. Robotica 20(06):673–685

    Article  Google Scholar 

  • Sarkar N (2002) Psychophysiological control architecture for human-robot coordination-concepts and initial experiments. In: Proceedings of the ICRA’02 IEEE International Conference on Robotics and Automation, 2002. IEEE, pp 3719–3724

    Google Scholar 

  • Scassellati B, Admoni H, Mataric M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294

    Article  Google Scholar 

  • Severinson-Eklundh K, Green A, Hüttenrauch H (2003) Social and collaborative aspects of interaction with a service robot. Robot Auton Syst 42(3):223–234

    Article  MATH  Google Scholar 

  • Tao J, Tan T (2005) Affective computing: A review. Affective computing and intelligent interaction. Springer, Heidelberg, pp 981–995

    Google Scholar 

  • Tapus A, Mataric M (2008) Socially assistive robots: The link between personality, empathy, physiological signals, and task performance. In: AAAI Spring, 2008. pp 3–4

    Google Scholar 

  • Thrun S (2004) Toward a framework for human-robot interaction. Hum Comput Interact 19(1–2):9–24

    Article  Google Scholar 

  • Vansteelandt K, Van Mechelen I, Nezlek JB (2005) The co-occurrence of emotions in daily life: A multilevel approach. J Res Pers 39(3):325–335

    Article  Google Scholar 

  • Viventi J, Kim D-H, Moss JD, Kim Y-S, Blanco JA, Annetta N, Hicks A, Xiao J, Huang Y, Callans DJ (2010) A conformal, bio-interfaced class of silicon electronics for mapping cardiac electrophysiology. Sci Transl Med 2(24):24ra22

    Google Scholar 

  • Wagner J, Kim J, André E (2005) From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In: IEEE International Conference on Multimedia and Expo, 2005 ICME 2005. IEEE, pp 940–943

    Google Scholar 

  • Welch K, Lahiri U, Liu C, Weller R, Sarkar N, Warren Z (2009) An affect-sensitive social interaction paradigm utilizing virtual reality environments for autism intervention. Human-Computer Interaction Ambient, Ubiquitous and Intelligent Interaction, pp 703–712

    Google Scholar 

  • Wiering MA (2005) QV (lambda)-learning: A new on-policy reinforcement learning algorithm. Proceedings of the 7th European Workshop on Reinforcement Learning

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by a Marino Autism Research Institute (MARI) grant, an Autism Speaks Foundation Pilot grant, the National Science Foundation Grant [award number 0967170], and the National Institute of Health Grant [award number 1R01MH091102-01A1]. We would like to thank all colleagues that helped in this research and give special thanks to all subjects and their families.

Author information

Authors and Affiliations

  1. Vanderbilt University, 518 Olin hall, 2400 Highland Ave, Nashville, TN, 37212, USA

    Esubalew Bekele & Nilanjan Sarkar

Authors
  1. Esubalew Bekele
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nilanjan Sarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esubalew Bekele .

Editor information

Editors and Affiliations

  1. Liverpool John Moores University School of Natural Sciences & Psychology, Liverpool, Merseyside, United Kingdom

    Stephen H. Fairclough

  2. Liverpool John Moores University School of Natural Sciences & Psychology, Liverpool, Merseyside, United Kingdom

    Kiel Gilleade

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Bekele, E., Sarkar, N. (2014). Psychophysiological Feedback for Adaptive Human–Robot Interaction (HRI). In: Fairclough, S., Gilleade, K. (eds) Advances in Physiological Computing. Human–Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-4471-6392-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-6392-3_7

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-6391-6

  • Online ISBN: 978-1-4471-6392-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation