Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Conversational Agents and Robot Interaction

  • Conference paper
  • First Online:
HCI International 2022 - Late Breaking Papers. Multimodality in Advanced Interaction Environments (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13519))

Included in the following conference series:

  • 1015 Accesses

Abstract

The paper concerns conversational interaction for social robots and presents work on combining conversational AI technology and humanoid robot agents. It contributes to designing human-robot interaction with the help of state-of-the-art conversational modelling techniques and discusses issues related to spoken dialogues, social interaction, and context-awareness. The paper explores the use of knowledge graphs in dialogue modelling with the goal of developing interactive robot applications with natural dialogue capabilities. The work thus supports the design of social robot applications where robot agents have capabilities for a more symbiotic relationship with humans.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.gartner.com/smarterwithgartner/chatbots-will-appeal-to-modern-workers/.

  2. 2.

    https://www.ibm.com/blogs/internet-of-things/doing-the-robot-watson-really-is-all-singing-all-dancing/.

  3. 3.

    https://www8.cao.go.jp/cstp/english/society5_0/.

  4. 4.

    RASA Homepage, https://rasa.com/.

  5. 5.

    https://www.dagstuhl.de/en/program/calendar/semhp/?semnr=20021.

  6. 6.

    http://sap.ist.i.kyoto-u.ac.jp/ijcai2020/robotdial/.

  7. 7.

    https://www.softbankrobotics.com/emea/en/nao.

  8. 8.

    https://developer.softbankrobotics.com/pepper-qisdk.

  9. 9.

    https://furhatrobotics.com/.

  10. 10.

    Object Management Group https://omg.org.

  11. 11.

    https://www.iso.org/standard/80886.html.

  12. 12.

    https://www.iso.org/standard/53820.html.

  13. 13.

    https://www.iso.org/news/2016/03/Ref2057.html.

  14. 14.

    https://www.iso.org/news/Ref2169.htm.

  15. 15.

    https://www.iso.org/standard/80773.html.

  16. 16.

    https://www.iso.org/sites/hitechstandards/.

  17. 17.

    https://www.iso.org/standard/76443.html.

  18. 18.

    https://mpai.community/.

  19. 19.

    https://www.ros.org/.

  20. 20.

    https://opencv.org/.

  21. 21.

    https://kaldi-asr.org/doc/.

  22. 22.

    https://github.com/espnet/espnet.

  23. 23.

    https://rasa.com/.

  24. 24.

    Arrows app whiteboard Homepage, https://neo4j.com/labs/arrows/.

References

  1. Baltrusaitis, T., Ahuja, C., Morency, L.-P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 4(2) (2019)

    Google Scholar 

  2. Bocklisch, T., Faulkner, J., Pawlowski, N., Nichol, A.: Rasa: open source language understanding and dialogue management. arXiv preprint arXiv:1712.05181 (2017)

  3. Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)

    Google Scholar 

  4. Broz, F., Lehmann, H., Mutlu, B., Nakano, Y. (eds.): Gaze in Human-Robot Communication. John Benjamins Publishing Company (2015)

    Google Scholar 

  5. Bunt, H., et al.: The ISO standard for dialogue act annotation, second edition. In: Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pp. 549–558. European Language Resources Association (ELRA), licensed under CC-BY-NC 549 (2020)

    Google Scholar 

  6. Chung, Y-A., Glass, J.: Speech2vec: a sequence-to-sequence framework for learning word embeddings from speech. In: Proceedisngs of Interspeech, pp. 811–815 (2018)

    Google Scholar 

  7. Clark, H., Wilkes-Gibbs, D.: Referring as a collaborative process. Cognition 22, 1–39 (1986)

    Article  Google Scholar 

  8. Csapo, A., et al.: Multimodal conversational interaction with a humanoid robot. In: Proceedings of the 3rd IEEE Conference on Cognitive Infocommunications, Kosice (2012)

    Google Scholar 

  9. Dohr, A., Modre-Opsrian, R., Drobics, M., Hayn, Schreier, G.D.: The Internet of Things for ambient assisted living. In: The 7th International Conference on Information Technology, pp. 804–809. IEEE (2010)

    Google Scholar 

  10. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: part I. IEEE Robot. Autom. Mag. 13(2), 99–110 (2006)

    Article  Google Scholar 

  11. Feldman, R.S., Rim, B.: Fundamentals of Nonverbal Behavior. Cambridge University Press (1991)

    Google Scholar 

  12. Fujii, A., Jokinen, K.: Open source system integration towards natural interaction with robots. In: HRI 2022: ACM/IEEE International Conference on Human-Robot Interaction Late-Breaking Reports (2022)

    Google Scholar 

  13. Harnad, S.: The symbol grounding problem. Physica D 42, 335–346 (1990)

    Article  Google Scholar 

  14. Heinrich, S., et al.: Crossmodal language grounding in an embodied neurocognitive model. Front. Neurorobot. 14, 1662–5218 (2020)

    Article  Google Scholar 

  15. Heylen, D., Krenn, B., Payr, S.: Companions, virtual butlers, assistive robots: empirical and theoretical insights for building long-term social relationships. In: Trappl, R. (ed.): Cybernetics and Systems 2010, pp. 539–570. Austrian Society for Cybernetic Studies, Vienna, Austria (2010)

    Google Scholar 

  16. Ide, N., Bunt, H.: Anatomy of annotation schemes: mapping to GrAF. In: Proceedings 4th Linguistic Annotation Workshop (LAW IV), Uppsala, pp. 247–255 (2010)

    Google Scholar 

  17. Jara, A.J., Zamora, M.A., Skarmeta, A.F.: An internet of things-based personal device for diabetes therapy management in ambient assisted living (AAL). Pers. Ubiquit. Comput. 15(4), 431–440 (2011)

    Article  Google Scholar 

  18. Jokinen, K.: Constructive Dialogue Modelling – Speech Interaction and Rational Agents. John Wiley (2009)

    Google Scholar 

  19. Jokinen, K.: Dialogue models for socially intelligent robots. In: Ge, S.S., et al. (eds.) ICSR 2018. LNCS (LNAI), vol. 11357, pp. 127–138. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05204-1_13

    Chapter  Google Scholar 

  20. Jokinen, K., Watanabe, K.: Boundary Crossing Agents. Societal Impact of Interactions with Socially Capable Agents. ICSR, Springer (2019)

    Google Scholar 

  21. Jokinen, K., Wilcock, G.: Next steps for social robots - a robot talking fluently about thousands of topics using English and Japanese Wikipedias. In: The 9th International Conference on Social Robotics (ICSR). sBest Robot Design Award (Software Category), Special Recognition (2017)

    Google Scholar 

  22. Jokinen, K., Wilcock, G.: Multimodal open-domain conversations with robotic platforms. In: Alameda-Pineda, X., Ricci, E., Sebe, N. (eds.) Multimodal Behaviour Analysis in the Wild - Advances and Challenges, pp. 9–26. Elsevier/Academic Press (2019)

    Google Scholar 

  23. Jokinen, K., Wilcock, G.: Towards long-term relations in social robot interactions. In: HRI (2019)

    Google Scholar 

  24. Jokinen, K., Nishimura, S., Watanabe, K., Nishimura, T.: Human-robot dialogues for explaining activities. In: D’Haro, L.F., Banchs, R.E., Li, H. (eds.) The 9th International Workshop on Spoken Spoken Dialogue System Technology. Springer, Singapore https://doi.org/10.1007/978-981-13-9443-0_20

  25. Kanda, T., Ishiguro, H.: Human-Robot Interaction in Social Robotics (eBook Published 2017). CRC Press (2013). https://doi.org/10.1201/b13004

  26. Kawahara, T., Jokinen, K.: Spoken Dialogue for Social Robots. Tutorial at Interspeech 2020, Shanghai (2020). http://www.interspeech2020.org/index.php?m=content&c=index&a=show&catid=369&id=291

  27. Leite, I., Martinho, C., Paiva, A.: Social robots for long-term interaction: a survey. Int. J. Soc. Robot. 5(2), 291–308 (2013). https://doi.org/10.1007/s12369-013-0178-y

    Article  Google Scholar 

  28. Liu, B., Tur, G., Hakkani-Tur, G., Shah, P., Heck, L.: End-to-end optimization of task-oriented dialogue model with deep reinforcement learning. In: NIPS Workshop on Conversational AI (2017)

    Google Scholar 

  29. Marge, M., Espy-Wilson, C., Ward, N.: Spoken language interaction with robots: Research issues and recommendations (2020). arXiv: https://arxiv.org/pdf/2011.05533.pdf

  30. McTear, M.: Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots. Morgan & Claypool Publishers, Kentfield (2020)

    Google Scholar 

  31. Mikolov, T., Chen, K., Corrado, G., Dean J.: Efficient Estimation of Word Representations in Vector Space (2013). arXiv:1301.3781

  32. Papangelis, A., Namazifar, M., Khatri, C., Wang, Y-C., Molino, P., Tur, G.: Plato Dialogue System: A Flexible Conversational AI Research Platform (2020)

    Google Scholar 

  33. Robinson, I., Webber, J., Eifrem, E.: Graph Databases (2nd edition). O’Reilly Media (2015)

    Google Scholar 

  34. Sarikaya, R., et al.: An overview of end-to-end language understanding and dialog management for personal digital assistants. In: Workshop on Spoken Language Technology. IEEE (2016)

    Google Scholar 

  35. Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) (2016)

    Google Scholar 

  36. Sidner, C., Rich, C., Shayganfar, M., Bickmore, T., Ring, L., Zhang, Z.: A robotic companion for social support of isolated older adults. In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts (HRI 2015 Extended Abstracts), pp. 289–289. ACM, New York (2015)

    Google Scholar 

  37. Ultes, S., et al.: PyDial: a multi-domain statistical dialogue system toolkit. In: Proceedings of ACL 2017, System Demonstrations, pp. 73–78 (2017)

    Google Scholar 

  38. Vlasov, V., Mosig, J.M.E., Nichol, A.: Dialogue transformers (2019). ArXiv:1910.00486

  39. Wilcock, G.: Using a deep learning dialogue research toolkit in a multilingual multi-domain practical application. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI 2018), Stockholm, Sweden, pp. 5880–5882 (2018)

    Google Scholar 

  40. Wilcock, G., Jokinen, K.: Conversational AI and Knowledge Graphs for Social Robot Interaction. In: HRI 2022: ACM/IEEE International Conference on Human-Robot Interaction, Sapporo, Japan (2022)

    Google Scholar 

  41. Wrede, B., et al.: Research issues for designing robot companions: BIRON as a case study. In: Proceedings of the IEEE Conference of Mechatronics & Robotics (2004)

    Google Scholar 

Download references

Acknowledgment

I thank all my colleagues and especially Graham Wilcock for fruitful and inspiring conversations. The study is based on results obtained from project JPNP20006 commissioned by the New Energy and Industrial Technology Development Organization (NEDO).

Author information

Authors and Affiliations

  1. AI Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan

    Kristiina Jokinen

Authors
  1. Kristiina Jokinen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristiina Jokinen .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  2. Tokyo University of Science, Tokyo, Saitama, Japan

    Sakae Yamamoto

  3. Tokyo City University, Tokyo, Japan

    Hirohiko Mori

  4. Soar Technology Inc., Orlando, FL, USA

    Dylan D. Schmorrow

  5. Katmai Government Services, Orlando, FL, USA

    Cali M. Fidopiastis

  6. Smart Future Initiative, Frankfurt am Main, Germany

    Norbert A. Streitz

  7. Kyushu University, Fukuoka, Japan

    Shin'ichi Konomi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jokinen, K. (2022). Conversational Agents and Robot Interaction. In: Kurosu, M., et al. HCI International 2022 - Late Breaking Papers. Multimodality in Advanced Interaction Environments. HCII 2022. Lecture Notes in Computer Science, vol 13519. Springer, Cham. https://doi.org/10.1007/978-3-031-17618-0_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-17618-0_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-17617-3

  • Online ISBN: 978-3-031-17618-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation