Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Adaptive Social Planner to Accompany People in Real-Life Dynamic Environments

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Robots must develop the ability to socially navigate in uncontrolled urban environments to be able to be included in our daily lives. This paper presents a new robot navigation framework called the adaptive social planner (ASP) and a robotic system, which includes the ASP. Our results and previous work show that the ASP can adapt to different collaborative tasks involving humans and robots, such as independent robot navigation, human-robot accompaniment, a robot approaching people, robot navigation tasks that combine learning techniques, and human-drone interactions. Our approach in this paper focuses on demonstrating how the ASP can be customized to implement two new methods for group accompaniment: the adaptive social planner using a V-formation model to accompany groups of people (ASP-VG) and the adaptive social planner using a side-by-side model to accompany groups of people (ASP-SG). These two methods result in a robot accompanying groups of people by anticipating human and uncontrolled urban environment behaviors. Also, we develop four new robot skills to deal with unexpected human behaviors, such as rearrangement of the position of the companions inside the group, unforeseen changes in the velocity of the robot companions, occlusions among group members, and changes in the direction toward destinations in the environment. Moreover, we develop different performance metrics, based on social distances, to evaluate the tasks of the robot. In addition, we present the guidelines followed in performing the real-life experiments with volunteers, including a human-robot speech interaction to help humans create a relationship with the robot to be genuinely involved in the mutual accompaniment. Finally, we include an exhaustive validation of the methods by evaluating the behavior of the robot through synthetic and real-life experiments. We incorporate five user studies to evaluate aspects related to social acceptability and preferences of people regarding both types of robot group accompaniment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

Not applicable.

Notes

  1. http://wiki.ros.org/amcl

  2. http://www.iri.upc.edu/people/erepiso/Journal_group_accompaniment_Vform_SidebySide.html

  3. In Eq. 11 we are assuming \(-\pi< \theta < \pi \), and using \(\hbox {sign}(0)=-1\) in order to have a continuous potential. Refer to the original work for details [68].

  4. http://wiki.ros.org/ps3joy

  5. Cronbach’s alpha is a measure used to determine how reliably a set of questions measures a single dimension. Values less than 0.7 imply that the scale is measuring more than one thing; higher levels indicate that the questions are essentially asking about the same thing, so the items can be combined for analysis.

References

  1. Ferrer G, Sanfeliu A (2019) Anticipative kinodynamic planning: multi-objective robot navigation in urban and dynamic environments. Auton Robot 43(6):1473–1488

    Article  Google Scholar 

  2. Repiso E, Ferrer G, Sanfeliu A (2017) On-line adaptive side-by-side human robot companion in dynamic urban environments. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 872–877

  3. Repiso E, Zanlungo F, Kanda T, Garrell A, Sanfeliu A (2019) People’s v-formation and side-by-side model adapted to accompany groups of people by social robots. In: IEEE/RSJ international conference on Intelligent robots and systems

  4. Repiso E, Garrell A, Sanfeliu A (2020) People’s adaptive side-by-side model evolved to accompany groups of people by social robots. In: IEEE robotics and automation letters and IEEE/RSJ international conference on robotics and automation (RA-L and ICRA)

  5. Galvan M, Repiso E, Sanfeliu A (2019) Robot navigation to approach people using g\(^{2}\)-spline path planning and extended social force model. In: Iberian robotics conference. Springer, pp 15–27

  6. Repiso E, Garrell A, Sanfeliu A (2019) Adaptive side-by-side social robot navigation to approach and interact with people. Int J Soc Robot, pp 1–22

  7. Costa M (2010) Interpersonal distances in group walking. J Nonverbal Behav 34(1):15–26

    Article  Google Scholar 

  8. Moussaïd M, Perozo N, Garnier S, Helbing D, Theraulaz G (2010) The walking behaviour of pedestrian social groups and its impact on crowd dynamics. PLoS ONE 5(4):e10047

    Article  Google Scholar 

  9. Schultz M, Rößger L, Fricke H, Schlag B (2014) Group dynamic behavior and psychometric profiles as substantial driver for pedestrian dynamics. Pedestr Evac Dyn 2012:1097–1111

    Google Scholar 

  10. Zanlungo F, Kanda T (2015) A mesoscopic model for the effect of density on pedestrian group dynamics. EPL (Europhys Lett) 111(3):38007

    Article  Google Scholar 

  11. Hall ET, Edward T (1969) Hall. The hidden dimension. Anchor Books, New York 20:71

  12. Stubbs K, Hinds PJ, Wettergreen D (2007) Autonomy and common ground in human-robot interaction: a field study. IEEE Intel Syst 22(2)

  13. Garrell A, Sanfeliu A (2012) Cooperative social robots to accompany groups of people. Int J Robot Res 31(13):1675–1701

    Article  Google Scholar 

  14. Garrell A, Villamizar M, Moreno-Noguer F, Sanfeliu A (2013) Proactive behavior of an autonomous mobile robot for human-assisted learning. In: 2013 IEEE RO-MAN. IEEE, pp 107–113

  15. Repiso E, Garrell A, Sanfeliu A (2018) Robot approaching and engaging people in a human-robot companion framework. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 8200–8205

  16. Gil Ó, Garrell A, Sanfeliu A (2021) Social robot navigation tasks: combining machine learning techniques and social force model. Sensors 21(21):7087

    Article  Google Scholar 

  17. Garrell A, Garza-Elizondo L, Villamizar M, Herrero F, Sanfeliu A (2017) Aerial social force model: A new framework to accompany people using autonomous flying robots. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 7011–7017

  18. Garrell A, Coll C, Alquézar R, Sanfeliu A (2019) Teaching a drone to accompany a person from demonstrations using non-linear asfm. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 1985–1991

  19. Ferrer G, Sanfeliu A (2014) Bayesian human motion intentionality prediction in urban environments. Pattern Recogn Lett 44:134–140

    Article  Google Scholar 

  20. Helbing D, Molnar P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282

    Article  Google Scholar 

  21. Charalampous K, Kostavelis I, Gasteratos A (2017) Recent trends in social aware robot navigation: A survey. Robot Auton Syst 93:85–104

    Article  Google Scholar 

  22. Khambhaita H, Alami R (2017) Viewing robot navigation in human environment as a cooperative activity. arXiv preprint arXiv:1708.01267

  23. Sisbot EA, Marin-Urias LF, Alami R, Simeon T (2007) A human aware mobile robot motion planner. IEEE Trans Rob 23(5):874–883

    Article  Google Scholar 

  24. Che Y, Okamura AM, Sadigh D (2020) Efficient and trustworthy social navigation via explicit and implicit robot–human communication. IEEE Trans Robot

  25. Fleishman V, Honig S, Oron-Gilad T, Edan Y (2018) Proxemic preferences when being followed by a robot. Israeli ministry of science and technology, report, pp 3–12 060

  26. Wolf K, Lischke L (2014) Urban proxemics for public guidance. In: Proceedings of the nordichi workshop on ubicomp beyond devices: people, objects, space and meaning, ACM, vol 10, pp 2 639 189–2 654 842

  27. Rios-Martinez J, Spalanzani A, Laugier C (2015) From proxemics theory to socially-aware navigation: A survey. Int J Soc Robot 7(2):137–153

    Article  Google Scholar 

  28. MacArthur KR, Stowers K, Hancock P (2017) Human-robot interaction: Proximity and speed-slowly back away from the robot! In: Advances in human factors in robots and unmanned systems. Springer, pp 365–374

  29. Pacchierotti E, Christensen HI, Jensfelt P (2006) Evaluation of passing distance for social robots. In: Roman 2006-the 15th IEEE international symposium on robot and human interactive communication. IEEE, pp 315–320

  30. Lasota PA, Fong T, Shah JA et al (2017) A survey of methods for safe human-robot interaction. Found Trends Robot 5(4):261–349

    Article  Google Scholar 

  31. Šabanović S, Reeder SM, Kechavarzi B (2014) Designing robots in the wild: in situ prototype evaluation for a break management robot. J Hum-Robot Interact 3(1):70–88

    Article  Google Scholar 

  32. Clodic A, Fleury S, Alami R, Chatila R, Bailly G, Brethes L, Cottret M, Danes P, Dollat X, Elisei F, et al. (2006) Rackham: An interactive robot-guide. In: ROMAN 2006-The 15th IEEE international symposium on robot and human interactive communication. IEEE, pp 502–509

  33. Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2009) An affective guide robot in a shopping mall. In: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. ACM, pp 173–180

  34. Yousuf MA, Kobayashi Y, Kuno Y, Yamazaki A, Yamazaki K (2012) Development of a mobile museum guide robot that can configure spatial formation with visitors. In: International conference on intelligent computing. Springer, pp 423–432

  35. Merino L, Ballesteros J, Pérez-Higueras N, Vigo RR, Pérez-Lara J, Caballero F (2014) Robust person guidance by using online pomdps. In: ROBOT2013: First Iberian robotics conference. Springer, pp 289–303

  36. Triebel R, Arras K, Alami R, Beyer L, Breuers S, Chatila R, Chetouani M, Cremers D, Evers V, Fiore M, et al. (2016) Spencer: a socially aware service robot for passenger guidance and help in busy airports. In: Field and service robotics. Springer, pp 607–622

  37. Takemura H, Nemoto Z, Ito K, Mizoguchi H (2010) Development of vision based person following module for mobile robots in RT-middleware. In: Climbing and Walking Robots, p 409

  38. Katz D (2016) Development of algorithms for a human-following robot equipped with kinect vision and laser sensors in an unknown indoor environment with obstacles and corners. In: Ben- Gurion University of the Negev. Citeseer, pp 463–466

  39. Kobayashi T, Chugo D, Yokota S, Muramatsu S, Hashimoto H (2015) Design of personal mobility motion based on cooperative movement with a companion. In: 2015 6th IEEE international conference on cognitive infocommunications (CogInfoCom). IEEE, pp 165–170

  40. Honig SS, Oron-Gilad T, Zaichyk H, Sarne-Fleischmann V, Olatunji S, Edan Y (2018) Toward socially aware person-following robots. IEEE Trans Cognit Dev Syst 10(4):936–954

    Article  Google Scholar 

  41. Prassler E, Bank D, Kluge B, Hagele M (2002) Key technologies in robot assistants: motion coordination between a human and a mobile robot. Trans Control, Autom Syst Eng 4(1):56–61

    Google Scholar 

  42. Murakami R, Morales Saiki LY, Satake S, Kanda T, Ishiguro H (2014) Destination unknown: walking side-by-side without knowing the goal. In: Proceedings of the ACM/IEEE international conference on Human-robot interaction. ACM, pp 471–478

  43. Morales Y, Kanda T, Hagita N (2014) Walking together: side by side walking model for an interacting robot. J Hum-Robot Interact 3(2):51–73

    Article  Google Scholar 

  44. Kuderer M, Kretzschmar H, Burgard W (2013) Teaching mobile robots to cooperatively navigate in populated environments. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3138–3143

  45. Rodić A, Vujović M, Stevanović I, Jovanović M (2016) Development of human-centered social robot with embedded personality for elderly care. In: New trends in medical and service robots. Springer, pp 233–247

  46. Scheggi S, Morbidi F, Prattichizzo D (2014) Human-robot formation control via visual and vibrotactile haptic feedback. IEEE Trans Haptics 7(4):499–511

    Article  Google Scholar 

  47. Du Y, Hetherington NJ, Oon CL, Chan WP, Quintero CP, Croft E, Van der Loos HM (2019) Group surfing: A pedestrian-based approach to sidewalk robot navigation. In: 2019 international conference on robotics and automation (ICRA). IEEE, pp 6518–6524

  48. Karunarathne D, Morales Y, Kanda T, Ishiguro H (2018) Model of side-by-side walking without the robot knowing the goal. Int J Soc Robot 10(4):401–420

    Article  Google Scholar 

  49. Ferrer G, Garrell A, Herrero F, Sanfeliu A (2016) Robot social-aware navigation framework to accompany people walking side-by-side. Auton Robots, pp 1–19

  50. Kuderer M, Burgard W (2014) An approach to socially compliant leader following for mobile robots. In: International conference on social robotics. Springer, pp 239–248

  51. Scheggi S, Aggravi M, Morbidi F, Prattichizzo D (2014) Cooperative human-robot haptic navigation. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, 2014, pp 2693–2698

  52. Saez-Pons J, Alboul L, Penders J (2011) Experiments in cooperative human multi-robot navigation. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1–4

  53. Urcola P, Montano L (2011) Adapting robot team behavior from interaction with a group of people. In: 2011 IEEE/RSJ international conference on Intelligent robots and systems (IROS). IEEE, p 2887

  54. Iqbal T, Rack S, Riek LD (2016) Movement coordination in human-robot teams: a dynamical systems approach. IEEE Trans Rob 32(4):909–919

    Article  Google Scholar 

  55. Díaz-Boladeras M, Paillacho D, Angulo C, Torres O, González-Diéguez J, Albo-Canals J (2015) Evaluating group-robot interaction in crowded public spaces: a week-long exploratory study in the wild with a humanoid robot guiding visitors through a science museum. Int J Hum Robot 12(04)

  56. Leaman J, La HM (2017) A comprehensive review of smart wheelchairs: past, present and future. arXiv preprint arXiv:1704.04697

  57. Prassler E, Bank D, Kluge B (2001) Motion coordination between a human and a robotic wheelchair. In: Proceedings. 10th IEEE international workshop on robot and human interactive communication, pp 412–417

  58. Kobayashi Y, Kinpara Y, Shibusawa T, Kuno Y (2009) Robotic wheelchair based on observations of people using integrated sensors. In: IEEE/RSJ international conference on intelligent robots and systems, pp 2013–2018

  59. Suzuki R, Yamada T, Arai M, Sato Y, Kobayashi Y, Kuno Y (2014) Multiple robotic wheelchair system considering group communication. In: International symposium on visual computing. Springer, pp 805–814

  60. Herrero Cotarelo F (2012) Detección automática de personas mediante láser 2d y su aplicación a la robótica de servicios

  61. Arras KO, Mozos OM, Burgard W (2007) Using boosted features for the detection of people in 2d range data. In: Proceedings 2007 IEEE international conference on robotics and automation. IEEE, pp 3402–3407

  62. Reid D (1979) An algorithm for tracking multiple targets. IEEE Trans Autom Control 24(6):843–854

    Article  Google Scholar 

  63. Luber M, Diego Tipaldi G, Arras KO (2011) Place-dependent people tracking. Int J Robot Res 30(3):280–293

    Article  Google Scholar 

  64. Vaquero V, Repiso E, Sanfeliu A (2019) Robust and real-time detection and tracking of moving objects with minimum 2d LiDAR information to advance autonomous cargo handling in ports. Sensors 19(1):107

    Article  Google Scholar 

  65. Ferrer G, Sanfeliu A (2014) Behavior estimation for a complete framework for human motion prediction in crowded environments. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 5940–5945

  66. Luber M, Stork JA, Tipaldi GD, Arras KO (2010) People tracking with human motion predictions from social forces. In: 2010 IEEE international conference on robotics and automation. IEEE, pp 464–469

  67. Zanlungo F, Ikeda T, Kanda T (2011) Social force model with explicit collision prediction. EPL (Europhys Lett) 93(6):68005

    Article  Google Scholar 

  68. Zanlungo F, Ikeda T, Kanda T (2014) Potential for the dynamics of pedestrians in a socially interacting group. Phys Rev E 89(1):012811

    Article  Google Scholar 

  69. Weiss A, Bernhaupt R, Lankes M, Tscheligi M (2009) The usus evaluation framework for human-robot interaction. In: AISB2009: proceedings of the symposium on new frontiers in human-robot interaction 4(1):Citeseer, pp 11–26

  70. Garrell A, Villamizar M, Moreno-Noguer F, Sanfeliu A (2017) Teaching robot’s proactive behavior using human assistance. Int J Soc Robot 2(9):231–249

    Article  Google Scholar 

  71. Ji T, Pachi A (2005) Frequency and velocity of people walking. Struct Eng 84(3):36–40

    Google Scholar 

  72. Ge W, Collins RT, Ruback B (2009) Automatically detecting the small group structure of a crowd. In: 2009 workshop on applications of computer vision (WACV). IEEE, pp 1–8

  73. Zanlungo F, Yücel Z, Kanda T (2019) Intrinsic group behaviour ii: On the dependence of triad spatial dynamics on social and personal features; and on the effect of social interaction on small group dynamics. PLoS ONE 14(12):e0225704

    Article  Google Scholar 

  74. Gross H-M, Boehme H, Schroeter C, Müller S, König A, Einhorn E, Martin C, Merten M, Bley A (2009) Toomas: interactive shopping guide robots in everyday use-final implementation and experiences from long-term field trials. In: IEEE/RSJ international conference on intelligent robots and systems, pp 2005–2012

  75. Chen Y, Wu F, Shuai W, Wang N, Chen R, Chen X (2015) Kejia robot–an attractive shopping mall guider. In: International conference on social robotics. Springer, pp 145–154

  76. Gross H-M, Schroeter C, Mueller S, Volkhardt M, Einhorn E, Bley A, Martin C, Langner T, Merten M (2011) Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment. In: IEEE/RSJ international conference on intelligent robots and systems, pp 2430–2437

  77. Zhao P, Sun L, Cui L, Luo W, Ding Y (2016) The walking behaviours of pedestrian social group in the corridor of subway station. In: Proceedings of the 2016 Pedestrian and evacuation dynamics conference

Download references

Acknowledgements

Work supported by the Spanish Ministry of Science and Innovation under project ROCOTRANSP, and the EU project TERRINET. Ely Repiso was also supported during her PhD by Spanish Ministry of Science and Innovation under a FPI-grant (BES-2014-067713), and under the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656) during her last year. The Japan stay was supported by Spanish Ministry of Science and Innovation under an Aid to predoctoral mobility for short stays in R & D centers 2017 (EEBB-I-18-12958). We need to thank the contributions of Takayuki Kanda and Francesco Zanlungo during the Japan stay and in the IROS2019 paper, which allows us to develop the method of robot’s group accompaniment using a V-formation. Finally, we must thank Aurélie Clodic and the AI4HRI project that supports Ely Repiso Polo during this last year and allows her to do part of her work at the end of this article. (This work has been supported during the review process by the Artificial Intelligence for Human-Robot Interaction (AI4HRI) project ANR-20-IADJ-0006)

Funding

ROCOTRANSP: National Project. Funder: Ministerio de Ciencia e Innovacion (MCIN) y Agencia Española de Investigacion (AEI). Award Number: PID2019-106702RB-C21 MCIN / AEI/10.13039/501100011033. Grant Recipient: Alberto Sanfeliu Cortés (UPC). TERRINET: European Project. Funder: European Commission. Award Number: H2020-INFRAIA-2017-1-two-stage-730994. Grant Recipient: Alberto Sanfeliu Cortés (UPC).

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial (CSIC-UPC), Llorens Artigas 4-6, 08028, Barcelona, Spain

    Ely Repiso, Anaís Garrell & Alberto Sanfeliu

  2. LAAS-CNRS, Université de Toulouse, CNRS, Toulouse, France, 7, Avenue du Colonel Roche, BP 54200, 31031, Toulouse cedex 4, France

    Ely Repiso

Authors
  1. Ely Repiso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anaís Garrell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto Sanfeliu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ely Repiso.

Ethics declarations

Conflict of Interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical Approval

This paper or a similar version is not currently under review by a journal or conference. This paper is void of plagiarism or self-plagiarism as defined by the Committee on Publication Ethics and Springer Guidelines. The Ethics Review Board of the Universitat Politecnica de Catalunya approved the study, with title: Navegación Robot-Humano Colaborativa en entornos con personas y predicción de movimiento humano. The Ethics Review Board decision number is: 2021-11 UPC. Attached as supplementary material.

Informed Consent Statement to Participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Repiso, E., Garrell, A. & Sanfeliu, A. Adaptive Social Planner to Accompany People in Real-Life Dynamic Environments. Int J of Soc Robotics 16, 1189–1221 (2024). https://doi.org/10.1007/s12369-022-00937-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-022-00937-3

Keywords

Search

Navigation