Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Generating collective behavior of a robotic swarm using an attention agent with deep neuroevolution

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

This paper focuses on generating collective behavior of a robotic swarm using an attention agent. The selective attention mechanism enables an agent to cope with environmental variations which are irrelevant to the task. This paper applies attention mechanisms to a robotic swarm for enhancing system-level properties, such as flexibility or scalability. To train an attention agent, evolutionary computations become a promising method, because a controller structure is not restricted by a gradient-based method. Therefore, this paper employs a deep neuroevolution approach to generating collective behavior in a robotic swarm. The experiments are conducted by computer simulations that consist of the Unity 3D game engine. The performance of the attention agent is compared with the convolutional neural network approach. The experimental results showed that the attention agent obtained generalization abilities in a robotic swarm similar to single-agent problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Find the latest articles, discoveries, and news in related topics.

References

  1. Şahin E (2005) Swarm robotics: From sources of inspiration to domains of application. In: International workshop on swarm robotics, pp 10–20

  2. Dorigo M et al (2014) Swarm robotics. Scholarpedia 9(1):14–63

    Article  Google Scholar 

  3. Manuele B et al (2013) Swarm robotics: a review from the swarm engineering perspective. Swarm Intell 7(1):1–41

    Article  Google Scholar 

  4. Nolfi S, Floreano D (2000) Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines. MIT Press, Cambridge

    Google Scholar 

  5. Baldassarre Gianluca et al (2003) Evolving mobile robots able to display collective behaviors. Artif Life 9(3):255–267

    Article  Google Scholar 

  6. Onur Soysal et al (2007) Aggregation in swarm robotic systems: evolution and probabilistic control. Turk J Electr Eng Comput Sci 15(2):199–225

    Google Scholar 

  7. Valerio Sperati et al (2011) Self-organised path formation in a swarm of robots. Swarm Intell 5:97–119

    Article  Google Scholar 

  8. Alkilabi MHM et al (2017) Cooperative object transport with a swarm of e-puck robots: robustness and scalability of evolved collective strategies. Swarm Intell 11:185–209

    Article  Google Scholar 

  9. Floreano Dario et al (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62

    Article  Google Scholar 

  10. Geoffrey Hinton et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29:82–97

    Article  Google Scholar 

  11. Yichuan T et al (2012) Deep Lambertian networks. arXiv preprint arXiv:1206.6445

  12. Mohammad Havaei et al (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  13. David Silver et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484–489

    Article  Google Scholar 

  14. Kun S et al (2019) A survey of deep reinforcement learning in video games. arXiv preprint arXiv:1912.10944

  15. Nicolas H et al (2017) Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286

  16. Zhaoming X et al (2019) Iterative reinforcement learning based design of dynamic locomotion skills for cassie. arXiv preprint arXiv:1903.09537

  17. Tabish Rashid et al (2020) Monotonic value function factorisation for deep multi-agent reinforcement learning. J Mach Learn Res 21(1):7234–7284

    MathSciNet  MATH  Google Scholar 

  18. Terry J et al (2021) Pettingzoo: gym for multi-agent reinforcement learning. Ad Neural Inf Process Syst 34:15032–15043

    Google Scholar 

  19. Hüttenrauch et al (2017) Guided deep reinforcement learning for swarm systems. arXiv preprint arXiv:1709.06011

  20. Yixin H et al (2020) A multi-agent reinforcement learning method for swarm robots in space collaborative exploration. In: 2020 6th international conference on control, automation and robotics (ICCAR), pp 139–144

  21. Lipton Zachary C (2018) The mythos of model interpretability: in machine learning, the concept of interpretability is both important and slippery. Queue 16(3):31–57

    Article  Google Scholar 

  22. Bolei Z et al (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

  23. Andrei K et al (2019) Xrai: better attributions through regions. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4948–4957

  24. Petroski SF et al (2017) Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprint arXiv:1712.06567

  25. Tim S et al (2017) Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864

  26. Tang Y et al (2020) Neuroevolution of self-interpretable agents. In: Proceedings of the 2020 genetic and evolutionary computation conference, pp 414–424

  27. Spelke Elizabeth S, Kinzler Katherine D (2007) Core knowledge. Dev Sci 10(1):89–96

    Article  Google Scholar 

  28. Yuezhang L et al (2018) An initial attempt of combining visual selective attention with deep reinforcement learning. arXiv preprint arXiv1811.04407

  29. Mott A et al (2019) Towards interpretable reinforcement learning using attention augmented agents. Adv Neural Inf Process Syst 32

  30. Ha D, Schmidhuber J (2018) World models. arXiv preprint arXiv:1803.10122

  31. Risi S, Stanley KO (2019) Deep neuroevolution of recurrent and discrete world models. In: Proceedings of the genetic and evolutionary computation conference

  32. Unity Technologies. Unity—game engine. https://unity3d.com/

  33. Wierstra D et al (2014) Natural evolution strategies. J Mach Learn Res 15(1):949–980

    MathSciNet  MATH  Google Scholar 

  34. He K et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026 –1034

  35. John S et al (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

  36. Dosovitskiy A et al (2019) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

  37. Liu Z et al (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022

Download references

Author information

Authors and Affiliations

  1. Hiroshima University, Hiroshima, Japan

    Arumu Iwami, Daichi Morimoto, Naoya Shiozaki & Kazuhiro Ohkura

  2. Kyoto Institute of Technology, Kyoto, Japan

    Motoaki Hiraga

Authors
  1. Arumu Iwami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daichi Morimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naoya Shiozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Motoaki Hiraga
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kazuhiro Ohkura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arumu Iwami.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part at the joint symposium of the 28th International Symposium on Artificial Life and Robotics, the 8th International Symposium on BioComplexity, and the 6th International Symposium on Swarm Behavior and Bio-Inspired Robotics (Beppu, Oita and Online, January 25–27, 2023).

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iwami, A., Morimoto, D., Shiozaki, N. et al. Generating collective behavior of a robotic swarm using an attention agent with deep neuroevolution. Artif Life Robotics 28, 669–679 (2023). https://doi.org/10.1007/s10015-023-00902-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-023-00902-x

Keywords

Search

Navigation