Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Bionic Hand Motion Control Method Based on Imitation of Human Hand Movements and Reinforcement Learning

  • Research Article
  • Published:
Journal of Bionic Engineering Aims and scope Submit manuscript

Abstract

Bionic hands are promising devices for assisting individuals with hand disabilities in rehabilitation robotics. Controlled primarily by bioelectrical signals such as myoelectricity and EEG, these hands can compensate for lost hand functions. However, developing model-based controllers for bionic hands is challenging and time-consuming due to varying control parameters and unknown application environments. To address these challenges, we propose a model-free approach using reinforcement learning (RL) for designing bionic hand controllers. Our method involves mimicking real human hand motion with the bionic hand and employing a human hand motion decomposition technique to learn complex motions from simpler ones. This approach significantly reduces the training time required. By utilizing real human hand motion data, we design a multidimensional sampling proximal policy optimization (PPO) algorithm that enables efficient motion control of the bionic hand. To validate the effectiveness of our approach, we compare it against advanced baseline methods. The results demonstrate the quick learning capabilities and high control success rate of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The raw/processed data required to reproduce these findings cannot be shared at this time as the data also form part of an ongoing study.

References

  1. Fu, Q. S., Shao, F. C., & Santello, M. (2019). Inter-limb transfer of grasp force perception with closed-loop hand prosthesis. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(5), 927–936.

    Article  Google Scholar 

  2. Scheme, E., Lock, B., Hargrove, L., Hill, W., Kuruganti, U., & Englehart, K. (2013). Motion normalized proportional control for improved pattern recognition-based myoelectric control. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(1), 149–157.

    Article  Google Scholar 

  3. Gan, Y. H., & Dai, X. Z. (2012). Human-like manipulation planning for articulated manipulator. Journal of Bionic Engineering, 9(4), 434–445.

    Article  Google Scholar 

  4. Park, H., Kwak, B., & Bae, J. (2018). Inverse kinematics analysis and COG trajectory planning algorithms for stable walking of a quadruped robot with redundant DOFs. Journal of Bionic Engineering, 15, 610–622.

    Article  Google Scholar 

  5. Liu, W. J., Zl, Jing, Pan, H., Qiao, L. F., Leung, H., & Chen, W. J. (2020). Distance-directed target searching for a deep visual servo sma driven soft robot using reinforcement learning. Journal of Bionic Engineering, 17, 1126–1138.

    Article  Google Scholar 

  6. Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., et al. (2020). Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1), 3–20.

    Article  Google Scholar 

  7. Khodadadi, N., Soleimanian Gharehchopogh, F., & Mirjalili, S. (2022). MOAVOA: a new multi-objective artificial vultures optimization algorithm. Neural Computing and Applications, 34(23), 20791–20829.

    Article  Google Scholar 

  8. Sorkhabi, L. B., Gharehchopogh, F. S., & Shahamfar, J. (2020). A systematic approach for pre-processing electronic health records for mining: Case study of heart disease. International Journal of Data Mining and Bioinformatics, 24(2), 97–120.

    Article  Google Scholar 

  9. Gharehchopogh, F. S., Namazi, M., Ebrahimi, L., & Abdollahzadeh, B. (2023). Advances in sparrow search algorithm: a comprehensive survey. Archives of Computational Methods in Engineering, 30(1), 427–455.

    Article  Google Scholar 

  10. Gharehchopogh, F. S., Ucan, A., Ibrikci, T., Arasteh, B., & Isik, G. (2023). Slime mould algorithm: A comprehensive survey of its variants and applications. Archives of Computational Methods in Engineering, 30(4), 2683–2723.

    Article  Google Scholar 

  11. Gharehchopogh, F. S. (2023). An improved Harris Hawks optimization algorithm with multi-strategy for community detection in social network. Journal of Bionic Engineering, 20(3), 1175–1197.

    Article  Google Scholar 

  12. Gharehchopogh, F. S. (2023). Quantum-inspired metaheuristic algorithms: comprehensive survey and classification. Artificial Intelligence Review, 56(6), 5479–5543.

    Article  Google Scholar 

  13. Shishavan, S. T., & Gharehchopogh, F. S. (2022). An improved cuckoo search optimization algorithm with genetic algorithm for community detection in complex networks. Multimedia Tools and Applications, 81(18), 25205–25231.

    Article  Google Scholar 

  14. Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C. L., & Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking. arXiv preprint arXiv:2006.10214.

  15. Feix, T., Romero, J., Schmiedmayer, H., Dollar, A., & Kragic, D. (2015). The grasp taxonomy of human grasp types. IEEE Transactions on Human-Machine Systems, 46(1), 66–77.

    Article  Google Scholar 

  16. Soltoggio, A., Lemme, A., & Steil, J. (2012). Using movement primitives in interpreting and decomposing complex trajectories in learning-by-doing. 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), Guangzhou, China, pp. 1427–1433.

  17. Luo, P., Zhang, M., Cheng, C. C., & Zhang, J. S. (2021). Stable grasping control of three-finger manipulator based on sliding detection. 2021 China Automation Congress (CAC), Beijing, China, pp. 814–818.

  18. Xu, J. J., & Li, Z. X. (2008). A kinematic model of finger gaits by multifingered hand as hybrid automaton. IEEE Transactions on Automation Science and Engineering, 5(3), 467–479.

    Article  Google Scholar 

  19. Fan, Y. X., Tang, T., Lin, H. C., Zhao, Y., & Tomizuka, M. (2017). Real-time robust finger gaits planning under object shape and dynamics uncertainties. In B. C. Vancouver (Ed.), 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1267–1273). Canada.

    Chapter  Google Scholar 

  20. Morgan, A., Hang, K. Y., Wen, B., Bekris, K., & Dollar, A. (2022). Complex in-hand manipulation via compliance-enabled finger gaiting and multi-modal planning. IEEE Robotics and Automation Letters, 7(2), 4821–4828.

    Article  Google Scholar 

  21. Yoshida, M., Arimoto, S., & Tahara, K. (2009). Manipulation of 2D object with arbitrary shape by robot finger under rolling constraint. 2009 ICCAS-SICE, Fukuoka, Japan, pp. 695–699.

  22. Doulgeri, Z., & Droukas, L. (2013). On rolling contact motion by robotic fingers via prescribed performance control.2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, pp. 3976–3981.

  23. Obregón-Flores, J., Arechavaleta, G., Becerra, H., & Morales-Díaz, A. (2021). Predefined-time robust hierarchical inverse dynamics on torque-controlled redundant manipulators. IEEE Transactions on Robotics, 37(3), 962–978.

    Article  Google Scholar 

  24. Ma, B., Dong, B., Zhou, F., & Li, Y. C. (2020). Adaptive dynamic programming-based fault-tolerant position-force control of constrained reconfigurable manipulators. IEEE Access, 8, 183286–183299.

    Article  Google Scholar 

  25. Son, C. M. (2014). Intelligent jamming region division with machine learning and fuzzy optimization for control of robot’s part micro-manipulative task. Information Science, 256, 211–224.

    Article  MathSciNet  Google Scholar 

  26. Chen, G., Chen, Z. F., Wang, L. M., & Zhang, W. G. (2023). Deep deterministic policy gradient and active disturbance rejection controller based coordinated control for gearshift manipulator of driving robot. Engineering Applications of Artificial Intelligence, 117, 105586.

    Article  Google Scholar 

  27. Ficuciello, F., Falco, P., & Calinon, S. (2018). A brief survey on the role of dimensionality reduction in manipulation learning and control. IEEE Robotics and Automation Letters, 3(3), 2608–2615.

    Article  Google Scholar 

  28. Lopes, M., & Santos-Victor, J. (2007). A developmental roadmap for learning by imitation in robots. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 37(2), 308–321.

    Article  Google Scholar 

  29. Nguyen, K., & Perdereau, V. (2011). Arm-hand movement: Imitation of human natural gestures with tenodesis effect. 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, pp. 1459–1464.

  30. Gupta, A., Eppner, C., Levine, S., & Abbeel, P. (2016). Learning dexterous manipulation for a soft robotic hand from human demonstrations. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea (South), pp. 3786–3793.

  31. Yuan, Y. L., Yu, Z. L., Hua, L., Cheng, Y., Li, J. H., & Sang, X. H. (2023). Hierarchical dynamic movement primitive for the smooth movement of robots based on deep reinforcement learning. Applied Intelligence, 53(2), 1417–1434.

    Article  Google Scholar 

  32. Satheeshbabu, S., Uppalapati, N., Chowdhary, G., & Krishnan, G. (2019). Open loop position control of soft continuum arm using deep reinforcement learning. In Q. C. Montreal (Ed.), 2019 International Conference on Robotics and Automation (ICRA) (pp. 5133–5139). Canada.

  33. Naughton, N., Sun, J. R., Tekinalp, A., Parthasarathy, T., Chowdhary, G., & Gazzola, M. (2021). Elastica: A compliant mechanics environment for soft robotic control. IEEE Robotics and Automation Letters, 6(2), 3389–3396.

    Article  Google Scholar 

  34. Kumar, V., Todorov, E., & Levine, S. (2016). Optimal control with learned local models: Application to dexterous manipulation. 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, pp. 378–383.

  35. Van Hoof, H., Hermans, T., Neumann, G., & Peters, J. (2015). Learning robot in-hand manipulation with tactile features. 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea (South), pp. 121–127.

  36. Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.

  37. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

  38. Csiba, D., & Richtárik, P. (2018). Importance sampling for minibatches. The Journal of Machine Learning Research, 19(1), 962–982.

    MathSciNet  Google Scholar 

  39. Karlik, B., & Olgac, A. (2011). Performance analysis of various activation functions in generalized MLP architectures of neural networks. International Journal of Artificial Intelligence and Expert Systems, 1(4), 111–122.

    Google Scholar 

  40. Chen, C. F., Du, Z. J., He, L., Shi, Y. J., Wang, J. Q., & Dong, W. (2021). A novel gait pattern recognition method based on LSTM-CNN for lower limb exoskeleton. Journal of Bionic Engineering, 18, 1059–1072.

    Article  Google Scholar 

  41. Han, K., Xiao, A., Wu, E. H., Guo, J. Y., Xu, C. J., & Wang, Y. H. (2021). Transformer in transformer. Advances in Neural Information Processing Systems, 34, 15908–15919.

    Google Scholar 

  42. Xu, J. J., Sun, X., Zhang, Z. Y., Zhao, G. X., Lin, J. Y. (2019). Understanding and improving layer normalization. Advances in Neural Information Processing Systems, 32.

  43. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

  44. Sharma, D., Tokas, K., Puri, A., & Sharda, K. (2014). Shadow hand. Journal of Advance Research in Applied Science, 1(1), 04–07.

    Google Scholar 

  45. Todorov, E., Erez, T., & Tassa, Y. (2012). Mujoco: A physics engine for model-based control. 2012 IEEE/RSJ international conference on intelligent robots and systems, Vilamoura-Algarve, Portugal, pp. 5026–5033.

  46. Bai, J. B., Li, B. J., Wang, H. Y., & Guo, Y. T. (2022). Tactile perception information recognition of prosthetic hand based on dnn-lstm. IEEE Transactions on Instrumentation and Measurement, 71, 1–10.

    Google Scholar 

Download references

Funding

This work was supported by Su Yan Yuan (“Development and industrialization of intelligent multi-degree-of-freedom arm based on perceptual fusion and collaborative control" (Su Yan Yuan [2019] No. 107)) and Shanghai DianJi University(“Research on flexible joint and adaptive control technology for new upper limb prosthesis" (scientific research start-up fund project of Shanghai DianJi University) and “Research on robot intelligent grasping technology based on visual touch fusion in unstructured environment" (Science and technology [2020] No. 79 of Shanghai DianJi University)).

Author information

Author notes

  1. Baojiang Li, Xichao Wang, Haiyan Wang, and Yuting Guo have contributed equally to this work.

Authors and Affiliations

  1. The School of Electrical Engineering, Shanghai DianJi University, Shanghai, 201306, China

    Jibo Bai, Baojiang Li, Xichao Wang, Haiyan Wang & Yuting Guo

Authors
  1. Jibo Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baojiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xichao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haiyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuting Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by JB, YG, HW and BL. The first draft of the manuscript was written by JB and all the authors commented on previous versions of the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Baojiang Li.

Ethics declarations

Conflict of Interest

Financial interests: The authors Jibo Bai, Haiyan Wang and Yutin Guo declare they have no financial interests. The author Baojiang Li received research funding from Su Yan Yuan and Shanghai DianJi University.

Ethical Approval

The study did not address ethical issues.

Consent to Participate

Informed consent was obtained from all the individual participants included in the study.

Consent to Publish

The authors affirm that human research participants provided informed consent for publication of the images in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bai, J., Li, B., Wang, X. et al. Bionic Hand Motion Control Method Based on Imitation of Human Hand Movements and Reinforcement Learning. J Bionic Eng 21, 764–777 (2024). https://doi.org/10.1007/s42235-023-00472-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42235-023-00472-5

Keywords

Search

Navigation