Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

SpringerLink
Log in

Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Jump locomotion is the basic movement of human. However, no thorough research on the recognition of jump sub-phases has been carried so far. This paper aims to use multi-sensor information fusion and machine learning to recognize the human jump phase, which is crucial to the development of exoskeleton that assists jumping. The method of information fusion for sensors including sEMG, IMU, and footswitch sensor is studied. The footswitch signals are filtered by median filter. A processing method of synthesizing Euler angles into phase angle is proposed, which is beneficial to data integration. The jump locomotion is creatively segmented into five phases. The onset and offset of active segment are detected by sample entropy of sEMG and standard deviation of acceleration signal. The features are extracted from analysis windows using multi-sensor information fusion, and the dimension of feature matrix is selected. By comparing the performances of state-of-the-art machine learning classifiers, feature subsets of sEMG, IMU, and footswitch signals are selected from time domain features in a series of analysis window parameters. The average recognition accuracy of sEMG and IMU is 91.76% and 97.68%, respectively. When using the combination of sEMG, IMU, and footswitch signals, the average accuracy is 98.70%, which outperforms the combination of sEMG and IMU (97.97%, p < 0.01).

The sub-phases of human locomotion are recognized based on multi-sensor information fusion and machine learning method. The feature data of the sub-phases is visualized in 3-dimensional space. The predicted states and the true states in a complete jump are compared along the time axis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Wang I L, Hsiao C Y, Shen J, Wang Y, Huang C C, Chen Y M (2019) The effects of Jilin sika Deer’s (Cervus dybowski) tendon liquid supplementation on endurance drop jumps performance, biochemistry profile of free boxing players. J Ethnopharmacol 245:1–10. https://doi.org/10.1016/j.jep.2019.112119

    Article  Google Scholar 

  2. Hwang S H, Lee S C, Shin D B, Baek I H, Kim M J, Sun D I, Kim B S, Hwang S W, Han C S (2019) Intuitive gait pattern generation for an exoskeleton robot. Int J Precis Eng Manuf 20:1905–1913. https://doi.org/10.1007/s12541-019-00184-z

    Article  Google Scholar 

  3. Boudali A M, Sinclair P J, Manchester I R (2019) Predicting transitioning walking gaits: hip and knee joint trajectories fromthe motion of walking canes. IEEE Trans Neural Syst Rehabil Eng 27:1791–1800. https://doi.org/10.1109/TNSRE.2019.2933896

    Article  Google Scholar 

  4. Organero M M, Littlewood C, Parker J, Powell L, Grindell C, Mawson S (2017) Identification of walking strategies of people with osteoarthritis of the knee using insole pressure sensors. IEEE Sensors J 17:3909–3920. https://doi.org/10.1109/JSEN.2017.2696303

    Article  Google Scholar 

  5. Cappa D F, Behm D G (2011) Training specificity of hurdle vs. countermovement jump training. J Strength Cond Res 25:2715–2720. https://doi.org/10.1519/jsc.0b013e318208d43c

    Article  PubMed  Google Scholar 

  6. Lorenzetti S, Ammann F, Windmuller S, Haberle R, Muller S, Gross M, Pluss M, Pluss S, Schodler B, Hubner K (2019) Conditioning exercises in ski jumping: biomechanical relationship of squat jumps, imitation jumps, and hill jumps. Sports Biomech 18:63–74. https://doi.org/10.1080/14763141.2017.1383506

    Article  PubMed  Google Scholar 

  7. Ramasso E, Rombaut M, Pellerin D (2007) State filtering and change detection using TBM conflict application to human action recognition in athletics videos. IEEE Trans Circ Syst Video Technol 17:944–949. https://doi.org/10.1109/tcsvt.2007.896652

    Article  Google Scholar 

  8. Panagiotakis C, Ramasso E, Tziritas G, Rombaut M, Pellerin D (2006) Shape-motion based athlete tracking for multilevel action recognition. Articulated Motion Deformable Objects 4069:385–394. https://doi.org/10.1007/11789239_40

    Article  Google Scholar 

  9. Ramasso E, Panagiotakis C, Pellerin D, Rombaut M (2007) Human action recognition in videos based on the transferable belief model. Pattern Anal Applic 11:1–19. https://doi.org/10.1007/s10044-007-0073-y

    Article  Google Scholar 

  10. Ramasso E, Panagiotakis C, Rombaut M, Pellerin D (2010) Belief scheduler based on model failure detection in the TBM framework. Int J Approx Reason 51:846–865. https://doi.org/10.1016/j.ijar.2010.04.005

    Article  Google Scholar 

  11. Chen H, Ma K, Chuang J, Lin H (2013) Recognizing jump patterns with physics-based validation in human moving trajectory. J Vis Commun Image Represent 24:1191–1203. https://doi.org/10.1016/j.jvcir.2013.08.006

    Article  Google Scholar 

  12. Si W, Yang G, Chen X, Jia J (2018) Gait identification using fractal analysis and support vector machine. Soft Comput 23:9287–9297. https://doi.org/10.1007/s00500-018-3609-8

    Article  Google Scholar 

  13. Brock H, Ohgi Y (2017) Assessing motion style errors in ski jumping using inertial sensor devices. IEEE Sensors J 17:3794–3804. https://doi.org/10.1109/JSEN.2017.2699162

    Article  Google Scholar 

  14. Nielsen G, Holmgaard S, Jiang N, Englehart K, Farina D, Parker P A (2011) Simultaneous and proportional force estimation for multifunction myoelectric prostheses using mirrored bilateral training. IEEE Trans Biomed Eng 58:681–688. https://doi.org/10.1109/tbme.2010.2068298

    Article  PubMed  Google Scholar 

  15. Scheme E, Englehart K (2011) Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev 48:643–659. https://doi.org/10.1682/jrrd.2010.09.0177

    Article  PubMed  Google Scholar 

  16. Young A J, Kuiken T A, Hargrove L J (2014) Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses. J Neural Eng 11:1–12. https://doi.org/10.1088/1741-2560/11/5/056021

    Article  Google Scholar 

  17. Jochumsen M, Waris A, Kamavuako E N (2018) The effect of arm position on classification of hand gestures with intramuscular EMG. Biomed Sig Process Control 43:1–8. https://doi.org/10.1016/j.bspc.2018.02.013

    Article  Google Scholar 

  18. Bennett D A, Goldfarb M (2018) IMU-based wrist rotation control of a transradial myoelectric prosthesis. IEEE Trans Neural Syst Rehabil Eng 26:419–427. https://doi.org/10.1109/TNSRE.2017.2682642

    Article  PubMed  Google Scholar 

  19. Kamizi M A, Negri L H, Fabris J L, Muller M (2019) A smartphone based fiber sensor for recognizing walking patterns. IEEE Sensors J 19:9782–9789. https://doi.org/10.1109/JSEN.2019.2927456

    Article  Google Scholar 

  20. Jeong G, Truong P, Choi S (2017) Classification of three types of walking activities regarding stairs using plantar pressure sensors. IEEE Sensors J 17:2638–2639. https://doi.org/10.1109/JSEN.2017.2682322

    Article  Google Scholar 

  21. Ding S, Ouyang X, Liu T, Li Z, Yang H (2018) Gait event detection of a lower extremity exoskeleton robot by an intelligent IMU. IEEE Sensors J 18:9728–9735. https://doi.org/10.1109/JSEN.2018.2871328

    Article  Google Scholar 

  22. Mazumder O, Kundu A S, Lenka P K, Bhaumik S (2016) Ambulatory activity classification with dendogram-based support vector machine: application in lower-limb active exoskeleton. Gait Posture 50:53–59. https://doi.org/10.1016/j.gaitpost.2016.08.010

    Article  PubMed  Google Scholar 

  23. Lee S W, Yi T, Jung J W, Bien Z (2017) Design of a gait phase recognition system that can cope with EMG electrode location variation. IEEE Trans Autom Sci Eng 14:1429–1439. https://doi.org/10.1109/TASE.2015.2477283

    Article  Google Scholar 

  24. He J, Zhang D, Sheng X, Li S, Zhu X (2015) Invariant surface EMG feature against varying contraction level for myoelectric control based on muscle coordination. IEEE J Biomed Health Inform 19:874–882. https://doi.org/10.1109/JBHI.2014.2330356

    PubMed  Google Scholar 

  25. Spanias J A, Perreault E J, Hargrove L J (2016) Detection of and compensation for EMG disturbances for powered lower limb prosthesis control. IEEE Trans Neural Syst Rehabil Eng 24:226–234. https://doi.org/10.1109/TNSRE.2015.2413393

    Article  PubMed  Google Scholar 

  26. Yi C, Jiang F, Bhuiyan M Z A, Yang C, Gao X, Guo H, Ma J, Su S (2021) Smart healthcare-oriented online prediction of lower-limb kinematics and kinetics based on data-driven neural signal decoding. Futur Gener Comput Syst 114:96–105. https://doi.org/10.1016/j.future.2020.06.015

    Article  Google Scholar 

  27. Hargrove L J, Young A J, Simon A M, Fey N P, Lipschutz R D, Finucane S B, Halsne E G, Ingraham K A, Kuiken T A (2015) Intuitive control of a powered prosthetic leg during ambulation: a randomized clinical trial. JAMA 313:2244–2252. https://doi.org/10.1001/jama.2015.4527

    Article  CAS  PubMed  Google Scholar 

  28. Abbaspour S, Lindén M, Gholamhosseini H, Naber A, Catalan M (2020) Evaluation of surface EMG-based recognition algorithms for decoding hand movements. Med Biol Eng Comput 58:83–100. https://doi.org/10.1007/s11517-019-02073-z

    Article  PubMed  Google Scholar 

  29. Jochumsen M, Waris A, Kamavuako E N (2018) The effect of arm position on classification of hand gestures with intramuscular EMG. Biomed Sig Process Control 43:1–8. https://doi.org/10.1016/j.bspc.2018.02.013

    Article  Google Scholar 

  30. Chen B, Zi B, Wang Z, Qin L, Liao W H (2019) Knee exoskeletons for gait rehabilitation and human performance augmentation: a state-of-the-art. Mech Mach Theory 134:499–511. https://doi.org/10.1016/j.mechmachtheory.2019.01.016

    Article  Google Scholar 

  31. Lu Z, Stampas A, Francisco G E, Zhou P (2019) Offline and online myoelectric pattern recognition analysis and real-time control of a robotic hand after spinal cord injury. J Neural Eng 16:1–10. https://doi.org/10.1088/1741-2552/ab0cf0

    Article  Google Scholar 

  32. Marques N R, Hallal C Z, Spinoso D H, Morcelli M H, Crozara L F, Goncalves M (2017) Applying different mathematical variability methods to identify older fallers and non-fallers using gait variability data. Aging Clin Exp Res 29:473–481. https://doi.org/10.1007/s40520-016-0592-8

    Article  PubMed  Google Scholar 

  33. Lu Y, Wang H, Zhou B, Hu F, Xi H, Zhang Z, Li Y (2019) Human’s jump state recognition based on surface electromyography and wearable plantar-ground contact sensor. 2019 WRC symposium on advanced robotics and automation (WRC SARA), Beijing, China, pp 242–247. https://doi.org/10.1109/WRC-SARA.2019.8931957

  34. Han M, Günay S Y, Schirner G, Padır T, Erdoğmuş D (2019) HANDS: a multimodal dataset for modeling toward human grasp intent inference in prosthetic hands. Intel Serv Robotics 13:179–185. https://doi.org/10.1007/s11370-019-00293-8

    Article  PubMed  Google Scholar 

  35. Phinyomark A, Khushaba R N, Scheme E (2018) Feature extraction and selection for myoelectric control based on wearable EMG sensors. Sensors 18:1–17. https://doi.org/10.3390/s18051615

    Article  Google Scholar 

  36. Abbaspour S, Lindén M, Gholamhosseini H, Naber A, Catalan M (2020) Evaluation of surface EMG-based recognition algorithms for decoding hand movements. Med Biol Eng Comput 58:83–100. https://doi.org/10.1007/s11517-019-02073-z

    Article  PubMed  Google Scholar 

  37. Atzori M, Gijsberts A, Castellini C, Caputo B, Hager A G, Elsig S, Giatsidis G, Bassetto F, Muller H (2014) Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci Data 1:1–13. https://doi.org/10.1038/sdata.2014.53

    Article  Google Scholar 

  38. Menon R, Caterina G D, Lakany H, Petropoulakis L, Conway B A, Soraghan J (2017) Study on interaction between temporal and spatial information in classification of EMG signals for myoelectric prostheses. IEEE Trans Neural Syst Rehabil Eng 25:1832–1842. https://doi.org/10.1109/TNSRE.2017.2687761

    Article  PubMed  Google Scholar 

  39. Li Y, Zhang X, Gong Y, Cheng Y, Gao X, Chen X (2017) Motor function evaluation of hemiplegic upper-extremities using data fusion from wearable inertial and surface EMG sensors. Sensors 17:1–17. https://doi.org/10.3390/s17030582

    Article  Google Scholar 

  40. Zhang X, Chen X, Li Y, Lantz V, Wang K, Yang J (2011) A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans Syst Man Syst Hum 41:1064–1076. https://doi.org/10.1109/TSMCA.2011.2116004

    Article  Google Scholar 

  41. Kundu A S, Mazumder O, Lenka P K, Bhaumik S (2017) Hand gesture recognition based omnidirectional wheelchair control using IMU and EMG sensors. J Intell Robot Syst 91:529–541. https://doi.org/10.1007/s10846-017-0725-0

    Article  Google Scholar 

  42. Zhou P, Zhang X (2014) A novel technique for muscle onset detection using surface EMG signals without removal of ECG artifacts. Physiol Meas 35:45–54. https://doi.org/10.1088/0967-3334/35/1/45

    Article  PubMed  Google Scholar 

  43. Chen W, Zhuang J, Yu W, Wang Z (2009) Measuring complexity using FuzzyEn, ApEn, and SampEn. Med Eng Phys 31:61–68. https://doi.org/10.1016/j.medengphy.2008.04.005

    Article  PubMed  Google Scholar 

  44. Diab A, Hassan M, Karlsson B, Marque C (2013) Effect of decimation on the classification rate of non-linear analysis methods applied to uterine EMG signals. IRBM 34:326–329. https://doi.org/10.1016/j.irbm.2013.07.010

    Article  Google Scholar 

  45. Englehart K, Hudgins B (2003) A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng 50:848–854. https://doi.org/10.1001/jama.2015.4527

    Article  PubMed  Google Scholar 

  46. Rechy-Ramirez E J, Hu H (2015) Bio-signal based control in assistive robots: a survey. Digit Commun Netw 1:85–101. https://doi.org/10.1016/j.dcan.2015.02.004

    Article  Google Scholar 

  47. Smith L H, Hargrove L J, Lock B A, Kuiken T A (2011) Determining the optimal window length for pattern recognition-based myoelectric control: balancing the competing effects of classification error and controller delay. IEEE Trans Neural Syst Rehabil Eng 19:186–192. https://doi.org/10.1109/TNSRE.2010.2100828

    Article  PubMed  Google Scholar 

  48. Iskarous M M, Thakor N V (2019) E-Skins: biomimetic sensing and encoding for upper limb prostheses. Proc IEEE 107:2052–2064. https://doi.org/10.1109/JPROC.2019.2939369

    Article  Google Scholar 

  49. Bulea T C, Prasad S, Kilicarslan A, Contreras-Vidal J L (2014) Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution. Front Neurosci 8:1–19. https://doi.org/10.3389/fnins.2014.00376

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank all the volunteers for their help.

Funding

This study received funding from the National Key R & D Program of China (2017YFB1300300).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China

    Yanzheng Lu, Hong Wang, Fo Hu, Bin Zhou & Hailong Xi

Authors
  1. Yanzheng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hailong Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, Y., Wang, H., Hu, F. et al. Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning. Med Biol Eng Comput 59, 883–899 (2021). https://doi.org/10.1007/s11517-021-02335-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02335-9

Keywords

Search

Navigation