Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Integration of a Lightweight Customized 2D CNN Model to an Edge Computing System for Real-Time Multiple Gesture Recognition

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The human-machine interface (HMI) collects electrophysiology signals incoming from the patient and utilizes them to operate the device. However, most applications are currently in the testing phase and are typically unavailable to everyone. Developing wearable HMI devices that are intelligent and more comfortable has been a focus of study in recent times. This work developed a portable, eight-channel electromyography (EMG) signal-based device that can distinguish 21 different types of motion. To identify the EMG signals, an analog front-end (AFE) integrated chip (IC) was created, and an integrated EMG signal acquisition device combining a stretchy wristband was made. Using the EMG movement signals of 10 volunteers, a SIAT database of 21 gestures was created. Using the SIAT dataset, a lightweight 2D CNN-LSTM model was developed and specialized training was given. The signal recognition accuracy is 96.4%, and the training process took a median of 14 min 13 s. The model may be used on lower-performance edge computing devices because of its compact size, and it is anticipated that it will eventually be applied to smartphone terminals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Zheng, Y., Lv, X., Qian, L., Liu, X.: An optimal BP neural network track prediction method based on a GA–ACO hybrid algorithm. J. Mar. Sci. Eng. 10, 10 (2022)

    Article  Google Scholar 

  2. Venugopalan, A., Reghunadhan, R.: Applying deep neural networks for the automatic recognition of sign language words: a communication aid to deaf agriculturists. Expert Syst. Appl. 185 (2021)

  3. Alzubaidi, L., Zhang, J., Al-Dujaili, A., Humaidi, A.J., Duan, Y., Al-Shamma, O., Santamaría, L., Al-Amidie, M., Fadhel, M.A., Farhan, L.: Review of deep learning: concepts, cnn architectures, challenges, applications, future directions. J. Big Data. 8, 1 (2021)

    Article  Google Scholar 

  4. Liu, Z., Wen, C., Su, Z., Liu, S., Sun, J., Kong, W.,…, Yang, Z.: Emotion-semantic-aware dual contrastive learning for epistemic emotion identification of learner-generated reviews in MOOCs. IEEE Trans. Neural Netw. Learn. Syst. (2023)

  5. Qu, J., Mao, B., Li, Z., Xu, Y., Zhou, K., Cao, X., Wang, X.: Recent progress in advanced tactile sensing technologies for soft grippers. Adv. Funct. Mater. 33(41), 2306249 (2023)

    Article  Google Scholar 

  6. Qu, J., Yuan, Q., Li, Z., Wang, Z., Xu, F., Fan, Q., Xu, M.: All-in-one strain-triboelectric sensors based on environment-friendly ionic hydrogel for wearable sensing and underwater soft robotic grasping. Nano Energy 111, 108387 (2023)

    Article  Google Scholar 

  7. Fang, Y., Wang, K., Cheng, J., Lu, H.: A real-time hand gesture recognition method. In: 2007 IEEE International Conference on Multimedia and Expo 995–998 (2022)

  8. Zeng, Q., Bie, B., Guo, Q., Yuan, Y., Han, Q., Han, X.,… Zhou, X.: Hyperpolarized Xe NMR signal advancement by metal-organic framework entrapment in aqueous solution. Proc. Natl. Acad. Sci. 117(30), 17558–17563 (2020)

  9. Kattenborn, T., Leitloff, J., Schiefer, F., Hinz, S.: Review on convolutional neural networks (cnn) in vegetation remote sensing. ISPRS J. Photogramm. Remote Sens. 173, 24–49 (2021)

    Article  Google Scholar 

  10. Bing, P., Liu, Y., Liu, W., Zhou, J., Zhu, L.: Electrocardiogram classification using TSST-based spectrogram and ConViT. Front. Cardiovasc. Med. 9, (2022)

  11. Li, Y., Huang, J., Tian, F., Wang, H.-A., Dai, G.-Z.: Gesture interaction in virtual reality. Virtual Real. Intell. Hardw. 1, 84–112 (2019)

    Article  Google Scholar 

  12. Li, J., Deng, Y., Sun, W., Li, W., Li, R., Li, Q.,… Liu, Z.: Resource orchestration of cloud-edge–based smart grid fault detection. ACM Trans. Sen. Netw. 18(3) (2022)

  13. Mao, Y., You, C., Zhang, J., Huang, K., Letaief, K.B.: A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tutorials 19, 2322–2358 (2017)

    Article  Google Scholar 

  14. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J.C.S., Dustdar, S.,… Liu, J.: Task co-offloading for D2D-assisted mobile edge computing in industrial internet of things. IEEE Trans. Ind. Inform. 19(1), 480–490 (2023)

  15. Jiang, H., Dai, X., Xiao, Z., Iyengar, A.K.: Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Trans. Mobile Comput. (2022)

  16. Shen, X., Jiang, H., Liu, D., Yang, K., Deng, F., Lui, J.C.S.,… Luo, J.: PupilRec:Leveraging pupil morphology for recommending on smartphones. IEEE Internet Things J. 9(17), 15538–15553 (2022)

  17. Jain, R., Karsh, R.K., Barbhuiya, A.A.: Literature review of vision8208;based dynamic gesture recognition using deep learning techniques. Concurr. Comput.: Pract. Exp. 34, 22 (2022)

    Article  Google Scholar 

  18. Dai, X., Xiao, Z., Jiang, H., Lui, J.C.S.: UAV-assisted task offloading in vehicular edge computing networks. IEEE Trans. Mobile Comput. (2023)

  19. Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2020)

    Article  Google Scholar 

  20. Zhang, X., Huang, D., Li, H., Zhang, Y., Xia, Y.,… Liu, J.: Self-training maximum classifier discrepancy for EEG emotion recognition. CAAI Trans. Intell. Technol. (2023)

  21. Li, B., Zhou, X., Ning, Z., Guan, X., Yiu, K.C.: Dynamic event-triggered security control for networked control systems with cyber-attacks: a model predictive control approach. Inf. Sci. 612, 384–398 (2022)

    Article  Google Scholar 

  22. Yan, L., Shi, Y., Wei, M., Wu, Y.: Multi-feature fusing local directional ternary pattern for facial expressions signal recognition based on video communication system. Alexandria Eng. J. 63, 307–320 (2023)

    Article  Google Scholar 

  23. Wang, F., Wang, H., Zhou, X., Fu, R.: A driving fatigue feature detection method based on multifractal theory. IEEE Sens. J. 22(19), 19046–19059 (2022)

    Article  Google Scholar 

  24. Yu, J., Lu, L., Chen, Y., Zhu, Y., Kong, L.: An indirect eavesdropping Attack of keystrokes on touch screen through acoustic sensing. IEEE Trans. Mob. Comput. 20(2), 337–3512021 (2021)

    Article  Google Scholar 

  25. Wang, Y., Han, X., Jin, S.: MAP based modeling method and performance study of a task offloading scheme with time-correlated traffic and VM repair in MEC systems. Wireless Netw. (2022)

  26. Qu, Z., Liu, X., Zheng, M.: Temporal-spatial quantum graph convolutional neural network based on schrödinger approach for traffic congestion prediction. IEEE Trans. Intell. Transp. Syst. (2022)

  27. Ni, Q., Guo, J., Wu, W., Wang, H., Wu, J.: Continuous influence-based community partition for social networks. IEEE Trans. Netw. Sci. Eng. 9(3), 1187–1197 (2022)

    Article  MathSciNet  Google Scholar 

  28. Zhang, J., Tang, Y., Wang, H., Xu, K.: ASRO-DIO: active subspace random optimization based depth inertial odometry. IEEE Trans. Robot. 1–13, (2022)

  29. She, Q., Hu, R., Xu, J., Liu, M., Xu, K.,… Huang, H.: Learning High-DOF reaching-and-grasping via dynamic representation of gripper-object interaction. ACM Trans. Graph., 41(4), (2022)

  30. Cheng, B., Wang, M., Zhao, S., Zhai, Z., Zhu, D.,… Chen, J.: Situation-aware dynamic service coordination in an IoT environment. IEEE/ACM Trans. Netw. 25(4), 2082–2095 (2017)

  31. Zhuang, Y., Jiang, N., Xu, Y., Xiangjie, K., Kong, X.: Progressive distributed and parallel similarity retrieval of large ct image sequences in mobile telemedicine networks. Wirel. Commun. Mobile Comput. (2022)

  32. Tang, Y., Liu, S., Deng, Y., Zhang, Y., Yin, L.,… Zheng, W.: An improved method for soft tissue modeling. Biomed. Signal Process. Control 65, (2021)

  33. Lu, S., Yang, J., Bo, Y.: Analysis and design of surgical instrument localization algorithm. Comput. Model. Eng. Sci. 137(1), 669–685 (2023)

    Google Scholar 

  34. Lu, S., Liu, S., Hou, P., Yang, B., Liu, M., Yin, L.,… Zheng, W.: Soft tissue feature tracking based on deep matching network. Comput. Model. Eng. Sci. 136(1), 363–379 (2023)

Download references

Funding

This research received no specific grant from any funding agency.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Anhui University, Hefei, 230031, China

    Hulin Jin & Chunyang Fan

  2. Foothill Preparatory School, Temple City, CA, 91780, USA

    Zhiran Jin

  3. Department of Computer Engineering, Sejong University, Seoul, 05006, South Korea

    Yong-Guk Kim

Authors
  1. Hulin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiran Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong-Guk Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunyang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Hulin Jin: Conceptualization, Methodology, Formal analysis, Supervision, Writing - original draft, Writing - review & editing. Zhiran Jin: Writing - original draft, Writing - review & editing. Yong-Guk Kim: Investigation, Data Curation, Validation, Resources, Writing - review & editing. Chunyang Fan: Investigation, Data Curation, Validation, Resources, Writing - review & editing.

Corresponding author

Correspondence to Hulin Jin.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, H., Jin, Z., Kim, YG. et al. Integration of a Lightweight Customized 2D CNN Model to an Edge Computing System for Real-Time Multiple Gesture Recognition. J Grid Computing 21, 81 (2023). https://doi.org/10.1007/s10723-023-09715-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-023-09715-5

Keywords

Search

Navigation