Addressing Ergonomic Challenges in Agriculture through AI-Enabled Posture Classification
Abstract
:1. Introduction
2. Related Works
3. Methodology
3.1. Data Collection
3.2. Data Labeling
3.3. Model Training
3.3.1. Convolutional Neural Network (CNN)
3.3.2. Transfer Learning
3.3.3. MoveNet Feature Extraction
3.3.4. Customized Feature Calculation and Selection
3.3.5. Classification of Trunk Posture
3.4. Cross-Validation
3.5. Evaluation Metrics
4. Results
4.1. CNN and Transfer Learning with Pre-Trained Models
4.2. MoveNet and Classification
4.3. Transformed and Selected Features and Classification
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Barneo-Alcántara, M.; Díaz-Pérez, M.; Gómez-Galán, M.; Carreño-Ortega, Á.; Callejón-Ferre, Á.-J. Musculoskeletal disorders in agriculture: A review from web of science core collection. Agronomy 2021, 11, 2017. [Google Scholar] [CrossRef]
- Naeini, H.S.; Karuppiah, K.; Tamrin, S.B.; Dalal, K. Ergonomics in agriculture: An approach in prevention of work-related musculoskeletal disorders (WMSDs). J. Agric. Environ. Sci. 2014, 3, 33–51. [Google Scholar]
- Westgaard, R.H.; Winkel, J. Ergonomic intervention research for improved musculoskeletal health: A critical review. Int. J. Ind. Ergon. 1997, 20, 463–500. [Google Scholar] [CrossRef]
- Punnett, L.; Wegman, D.H. Work-related musculoskeletal disorders: The epidemiologic evidence and the debate. J. Electromyogr. Kinesiol. 2004, 14, 13–23. [Google Scholar] [CrossRef]
- Keyserling, W.M.; Brouwer, M.; Silverstein, B.A. A checklist for evaluating ergonomic risk factors resulting from awkward postures of the legs, trunk and neck. Int. J. Ind. Ergon. 1992, 9, 283–301. [Google Scholar] [CrossRef]
- Karhu, O.; Kansi, P.; Kuorinka, I. Correcting working postures in industry: A practical method for analysis. Appl. Ergon. 1977, 8, 199–201. [Google Scholar] [CrossRef] [PubMed]
- Garg, A.; Kapellusch, J.; Hegmann, K.; Wertsch, J.; Merryweather, A.; Deckow-Schaefer, G.; Malloy, E.J.; The WISTAH Hand Study Research Team. The Strain Index (SI) and Threshold Limit Value (TLV) for Hand Activity Level (HAL): Risk of carpal tunnel syndrome (CTS) in a prospective cohort. Ergonomics 2012, 55, 396–414. [Google Scholar] [CrossRef]
- Kuorinka, I.; Jonsson, B.; Kilbom, A.; Vinterberg, H.; Biering-Sørensen, F.; Andersson, G.; Jørgensen, K. Standardised Nordic questionnaires for the analysis of musculoskeletal symptoms. Appl. Ergon. 1987, 18, 233–237. [Google Scholar] [CrossRef]
- Thamsuwan, O.; Galvin, K.; Tchong-French, M.; Aulck, L.; Boyle, L.N.; Ching, R.P.; McQuade, K.J.; Johnson, P.W. Comparisons of physical exposure between workers harvesting apples on mobile orchard platforms and ladders, Part 2: Repetitive upper arm motions. Appl. Ergon. 2020, 89, 103192. [Google Scholar] [CrossRef]
- Thamsuwan, O.; Galvin, K.; Tchong-French, M.; Aulck, L.; Boyle, L.N.; Ching, R.P.; McQuade, K.J.; Johnson, P.W. Comparisons of physical exposure between workers harvesting apples on mobile orchard platforms and ladders, Part 1: Back and upper arm postures. Appl. Ergon. 2020, 89, 103193. [Google Scholar] [CrossRef]
- Sabatini, A.M. A review of wearable inertial sensors and algorithms for human motion pattern recognition. Sensors 2011, 11, 11556–11565. [Google Scholar]
- Choo, C.Z.Y.; Chow, J.Y.; Komar, J. Validation of the Perception Neuron system for full-body motion capture. PLoS ONE 2022, 17, e0262730. [Google Scholar] [CrossRef] [PubMed]
- Wu, Y.; Tao, K.; Chen, Q.; Tian, Y.; Sun, L. A Comprehensive Analysis of the Validity and Reliability of the Perception Neuron Studio for Upper-Body Motion Capture. Sensors 2022, 22, 6954. [Google Scholar] [CrossRef] [PubMed]
- Seo, J.; Yin, K.; Lee, S. Automated Postural Ergonomic Assessment Using a Computer Vision-Based Posture Classification. Constr. Res. Congr. 2016, 2016, 809–818. [Google Scholar] [CrossRef]
- Bulat, A.; Tzimiropoulos, G. Human Pose Estimation via Convolutional Part Heatmap Regression. Lect. Notes Comput. Sci. 2016, 9911, 717–732. [Google Scholar] [CrossRef]
- Guler, R.A.; Neverova, N.; Kokkinos, I. DensePose: Dense Human Pose Estimation in the Wild. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7297–7306. [Google Scholar] [CrossRef]
- Shotton, J.; Fitzgibbon, A.; Cook, M.; Sharp, T.; Finocchio, M.; Moore, R.; Kipman, A.; Blake, A. Real-time human pose recognition in parts from single depth images. In Proceedings of the CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011; pp. 1297–1304. [Google Scholar] [CrossRef]
- Antwi-Afari, M.F.; Qarout, Y.; Herzallah, R.; Anwer, S.; Umer, W.; Zhang, Y.; Manu, P. Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data. Autom. Constr. 2022, 136, 104181. [Google Scholar] [CrossRef]
- Liaqat, S.; Dashtipour, K.; Arshad, K.; Assaleh, K.; Ramzan, N. A Hybrid Posture Detection Framework: Integrating Machine Learning and Deep Neural Networks. IEEE Sens. J. 2021, 21, 9515–9522. [Google Scholar] [CrossRef]
- Pan, S.J.; Yang, Q. A Survey on Transfer Learning. IEEE Trans. Knowl. Data Eng. 2010, 22, 1345–1359. [Google Scholar] [CrossRef]
- Jose, J.; Shailesh, S. Yoga Asana Identification: A Deep Learning Approach. IOP Conf. Ser. Mater. Sci. Eng. 2021, 1110, 012002. [Google Scholar] [CrossRef]
- Chen, W.; Jiang, Z.; Guo, H.; Ni, X. Fall Detection Based on Key Points of Human-Skeleton Using OpenPose. Symmetry 2020, 12, 744. [Google Scholar] [CrossRef]
- Kendall, A.; Grimes, M.; Cipolla, R. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015. [Google Scholar]
- Zhu, J.; Cheng, C.; Shen, S.; Sun, L. MoveNet: Efficient Convolutional Neural Networks for Real-time Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar]
- Dorado Chaparro, J.; Fernández-Bermejo Ruiz, J.; Santofimia Romero, M.J.; del Toro García, X.; Cantarero Navarro, R.; Bolaños Peño, C.; Llumiguano Solano, H.; Villanueva Molina, F.J.; Gonçalves Silva, A.; López, J.C. Phyx.io: Expert-Based Decision Making for the Selection of At-Home Rehabilitation Solutions for Active and Healthy Aging. Int. J. Environ. Res. Public Health 2022, 19, 5490. [Google Scholar] [CrossRef] [PubMed]
- Feliandra, Z.B.; Khadijah, S.; Rachmadi, M.F.; Chahyati, D. Classification of Stroke and Non-Stroke Patients from Human Body Movements using Smartphone Videos and Deep Neural Networks. In Proceedings of the 2022 International Conference on Advanced Computer Science and Information Systems, Depok, Indonesia, 1–3 October 2022. [Google Scholar] [CrossRef]
- Gu, J.; Wang, Z.; Kuen, J.; Ma, L.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.; Wang, G.; Cai, J.; et al. Recent advances in convolutional neural networks. Pattern Recognit. 2018, 77, 354–377. [Google Scholar] [CrossRef]
- Andriluka, M.; Pishchulin, L.; Gehler, P.; Schiele, B. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014. [Google Scholar]
- Miller, P.; Styles, R.; Heath, T. Open data commons, a license for open data. Proc. LDOW 2008, 2008, 369. [Google Scholar]
- McAtamney, L.; Nigel Corlett, E. RULA: A survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 1993, 24, 91–99. [Google Scholar] [CrossRef] [PubMed]
- Hignett, S.; McAtamney, L. Rapid Entire Body Assessment (REBA). Appl. Ergon. 2000, 31, 201–205. [Google Scholar] [CrossRef]
- Shorten, C.; Khoshgoftaar, T.M. A survey on Image Data Augmentation for Deep Learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Jo, B.; Kim, S. Comparative Analysis of OpenPose, PoseNet, and MoveNet Models for Pose Estimation in Mobile Devices. Trait. Du Signal 2022, 39, 119–124. [Google Scholar] [CrossRef]
- Jin, X.; Xu, A.; Bie, R.; Guo, P. Machine Learning Techniques and Chi-Square Feature Selection for Cancer Classification Using SAGE Gene Expression Profiles. In Data Mining for Biomedical Applications; Springer: Berlin/Heidelberg, Germany, 2016; pp. 106–115. [Google Scholar] [CrossRef]
- Thaseen, I.S.; Kumar, C.A.; Ahmad, A. Integrated Intrusion Detection Model Using Chi-Square Feature Selection and Ensemble of Classifiers. Arab. J. Sci. Eng. 2019, 44, 3357–3368. [Google Scholar] [CrossRef]
- Zhai, Y.; Song, W.; Liu, X.; Liu, L.; Zhao, X. A Chi-Square Statistics Based Feature Selection Method in Text Classification. In Proceedings of the 2018 IEEE 9th International Conference on Software Engineering and Service Science, Beijing, China, 23–25 November 2018; pp. 160–163. [Google Scholar] [CrossRef]
- Rachburee, N.; Punlumjeak, W. A comparison of feature selection approach between greedy, IG-ratio, Chi-square, and mRMR in educational mining. In Proceedings of the 2015 7th International Conference on Information Technology and Electrical Engineering, Chiang Mai, Thailand, 29–30 October 2015; pp. 420–424. [Google Scholar] [CrossRef]
- Quinlan, R. Induction of decision trees. Mach. Learn. 1986, 1, 81–106. [Google Scholar] [CrossRef]
- Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
- Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Géron, A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems; O’Reilly Media: Sebastopol, CA, USA, 2019. [Google Scholar]
- Hand, D.J.; Gneiting, T.; Raftery, A.E. Comparing predictive accuracy. J. Bus. Econ. Stat. 2001, 19, 321–333. [Google Scholar]
- Hintze, J.L.; Nelson, R.D. Violin Plots: A Box Plot-Density Trace Synergism. Am. Stat. 1998, 52, 181–184. [Google Scholar] [CrossRef]
- Jocher, G. Ultralytics YOLOv5. Zenodo 2020. [Google Scholar] [CrossRef]
- Jiang, P.; Ergu, D.; Liu, F.; Cai, Y.; Ma, B. A Review of YOLO Algorithm Developments. Procedia Comput. Sci. 2022, 199, 1066–1073. [Google Scholar] [CrossRef]
Pre-Trained Models | Training Results | Test Results | |||
---|---|---|---|---|---|
Accuracy | Accuracy | Precision | Recall | F1-Score | |
MobileNet | 99.77% | 65.56% | 67.96% | 65.56% | 63.11% |
ResNet | 52.50% | 36.67% | 36.67% | 36.67% | 26.49% |
Inception | 94.32% | 62.22% | 63.18% | 62.22% | 62.18% |
VGG-16 | 99.77% | 60.00% | 62.96% | 60.00% | 58.98% |
Pre-Trained Models | Training Results | Test Results | |||
---|---|---|---|---|---|
Accuracy | Accuracy | Precision | Recall | F1-Score | |
SVM | 82.91% | 69.05% | 68.05% | 67.15% | 66.64% |
DT | 87.18% | 71.43% | 72.74% | 70.19% | 70.38% |
RF | 91.45% | 71.43% | 71.88% | 70.67% | 70.87% |
ANN | 94.02% | 80.49% | 80.61% | 78.93% | 79.92% |
Pre-Trained Models | Training Results | Test Results | |||
---|---|---|---|---|---|
Accuracy | Accuracy | Precision | Recall | F1-Score | |
SVM | 85.47% | 80.48% | 81.78% | 80.48% | 80.83% |
DT | 98.29% | 78.57% | 79.41% | 78.57% | 78.75% |
RF | 98.29% | 85.36% | 87.92% | 85.36% | 86.53% |
ANN | 94.44% | 87.80% | 87.46% | 87.52% | 87.41% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kapse, S.; Wu, R.; Thamsuwan, O. Addressing Ergonomic Challenges in Agriculture through AI-Enabled Posture Classification. Appl. Sci. 2024, 14, 525. https://doi.org/10.3390/app14020525
Kapse S, Wu R, Thamsuwan O. Addressing Ergonomic Challenges in Agriculture through AI-Enabled Posture Classification. Applied Sciences. 2024; 14(2):525. https://doi.org/10.3390/app14020525
Chicago/Turabian StyleKapse, Siddhant, Ruoxuan Wu, and Ornwipa Thamsuwan. 2024. "Addressing Ergonomic Challenges in Agriculture through AI-Enabled Posture Classification" Applied Sciences 14, no. 2: 525. https://doi.org/10.3390/app14020525