Using Learnable Physics for Real-Time Exercise Form Recommendations
Pages 688 - 695
Abstract
Good posture and form are essential for safe and productive exercising. Even in gym settings, trainers may not be readily available for feedback. Rehabilitation therapies and fitness workouts can thus benefit from recommender systems that provide real-time evaluation. In this paper, we present an algorithmic pipeline that can diagnose problems in exercises technique and offer corrective recommendations, with high sensitivity and specificity, in real-time. We use MediaPipe for pose recognition, count repetitions using peak-prominence detection, and use a learnable physics simulator to track motion evolution for each exercise. A test video is diagnosed based on deviations from the prototypical learned motion using statistical learning. The system is evaluated on six full and upper body exercises. These real-time recommendations, counseled via low-cost equipment like smartphones, will allow exercisers to rectify potential mistakes making self-practice feasible while reducing the risk of workout injuries.
Supplemental Material
MOV File
Demonstration video of person doing pushups and getting recommendations Stick figure representation of four full body exercises - Lunges, Pushups, Situps, Squats.
- Download
- 3.19 MB
MP4 File
Demonstration video of person doing pushups and getting recommendations Stick figure representation of four full body exercises - Lunges, Pushups, Situps, Squats.
- Download
- 443.77 KB
MP4 File
Demonstration video of person doing pushups and getting recommendations Stick figure representation of four full body exercises - Lunges, Pushups, Situps, Squats.
- Download
- 394.65 KB
MP4 File
Demonstration video of person doing pushups and getting recommendations Stick figure representation of four full body exercises - Lunges, Pushups, Situps, Squats.
- Download
- 439.13 KB
MP4 File
Demonstration video of person doing pushups and getting recommendations Stick figure representation of four full body exercises - Lunges, Pushups, Situps, Squats.
- Download
- 462.20 KB
References
[1]
Talal Alatiah and Chen Chen. 2020. Recognizing exercises and counting repetitions in real time. arXiv preprint arXiv:2005.03194 (2020).
[2]
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, 2016. Interaction networks for learning about objects, relations and physics. Advances in neural information processing systems 29 (2016).
[3]
Valentin Bazarevsky, Ivan Grishchenko, Karthik Raveendran, Tyler Zhu, Fan Zhang, and Matthias Grundmann. 2020. Blazepose: On-device real-time body pose tracking. arXiv preprint arXiv:2006.10204 (2020).
[4]
Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis & Machine Intelligence 43, 01 (2021), 172–186.
[5]
Matteo Cesari, Stephen B Kritchevsky, Brenda WHJ Penninx, Barbara J Nicklas, Eleanor M Simonsick, Anne B Newman, Frances A Tylavsky, Jennifer S Brach, Suzanne Satterfield, Douglas C Bauer, 2005. Prognostic value of usual gait speed in well-functioning older people—results from the Health, Aging and Body Composition Study. Journal of the American Geriatrics Society 53, 10 (2005), 1675–1680.
[6]
Keng-hao Chang, Mike Y Chen, and John Canny. 2007. Tracking free-weight exercises. In International Conference on Ubiquitous Computing. Springer, 19–37.
[7]
Steven Chen and Richard R Yang. 2020. Pose Trainer: correcting exercise posture using pose estimation. arXiv preprint arXiv:2006.11718 (2020).
[8]
William S Cleveland. 1981. LOWESS: A program for smoothing scatterplots by robust locally weighted regression. American Statistician 35, 1 (1981), 54.
[9]
Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. 2017. Rmpe: Regional multi-person pose estimation. In Proceedings of the IEEE international conference on computer vision. 2334–2343.
[10]
Gerald F Fletcher, Carolyn Landolfo, Josef Niebauer, Cemal Ozemek, Ross Arena, and Carl J Lavie. 2018. Promoting physical activity and exercise: JACC health promotion series. Journal of the American College of Cardiology 72, 14 (2018), 1622–1639.
[11]
Morteza Mohammady Gharasuie, Noah Jennings, and Shubham Jain. 2021. Performance Monitoring for Exercise Movements using Mobile Cameras. In Proceedings of the Workshop on Body-Centric Computing Systems. 1–6.
[12]
Oonagh M Giggins, Kevin T Sweeney, and Brian Caulfield. 2014. Rehabilitation exercise assessment using inertial sensors: a cross-sectional analytical study. Journal of neuroengineering and rehabilitation 11, 1 (2014), 1–10.
[13]
David Jiménez-Pavón, Ana Carbonell-Baeza, and Carl J Lavie. 2020. Physical exercise as therapy to fight against the mental and physical consequences of COVID-19 quarantine: Special focus in older people. Progress in cardiovascular diseases 63, 3 (2020), 386.
[14]
Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, and Mayank Goel. 2018. GymCam: Detecting, recognizing and tracking simultaneous exercises in unconstrained scenes. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1–17.
[15]
Yousef Kowsar, Masud Moshtaghi, Eduardo Velloso, Lars Kulik, and Christopher Leckie. 2016. Detecting unseen anomalies in weight training exercises. In Proceedings of the 28th Australian Conference on Computer-Human Interaction. 517–526.
[16]
Jaehyun Lee, Hyosung Joo, Junglyeon Lee, and Youngjoon Chee. 2020. Automatic classification of squat posture using inertial sensors: Deep learning approach. Sensors 20, 2 (2020), 361.
[17]
An-Lun Liu and Wei-Ta Chu. 2020. A posture evaluation system for fitness videos based on recurrent neural network. In 2020 International Symposium on Computer, Consumer and Control (IS3C). IEEE, 185–188.
[18]
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
[19]
Alex Moran, Bart Gebka, Joshua Goldshteyn, Autumn Beyer, Nathan Johnson, and Alexander Neuwirth. 2022. Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises. arXiv preprint arXiv:2203.12111 (2022).
[20]
Jiunn Ng. 2020. Posture evaluation for variants of weight-lifting workouts recognition. Ph. D. Dissertation. UTAR.
[21]
Cemal Ozemek, Carl J Lavie, and Øivind Rognmo. 2019. Global physical activity levels-Need for intervention. Progress in cardiovascular diseases 62, 2 (2019), 102–107.
[22]
Christian Seeger, Alejandro P Buchmann, and Kristof Van Laerhoven. 2011. myHealthAssistant: a phone-based body sensor network that captures the wearer’s exercises throughout the day. In BodyNets. 1–7.
[23]
Goran Šeketa, Dominik Džaja, Sara Žulj, Luka Celić, Igor Lacković, and Ratko Magjarević. 2015. Real-time evaluation of repetitive physical exercise using orientation estimation from inertial and magnetic sensors. In First European Biomedical Engineering Conference for Young Investigators. Springer, 11–15.
[24]
Leslie N Smith. 2018. A disciplined approach to neural network hyper-parameters: Part 1–learning rate, batch size, momentum, and weight decay. arXiv preprint arXiv:1803.09820 (2018).
[25]
Andrea Soro, Gino Brunner, Simon Tanner, and Roger Wattenhofer. 2019. Recognition and repetition counting for complex physical exercises with deep learning. Sensors 19, 3 (2019), 714.
[26]
Gabriele Spina, Guannan Huang, Anouk Vaes, Martijn Spruit, and Oliver Amft. 2013. COPDTrainer: a smartphone-based motion rehabilitation training system with real-time acoustic feedback. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 597–606.
[27]
Eduardo Velloso, Andreas Bulling, Hans Gellersen, Wallace Ugulino, and Hugo Fuks. 2013. Qualitative activity recognition of weight lifting exercises. In Proceedings of the 4th Augmented Human International Conference. 116–123.
[28]
Joe Verghese, Roee Holtzer, Richard B Lipton, and Cuiling Wang. 2009. Quantitative gait markers and incident fall risk in older adults. The Journals of Gerontology: Series A 64, 8 (2009), 896–901.
[29]
Parth Vyas. 2019. Pose estimation and action recognition in sports and fitness. (2019).
[30]
Jianbo Wang, Kai Qiu, Houwen Peng, Jianlong Fu, and Jianke Zhu. 2019. Ai coach: Deep human pose estimation and analysis for personalized athletic training assistance. In Proceedings of the 27th ACM International Conference on Multimedia. 374–382.
[31]
Lehao Wang, Yucheng Chen, and Weijun Duan. 2021. Monocular Keypoint based Pull-ups Measurement on Strict Pull-ups Benchmark. In 2021 4th International Conference on Computer Science and Software Engineering (CSSE 2021). 307–311.
[32]
Haocong Ying, Tie Liu, Mingxin Ai, Jiali Ding, and Yuanyuan Shang. 2021. AICoacher: A System Framework for Online Realtime Workout Coach. In Proceedings of the 29th ACM International Conference on Multimedia. 3787–3790.
[33]
Aras Yurtman and Billur Barshan. 2014. Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals. Computer methods and programs in biomedicine 117, 2 (2014), 189–207.
[34]
Wenbing Zhao, Roanna Lun, Deborah D Espy, and M Ann Reinthal. 2014. Realtime motion assessment for rehabilitation exercises: Integration of kinematic modeling with fuzzy inference. Journal of Artificial Intelligence and Soft Computing Research 4, 4 (2014), 267–285.
Index Terms
- Using Learnable Physics for Real-Time Exercise Form Recommendations
Comments
Information & Contributors
Information
Published In
September 2023
1406 pages
ISBN:9798400702419
DOI:10.1145/3604915
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
- SIGAI: ACM Special Interest Group on Artificial Intelligence
- SIGKDD: ACM Special Interest Group on Knowledge Discovery in Data
- SIGIR: ACM Special Interest Group on Information Retrieval
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
- SIGecom: Special Interest Group on Economics and Computation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 September 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
RecSys '23: Seventeenth ACM Conference on Recommender Systems
September 18 - 22, 2023
Singapore, Singapore
Acceptance Rates
Overall Acceptance Rate 254 of 1,295 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 321Total Downloads
- Downloads (Last 12 months)195
- Downloads (Last 6 weeks)11
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format