research-article

GymCam: Detecting, Recognizing and Tracking Simultaneous Exercises in Unconstrained Scenes

Authors:

Rushil Khurana,

Jennifer Mankoff,

Chris Harrison,

Mayank GoelAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 2, Issue 4

Article No.: 185, Pages 1 - 17

https://doi.org/10.1145/3287063

Published: 27 December 2018 Publication History

Abstract

Worn sensors are popular for automatically tracking exercises. However, a wearable is usually attached to one part of the body, tracks only that location, and thus is inadequate for capturing a wide range of exercises, especially when other limbs are involved. Cameras, on the other hand, can fully track a user's body, but suffer from noise and occlusion. We present GymCam, a camera-based system for automatically detecting, recognizing and tracking multiple people and exercises simultaneously in unconstrained environments without any user intervention. We collected data in a varsity gym, correctly segmenting exercises from other activities with an accuracy of 84.6%, recognizing the type of exercise at 93.6% accuracy, and counting the number of repetitions to within ± 1.7 on average. GymCam advances the field of real-time exercise tracking by filling some crucial gaps, such as tracking whole body motion, handling occlusion, and enabling single-point sensing for a multitude of users.

Supplementary Material

khurana (khurana.zip)

Supplemental movie, appendix, image and software files for, GymCam: Detecting, Recognizing and Tracking Simultaneous Exercises in Unconstrained Scenes

Download
60.01 MB

References

[1]

Fraser Anderson, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2013. YouMove: enhancing movement training with an augmented reality mirror. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 311--320.

Digital Library

[2]

D Antón, A Goñi, A Illarramendi, et al. 2015. Exercise recognition for Kinect-based telerehabilitation. Methods Inf Med 54, 2 (2015).

[3]

Ilktan Ar and Yusuf Sinan Akgul. 2014. A computerized recognition system for the home-based physiotherapy exercises using an RGBD camera. IEEE Transactions on Neural Systems and Rehabilitation Engineering 22, 6 (2014), 1160--1171.

[4]

Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 37.

Digital Library

[5]

Jean-Yves Bouguet. 2001. Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation 5, 1--10 (2001), 4.

[6]

Dena M Bravata, Crystal Smith-Spangler, Vandana Sundaram, Allison L Gienger, Nancy Lin, Robyn Lewis, Christopher D Stave, Ingram Olkin, and John R Sirard. 2007. Using pedometers to increase physical activity and improve health: a systematic review. Jama 298, 19 (2007), 2296--2304.

[7]

Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Real-time Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR.

[8]

Catherine B Chan, Daniel AJ Ryan, and Catrine Tudor-Locke. 2004. Health benefits of a pedometer-based physical activity intervention in sedentary workers. Preventive medicine 39, 6 (2004), 1215--1222.

[9]

Keng-Hao Chang, Mike Y Chen, and John Canny. 2007. Tracking free-weight exercises. In International Conference on Ubiquitous Computing. Springer, 19--37.

Digital Library

[10]

Rohit Chaudhri, Jonathan Lester, and Gaetano Borriello. 2008. An RFID based system for monitoring free weight exercises. In SenSys.

Digital Library

[11]

Yu Chen, Chunhua Shen, Xiu-Shen Wei, Lingqiao Liu, and Jian Yang. 2017. Adversarial posenet: A structure-aware convolutional network for human pose estimation. CoRR, abs/1705.00389 2 (2017).

[12]

Sheri R Colberg, Ronald J Sigal, Jane E Yardley, Michael C Riddell, David W Dunstan, Paddy C Dempsey, Edward S Horton, Kristin Castorino, and Deborah F Tate. 2016. Physical activity/exercise and diabetes: a position statement of the American Diabetes Association. Diabetes Care 39, 11 (2016), 2065--2079.

[13]

C Crema, A Depari, A Flammini, E Sisinni, T Haslwanter, and S Salzmann. 2017. IMU-based solution for automatic detection and classification of exercises in the fitness scenario. In Sensors Applications Symposium (SAS), 2017 IEEE. IEEE, 1--6.

[14]

Han Ding, Longfei Shangguan, Zheng Yang, Jinsong Han, Zimu Zhou, Panlong Yang, Wei Xi, and Jizhong Zhao. 2015. Femo: A platform for free-weight exercise monitoring with rfids. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. ACM, 141--154.

Digital Library

[15]

Gregory Heath, Elizabeth H Howze, Emily B Kahn, and Leigh Taylor Ramsey. 2001. Increasing physical activity. A report on recommendations of the Task Force on Community Preventive Services. MMWYR Recomm Rep 50 (2001), 1--14.

[16]

Ian Janssen and Allana G LeBlanc. 2010. Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. International journal of behavioral nutrition and physical activity 7, 1 (2010), 40.

[17]

Judy Kruger, Heidi Michels Blanck, and Cathleen Gillespie. 2006. Dietary and physical activity behaviors among adults successful at weight loss maintenance. International Journal of Behavioral Nutrition and Physical Activity 3, 1 (2006), 17.

[18]

Ofir Levy and Lior Wolf. 2015. Live repetition counting. In Proceedings of the IEEE International Conference on Computer Vision. 3020--3028.

Digital Library

[19]

Uwe Maurer, Asim Smailagic, Daniel P Siewiorek, and Michael Deisher. 2006. Activity recognition and monitoring using multiple sensors on different body positions. In Wearable and Implantable Body Sensor Networks, 2006. BSN 2006. International Workshop on. IEEE, 4--pp.

Digital Library

[20]

Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics 36, 4, 14.

Digital Library

[21]

Dan Morris, T Scott Saponas, Andrew Guillory, and Ilya Kelner. 2014. RecoFit: using a wearable sensor to find, recognize, and count repetitive exercises. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 3225--3234.

Digital Library

[22]

Michael Muehlbauer, Gernot Bahle, and Paul Lukowicz. 2011. What can an arm holster worn smart phone do for activity recognition?. In Wearable Computers (ISWC), 2011 15th Annual International Symposium on. IEEE, 79--82.

Digital Library

[23]

Rosemery O Nelson and Steven C Hayes. 1981. Theoretical explanations for reactivity in self-monitoring. Behavior Modification 5, 1 (1981), 3--14.

[24]

Konstantinos Papoutsakis, Costas Panagiotakis, and Antonis A Argyros. 2017. Temporal Action Co-segmentation in 3D Motion Capture Data and Videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6827--6836.

[25]

Kyle Rector, Cynthia L Bennett, and Julie A Kientz. 2013. Eyes-free yoga: an exergame using depth cameras for blind & low vision exercise. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 12.

Digital Library

[26]

Fernanda R Roque, Ana M Briones, Ana B García-Redondo, María Galán, Sonia Martínez-Revelles, Maria S Avendaño, Victoria Cachofeiro, Tiago Fernandes, Dalton V Vassallo, Edilamar M Oliveira, et al. 2013. Aerobic exercise reduces oxidative stress and improves vascular changes of small mesenteric and coronary arteries in hypertension. British journal of pharmacology 168, 3 (2013), 686--703.

[27]

Christian Seeger, Alejandro Buchmann, and Kristof Van Laerhoven. 2011. Adaptive gym exercise counting for myHealthAssistant. In Proceedings of the 6th International Conference on Body Area Networks. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 126--127.

Digital Library

[28]

Christian Seeger, Alejandro Buchmann, and Kristof Van Laerhoven. 2011. myHealthAssistant: a phone-based body sensor network that captures the wearer's exercises throughout the day. In Proceedings of the 6th International Conference on Body Area Networks. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 1--7.

Digital Library

[29]

Goran Šeketa, Dominik Džaja, Sara Žulj, Luka Celić, Igor Lacković, and Ratko Magjarević. 2015. Real-Time Evaluation of Repetitive Physical Exercise Using Orientation Estimation from Inertial and Magnetic Sensors. In First European Biomedical Engineering Conference for Young Investigators. Springer, 11--15.

[30]

Martyn Standage, Joan L Duda, and Nikos Ntoumanis. 2003. A model of contextual motivation in physical education: Using constructs from self-determination and achievement goal theories to predict physical activity intentions. Journal of educational psychology 95, 1 (2003), 97.

[31]

Eduardo Velloso, Andreas Bulling, and Hans Gellersen. 2013. MotionMA: motion modelling and analysis by demonstration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1309--1318.

Digital Library

[32]

Eduardo Velloso, Andreas Bulling, Hans Gellersen, Wallace Ugulino, and Hugo Fuks. 2013. Qualitative activity recognition of weight lifting exercises. In Proceedings of the 4th Augmented Human International Conference. ACM, 116--123.

Digital Library

[33]

Heng Wang, Alexander Kläser, Cordelia Schmid, and Cheng-Lin Liu. 2013. Dense trajectories and motion boundary descriptors for action recognition. International journal of computer vision 103, 1 (2013), 60--79.

[34]

Scott R Watterson, David Watterson, and Mark D Watterson. 2013. Systems and Methods to Generate a Customized Workout Routine. (Aug. 1 2013). US Patent App. 13/754, 361.

[35]

Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2016. Convolutional pose machines. In CVPR.

[36]

Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (2016), 1499--1503.

Cited By

Japhne FJanada KTheodorus AChowanda A(2024)Fitcam: detecting and counting repetitive exercises with deep learningJournal of Big Data10.1186/s40537-024-00915-811:1Online publication date: 3-Aug-2024
https://doi.org/10.1186/s40537-024-00915-8
Xia CMin TSugiura Y(2024)AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise GuidanceProceedings of the ACM on Human-Computer Interaction10.1145/36764898:MHCI(1-26)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676489
Liu XLi FCao YZhai SYang SWang Y(2024)PPGSpotter: Personalized Free Weight Training Monitoring Using Wearable PPG SensorIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621212(2468-2477)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621212
Show More Cited By

Index Terms

GymCam: Detecting, Recognizing and Tracking Simultaneous Exercises in Unconstrained Scenes
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

A unified rectification method for single viewpoint multi-camera system
AVSS '11: Proceedings of the 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance

Stereo matching and 3D reconstruction has been studied for decades as a fundamental problem in the field of computer vision. Recent years, stereo matching and 3D reconstruction with a large field of view, especially using omnidirectional vision and ...
A Unified Approach to the Linear Camera Calibration Problem

The camera calibration process relates camera system measurements (pixels) to known reference points in a three-dimensional world coordinate system. The calibration process is viewed as consisting of two independent phases: the first is removing ...
Kruppa equation based camera calibration from homography induced by remote plane

Camera calibration is an indispensable step in retrieving 3D metric information from 2D images. One classical self-calibration method is based on Kruppa equation derived from pairwise image correspondences. However, the calibration constraints derived ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 2, Issue 4

December 2018

1169 pages

EISSN:2474-9567

DOI:10.1145/3301777

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2018

Accepted: 01 October 2018

Revised: 01 August 2018

Received: 01 May 2018

Published in IMWUT Volume 2, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
1,417
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)9

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Japhne FJanada KTheodorus AChowanda A(2024)Fitcam: detecting and counting repetitive exercises with deep learningJournal of Big Data10.1186/s40537-024-00915-811:1Online publication date: 3-Aug-2024
https://doi.org/10.1186/s40537-024-00915-8
Xia CMin TSugiura Y(2024)AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise GuidanceProceedings of the ACM on Human-Computer Interaction10.1145/36764898:MHCI(1-26)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676489
Liu XLi FCao YZhai SYang SWang Y(2024)PPGSpotter: Personalized Free Weight Training Monitoring Using Wearable PPG SensorIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621212(2468-2477)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621212
Manivannan SPradhan YMuhammed ZPooja HBharathi R(2024)Automated Gym Exercise Form Checker: Deep Learning-Based Pose EstimationSmart Trends in Computing and Communications10.1007/978-981-97-1320-2_7(71-84)Online publication date: 14-Jun-2024
https://doi.org/10.1007/978-981-97-1320-2_7
Volschenk FVadapalli Hvan der Haar D(2024)Exercise Recognition and Repetition Counting for Automatic Workout Documentation Using Computer VisionDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management10.1007/978-3-031-61066-0_18(298-309)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-61066-0_18
Boonsongsrikul AEamsaard J(2023)Real-Time Human Motion Tracking by Tello EDU DroneSensors10.3390/s2302089723:2(897)Online publication date: 12-Jan-2023
https://doi.org/10.3390/s23020897
Kim JNam YLee JSuh YHwang I(2023)ProxiFitProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109207:3(1-32)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610920
Jaiswal AChauhan GSrivastava N(2023)Using Learnable Physics for Real-Time Exercise Form RecommendationsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608816(688-695)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608816
Eska BPhilip MWozniak PSchmidt AKarolus J(2023)[Don't] Let The Bodies HIIT The Floor: Fostering Body Awareness in Fast-Paced Physical Activity Using Body-Worn SensorsProceedings of the ACM on Human-Computer Interaction10.1145/36042507:MHCI(1-27)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604250
Lee EMin CLee JYu JKang SEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)GrooveMeter: Enabling Music Engagement-aware Apps by Detecting Reactions to Daily Music Listening via Earable SensingProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611968(7728-7736)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611968
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents