research-article

Graph-based analysis of physical exercise actions

Authors:

Oya Celiktutan,

Ceyhun Burak Akgul,

Christian Wolf,

Bülent SankurAuthors Info & Claims

MIIRH '13: Proceedings of the 1st ACM international workshop on Multimedia indexing and information retrieval for healthcare

Pages 23 - 32

https://doi.org/10.1145/2505323.2505330

Published: 22 October 2013 Publication History

Abstract

In this paper, we develop a graph-based method to align two dynamic skeleton sequences, and apply it to both action recognition tasks as well as to the objective quantification of the goodness of the action performance. The automated measurement of "action quality" has potential to be used to monitor action imitations, for example, during a physical therapy. We seek matches between a query sequence and model sequences selected with graph mining. The best matches are obtained through minimizing an energy function that jointly measures space and time domain deformations. This measure has been used for recognizing actions, for separating acceptable and unacceptable action performances, or as a continuous quantification of the action performance goodness. Experimental evaluation demonstrates the improved results of our scheme vis-à-vis its nearest competitors. Furthermore, a plausible relationship has been obtained between action perturbation, given by the joint noise variances, and quality measure, given by matching energies averaged over a sequence.

References

[1]

D. Alexiadis, P. Daras, P. Kelly, N. E. O'Connor, T. Boubekeur, and M. B. Moussa. Evaluating a dancer's performance using kinect-based skeleton tracking. In ACM Multimedia, Scottsdale, AZ, USA, 2011.

Digital Library

[2]

S. Bianco and F. Tisato. Karate moves recognition from skeletal motion. In 3D Image Processing (3DIP) and Applications, San Francisco, CA, USA, 2012.

[3]

O. Çeliktutan, C. Wolf, and B. Sankur. Fast exact matching and correspondence with hyper-graphs on spatio-temporal data. LIRIS UMR 5205 CNRS/INSA de Lyon/Université Claude Bernard Lyon 1/Université Lumiére Lyon 2/Ecole Centrale de Lyon, Report No. RR-LIRIS-2012-002, 2012.

[4]

O. Çeliktutan, C. Wolf, B. Sankur, and E. Lombardi. Real-time exact graph matching with application in human action recognition. In HBU, 2012.

Digital Library

[5]

S. Celebi, A. S. Aydın, T. T. Temiz, and T. Arici. Gesture recognition using skeleton data with weighted dynamic time warping. In VISAPP, Barcelona, Spain, 2013.

[6]

P. Chua, D. Ventura, R. Crivella, T. Camill, B. Daly, N. Hu, J. Hodgins, R. Schaaf, and R. Pausch. Training for physical tasks in virtual environments: tai chi. In Proc. of the IEEE Virtual Reality, pages 87--94, 2003.

Digital Library

[7]

D. Conte, P. Foggia, C. Sansone, and M. Vento. Thirty years of graph matching in pattern recognition. IJPRAI, 18(3):265--298, 2004.

[8]

C. Ellis, S. Masood, M. Tappen, J. L. Jr., and R. Sukthankar. Exploring the trade-off between accuracy and observational latency in action recognition. International Journal of Computer Vision, 101(3), 2013.

Digital Library

[9]

S. Essid, D. Alexiadis, R. Tournemenne, M. Gowing, P. Kelly, D. Monaghan, P. Daras, A. Drémeau, and N. E. O'Connor. An advanced virtual dance performance evaluator. In ICASSP, 2012.

[10]

W. Ilg, J. Mezger, and M. Giese. Estimation of skill levels in sports based on hierarchical spatio-temporal correspondences. In DAGM Symposium, 2003.

[11]

C. Keskin, E. Berger, and L. Akarun. A unified framework for concurrent usage of hand gesture, shape and pose. In CVPR 2012 Workshop on Gesture Recognition, 2012.

[12]

W. Q. Li, Z. Y. Zhang, and Z. C. Liu. Action recognition based on a bag of 3d points. In CVPR4HB10, pages 9--14, 2010.

[13]

S. Michelet, K. Karp, E. Delaherche, C. Achard, and M. Chetouani. Automatic imitation assessment in interaction. In HBU, 2012.

Digital Library

[14]

Microsoft. Introducing kinect for xbox 360. http://www.xbox.com/en-US/kinect. accessed at June, 2013.

[15]

L. Miranda, T. Vieira, D. Martinez, T. Lewiner, A. W. Vieira, and M. F. M. Campo. Real-time gesture recognition from depth data through key poses learning and decision forests. In Sibgrapi, pages 268--275, 2012.

Digital Library

[16]

F. Negin, F. Özdemir, C. B. Akgül, K. A. Yüksel, and A. Erçil. A decision forest based feature selection framework for action recognition from rgb-depth cameras. In ICIAR, 2013.

[17]

F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy. Sequence of the most informative joints (smij): A new representation for human skeletal action recognition. In HAU3D, 2012.

[18]

M. Raptis, D. Kirovski, and H. Hoppe. Real-time classification of dance gestures from skeleton animation. In ACM SIGGRAPH, 2011.

Digital Library

[19]

J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, and A. Blake. Real-time human pose recognition in parts from single depth images. In CVPR, Colorado Springs, USA, 2011.

Digital Library

[20]

V. Venkataraman, P. Turaga, N. Lehrer, M. Baran, T. Rikakis, and S. L. Wolf. Attractor-shape for dynamical analysis of human movement: Applications in stroke rehabilitation and action recognition. In Human Activity Understanding from 3D data HAU3D'13, 2012.

[21]

J. Wang, Z. Liu, Y. Wu, and J. Yuan. Mining actionlet ensemble for action recognition with depth cameras. In CVPR, 2012.

[22]

L. Wang, Y. Qiao, and X. Tang. Motionlets: Mid-level 3d parts for human motion recognition. In CVPR, 2013.

Digital Library

[23]

L. Xia, C.-C. Chen, and J. K. Aggarwal. View invariant human action recognition using histograms of 3d joints. In The 2nd International Workshop on Human Activity Understanding from 3D Data (HAU3D) in conjunction with IEEE CVPR, Providence, RI, 2012.

[24]

R. Zass and A. Shashua. Probabilistic graph and hypergraph matching. In CVPR, 2008.

Cited By

Wang PJiang RGuo ZLiu C(2024)Afitness: Fitness Monitoring on Smart Devices via Acoustic Motion ImagesACM Transactions on Sensor Networks10.1145/359261220:4(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3592612
He TChen YWang LCheng H(2024)An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises AssessmentIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340079032(1916-1925)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3400790
Guo XWang YCheng JChen YGuo XWang YCheng JChen Y(2024)Multi-person Fitness Assistance via Millimeter WaveMobile Technologies for Smart Healthcare System Design10.1007/978-3-031-57345-3_4(83-111)Online publication date: 3-Jul-2024
https://doi.org/10.1007/978-3-031-57345-3_4
Show More Cited By

Index Terms

Graph-based analysis of physical exercise actions
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
    2. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

On maximal actions and w-maximal actions of finite hypergroups

Sunder and Wildberger ( J. Algebr. Comb. 18 , 135---151, 2003) introduced the notion of actions of finite hypergroups, and studied maximal irreducible actions and *-actions. One of the main results of Sunder and Wildberger states that if a finite ...
Action Generator Based on Primitive Actions and Individual Properties
KES-AMSTA '09: Proceedings of the Third KES International Symposium on Agent and Multi-Agent Systems: Technologies and Applications

This paper presents a method of how to simulate various human actions. Actions are formed in the Action Hierarchy, and are reified in terms of expanding the hierarchy. While inheritance, super classes provide overall information of actions, and sub ...
Temporal Localization of Actions with Actoms

We address the problem of localizing actions, such as opening a door, in hours of challenging video data. We propose a model based on a sequence of atomic action units, termed "actoms," that are semantically meaningful and characteristic for the action. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MIIRH '13: Proceedings of the 1st ACM international workshop on Multimedia indexing and information retrieval for healthcare

October 2013

92 pages

ISBN:9781450323987

DOI:10.1145/2505323

Program Chairs:
Alexia Briasouli
CERTH -- ITI, Greece
,
Jenny Benois-Pineau
Univ. Bordeaux 1, France
,
Alexander Hauptmann
Carnegie-Mellon University, USA

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

hyper-graph matching

Qualifiers

Research-article

Conference

MM '13

Sponsor:

SIGMM

MM '13: ACM Multimedia Conference

October 22, 2013

Barcelona, Spain

Acceptance Rates

MIIRH '13 Paper Acceptance Rate 10 of 14 submissions, 71%;

Overall Acceptance Rate 10 of 14 submissions, 71%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
240
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang PJiang RGuo ZLiu C(2024)Afitness: Fitness Monitoring on Smart Devices via Acoustic Motion ImagesACM Transactions on Sensor Networks10.1145/359261220:4(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3592612
He TChen YWang LCheng H(2024)An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises AssessmentIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340079032(1916-1925)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3400790
Guo XWang YCheng JChen YGuo XWang YCheng JChen Y(2024)Multi-person Fitness Assistance via Millimeter WaveMobile Technologies for Smart Healthcare System Design10.1007/978-3-031-57345-3_4(83-111)Online publication date: 3-Jul-2024
https://doi.org/10.1007/978-3-031-57345-3_4
Guo XWang YCheng JChen YGuo XWang YCheng JChen Y(2024)Personalized Fitness Assistance Using Commodity WiFiMobile Technologies for Smart Healthcare System Design10.1007/978-3-031-57345-3_3(49-82)Online publication date: 3-Jul-2024
https://doi.org/10.1007/978-3-031-57345-3_3
Xie YGuo XWang YCheng JChen Y(2024)Universal Targeted Adversarial Attacks Against mmWave-Based Human Activity RecognitionNetwork Security Empowered by Artificial Intelligence10.1007/978-3-031-53510-9_7(177-211)Online publication date: 24-Feb-2024
https://doi.org/10.1007/978-3-031-53510-9_7
Garg BPostlmayr ACosman PDey SMao SWang SFang HGao WZhou G(2023)Short: Deep Learning Approach to Skeletal Performance Evaluation of Physical Therapy ExercisesProceedings of the 8th ACM/IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies10.1145/3580252.3586984(168-172)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3580252.3586984
Xie YLi FWu YWang Y(2023) HearFit + : Personalized Fitness Monitoring via Audio Signals on Smart Speakers IEEE Transactions on Mobile Computing10.1109/TMC.2021.312568422:5(2756-2770)Online publication date: 1-May-2023
https://doi.org/10.1109/TMC.2021.3125684
Xie YJiang RGuo XWang YCheng JChen Y(2023)Universal Targeted Adversarial Attacks Against mmWave-based Human Activity RecognitionIEEE INFOCOM 2023 - IEEE Conference on Computer Communications10.1109/INFOCOM53939.2023.10228887(1-10)Online publication date: 17-May-2023
https://doi.org/10.1109/INFOCOM53939.2023.10228887
Xie YJiang RGuo XWang YCheng JChen Y(2022)mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave2022 International Conference on Computer Communications and Networks (ICCCN)10.1109/ICCCN54977.2022.9868878(1-10)Online publication date: Jul-2022
https://doi.org/10.1109/ICCCN54977.2022.9868878
Wang SYang DZhai PYu QSuo TSun ZLi KZhang L(2021)A Survey of Video-based Action Quality Assessment2021 International Conference on Networking Systems of AI (INSAI)10.1109/INSAI54028.2021.00029(1-9)Online publication date: Nov-2021
https://doi.org/10.1109/INSAI54028.2021.00029
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents