research-article

Nrityantar: Pose oblivious Indian classical dance sequence classification system

Authors:

Prerana Mukherjee,

Brejesh LallAuthors Info & Claims

ICVGIP '18: Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing

Article No.: 65, Pages 1 - 7

https://doi.org/10.1145/3293353.3293419

Published: 03 May 2020 Publication History

Abstract

In this paper, we attempt to advance the research work done in human action recognition to a rather specialized application namely Indian Classical Dance (ICD) classification. The variation in such dance forms in terms of hand and body postures, facial expressions or emotions and head orientation makes pose estimation an extremely challenging task. To circumvent this problem, we construct a pose-oblivious shape signature which is fed to a sequence learning framework. The pose signature representation is done in two-fold process. First, we represent person-pose in first frame of a dance video using symmetric Spatial Transformer Networks (STN) to extract good person object proposals and CNN-based parallel single person pose estimator (SPPE). Next, the pose basis are converted to pose flows by assigning a similarity score between successive poses followed by non-maximal suppression. Instead of feeding a simple chain of joints in the sequence learner which generally hinders the network performance we constitute a feature vector of the normalized distance vectors, flow, angles between anchor joints which captures the adjacency configuration in the skeletal pattern. Thus, the kinematic relationship amongst the body joints across the frames using pose estimation helps in better establishing the spatio-temporal dependencies. We present an exhaustive empirical evaluation of state-of-the-art deep network based methods for dance classification on ICD dataset.

References

[1]

Caba Heilbron, F., Escorcia, V., Ghanem, B. and Carlos Niebles, J., 2015. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 961--970).

[2]

Krizhevsky, A., Sutskever, I. and Hinton, G.E., 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097--1105).

[3]

Ji, S., Xu, W., Yang, M. and Yu, K., 2013. 3D convolutional neural networks for human action recognition. IEEE transactions on pattern analysis and machine intelligence, 35(1), pp. 221--231.

[4]

Zolfaghari, M., Oliveira, G.L., Sedaghat, N. and Brox, T., 2017, October. Chained multi-stream networks exploiting pose, motion, and appearance for action classification and detection. In Computer Vision (ICCV), 2017 IEEE International Conference on (pp. 2923--2932). IEEE.

[5]

Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R. and Fei-Fei, L., 2014. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 1725--1732).

[6]

Chéron, G., Laptev, I. and Schmid, C., 2015. P-cnn: Pose-based cnn features for action recognition. In Proceedings of the IEEE international conference on computer vision (pp. 3218--3226).

[7]

Wang, L., Qiao, Y. and Tang, X., 2014. Action recognition and detection by combining motion and appearance features. THUMOS14 Action Recognition Challenge, 1(2), p.2.

[8]

LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P., 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), pp. 2278--2324.

[9]

Wang, H. and Schmid, C., 2013. Action recognition with improved trajectories. In Proceedings of the IEEE international conference on computer vision (pp. 3551--3558).

[10]

Laptev, I., 2005. On space-time interest points. International journal of computer vision, 64(2-3), pp. 107--123.

[11]

Bregonzio, M., Gong, S. and Xiang, T., 2009, June. Recognising action as clouds of space-time interest points. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 1948--1955). IEEE.

[12]

Dalal, N. and Triggs, B., 2005, June. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on (Vol. 1, pp. 886--893). IEEE.

[13]

Dalal, N., Triggs, B. and Schmid, C., 2006, May. Human detection using oriented histograms of flow and appearance. In European conference on computer vision (pp. 428--441). Springer, Berlin, Heidelberg.

[14]

Bobick, A.F. and Davis, J.W., 2001. The recognition of human movement using temporal templates. IEEE Transactions on pattern analysis and machine intelligence, 23(3), pp. 257--267.

Digital Library

[15]

Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X. and Van Gool, L., 2016, October. Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision (pp. 20--36). Springer, Cham.

[16]

Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L. and Xie, X., 2016, February. Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks. In AAAI (Vol. 2, No. 5, p. 6).

[17]

Simonyan, K. and Zisserman, A., 2014. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems (pp. 568--576).

[18]

Wang, L., Qiao, Y. and Tang, X., 2015. Action recognition with trajectory-pooled deep-convolutional descriptors. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4305--4314).

[19]

Bulat, A. and Tzimiropoulos, G., 2016, October. Human pose estimation via convolutional part heatmap regression. In European Conference on Computer Vision (pp. 717--732). Springer, Cham.

[20]

Newell, A., Yang, K. and Deng, J., 2016, October. Stacked hourglass networks for human pose estimation. In European Conference on Computer Vision (pp. 483--499). Springer, Cham.

[21]

Rogez, G., Weinzaepfel, P. and Schmid, C., 2017, June. Lcr-net: Localization-classification-regression for human pose. In CVPR 2017-IEEE Conference on Computer Vision & Pattern Recognition.

[22]

Mahendran, S., Ali, H. and Vidal, R., 2018. A mixed classification-regression framework for 3D pose estimation from 2D images. arXiv preprint arXiv:1805.03225.

[23]

Hochreiter, S., Bengio, Y., Frasconi, P. and Schmidhuber, J., 2001. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In A field guide to dynamical recurrent neural networks. IEEE Press.

[24]

Samanta, S., Purkait, P. and Chanda, B., 2012, January. Indian Classical Dance classification by learning dance pose bases. In Applications of Computer Vision (WACV), 2012 IEEE Workshop on (pp. 265--270). IEEE.

[25]

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K. and Fei-Fei, L., 2009, June. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 248--255). Ieee.

[26]

Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P. and Suleyman, M., 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950.

Cited By

Akalanka Siriwardana SMaduranga M(2024)Udarata Dance Pose Detection and Validation Using Computer Vision and Deep Neural Networks: A Comparative Study of Static and Dynamic Approaches2024 8th SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI)10.1109/SLAAI-ICAI63667.2024.10844944(1-6)Online publication date: 18-Dec-2024
https://doi.org/10.1109/SLAAI-ICAI63667.2024.10844944
Jayanthi JMaheswari P(2024)AI and augmented reality for 3D Indian dance pose reconstruction cultural revivalScientific Reports10.1038/s41598-024-58680-w14:1Online publication date: 4-Apr-2024
https://doi.org/10.1038/s41598-024-58680-w
Bera ANasipuri MKrejcar OBhattacharjee D(2023)Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark AnalysisIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.329356472(1-13)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3293564
Show More Cited By

Index Terms

Nrityantar: Pose oblivious Indian classical dance sequence classification system
1. Computing methodologies

Recommendations

Multi-Feature Based Emotion Recognition for Video Clips
ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

In this paper, we present our latest progress in Emotion Recognition techniques, which combines acoustic features and facial features in both non-temporal and temporal mode. This paper presents the details of our techniques used in the Audio-Video ...
Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM
Abstract
In view of the high dimensionality, nonrigidity, multiscale variation and the influence of illumination and angle on facial expressions, it is quite difficult to obtain facial expression images or videos using computers and analyze facial ...
Supervised and Neural Classifiers for Locomotion Analysis
UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

The goal of the Sussex-Huawei Locomotion-Transportation (SHL) recognition challenge is to classify 8 modes of transportation and locomotion activities recorded using a smartphone inertial sensor. In this paper, Team Orion extracts 36 quantitative ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICVGIP '18: Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing

December 2018

659 pages

ISBN:9781450366151

DOI:10.1145/3293353

Copyright © 2018 ACM.

© 2018 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 May 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICVGIP 2018

ICVGIP 2018: 11th Indian Conference on Computer Vision, Graphics and Image Processing

December 18 - 22, 2018

Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
56
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Akalanka Siriwardana SMaduranga M(2024)Udarata Dance Pose Detection and Validation Using Computer Vision and Deep Neural Networks: A Comparative Study of Static and Dynamic Approaches2024 8th SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI)10.1109/SLAAI-ICAI63667.2024.10844944(1-6)Online publication date: 18-Dec-2024
https://doi.org/10.1109/SLAAI-ICAI63667.2024.10844944
Jayanthi JMaheswari P(2024)AI and augmented reality for 3D Indian dance pose reconstruction cultural revivalScientific Reports10.1038/s41598-024-58680-w14:1Online publication date: 4-Apr-2024
https://doi.org/10.1038/s41598-024-58680-w
Bera ANasipuri MKrejcar OBhattacharjee D(2023)Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark AnalysisIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.329356472(1-13)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3293564
Bhuyan HDhaipule RDas P(2023)Sequence Recognition in Bharatnatyam DanceComputer Vision and Image Processing10.1007/978-3-031-31407-0_30(390-405)Online publication date: 7-May-2023
https://doi.org/10.1007/978-3-031-31407-0_30
Bhuyan HKilli JDash JDas PPaul S(2022)Motion Recognition in Bharatanatyam DanceIEEE Access10.1109/ACCESS.2022.318473510(67128-67139)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3184735
Rani CDevarakonda N(2022)An effectual classical dance pose estimation and classification system employing Convolution Neural Network –Long ShortTerm Memory (CNN-LSTM) network for video sequencesMicroprocessors & Microsystems10.1016/j.micpro.2022.10465195:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.micpro.2022.104651
Jain NBansal VVirmani DGupta VSalas-Morera LGarcia-Hernandez L(2021)An Enhanced Deep Convolutional Neural Network for Classifying Indian Classical Dance FormsApplied Sciences10.3390/app1114625311:14(6253)Online publication date: 6-Jul-2021
https://doi.org/10.3390/app11146253
Joshi MChakrabarty S(2021)An extensive review of computational dance automation techniques and applicationsProceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences10.1098/rspa.2021.0071477:2251Online publication date: 7-Jul-2021
https://doi.org/10.1098/rspa.2021.0071

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten