research-article

The Virtual Kinect

Authors:

Thiago Buarque de Gusmao Lafayette,

Alexandre de Queiroz Burle,

Arthur de Andrade Almeida,

Vinicius Lima Ventura,

Vitor Mendes Carvalho,

Alana Elza Fontes da Gama,

Joao Marcelo Xavier Natario Teixeira,

Veronica TeichriebAuthors Info & Claims

SVR '21: Proceedings of the 23rd Symposium on Virtual and Augmented Reality

Pages 111 - 119

https://doi.org/10.1145/3488162.3488215

Published: 03 January 2022 Publication History

Abstract

The COVID-19 pandemic impacted researches that depended on making tests on patients and teams that had to be divided to avoid crowding in laboratories. Sharing equipment is no longer a simple strategy; the acquisition of extra equipment is beyond the means of many researchers. Research with Kinect V2 applied to body tracking suffers from sanitary restrictions, product discontinuation and limited access to newer sensors (like the Azure Kinect). Kinect V2 is an RGB-D sensor with many applications in health, ergonomics, sports, games and other areas. That is why a lot of research is still under development with it. Because of the applicability of Kinect V2 on research and the current acquisition limitations, Virtual Kinect (VK) was created. VK is an open-source solution that enables the programmer to code using Kinect SDK functions and test it without a Kinect V2 sensor or a previous recording. It simulates the behavior of a Kinect V2 sensor through MediaPipe’s Pose estimation, providing RGB image and joint tracking information, correlating the joints of the two devices. This correlation is possible due to the proximity of the devices’ joint estimates. The VK was made to be simple and practical, so that its use only depends on the DLL exchange and the use of an ordinary RGB camera.

References

[1]

[n.d.]. Releases · google/mediapipe · GitHub. https://github.com/google/mediapipe/releases. (Accessed on 07/14/2021).

[2]

[n.d.]. Solutions - mediapipe. https://google.github.io/mediapipe/solutions/solutions.html. (Accessed on 07/14/2021).

[3]

Valentin Bazarevsky, Ivan Grishchenko, Karthik Raveendran, Tyler Zhu, Fan Zhang, and Matthias Grundmann. 2020. BlazePose: On-device Real-time Body Pose tracking. arXiv preprint arXiv:2006.10204(2020).

[4]

Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov, Karthik Raveendran, and Matthias Grundmann. 2019. BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs. CoRR abs/1907.05047(2019). arxiv:1907.05047http://arxiv.org/abs/1907.05047

[5]

Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2019. OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence 43, 1(2019), 172–186.

Digital Library

[6]

Anusorn Chaikaew, Kritsana Somkuan, and Thidalak Yuyen. 2021. Thai Sign Language Recognition: An Application of Deep Neural Network. 2021 Joint 6th International Conference on Digital Arts, Media and Technology with 4th ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunication Engineering, ECTI DAMT and NCON 2021 (2021), 128–131. https://doi.org/10.1109/ECTIDAMTNCON51128.2021.9425711

[7]

Chengjun Chen, Tiannuo Wang, Dongnian Li, and Jun Hong. 2020. Repetitive assembly action recognition based on object detection and pose estimation. Journal of Manufacturing Systems 55 (2020), 325–333.

[8]

Lin HUANG, Boshen ZHANG, Zhilin GUO, Yang XIAO, Zhiguo CAO, and Junsong YUAN. 2021. Survey on depth and RGB image-based 3D hand shape and pose estimation. Virtual Reality & Intelligent Hardware 3, 3 (2021), 207–234.

[9]

Elise Lachat, Hélène Macher, Tania Landes, and Pierre Grussenmeyer. 2015. Assessment and calibration of a RGB-D camera (Kinect v2 Sensor) towards a potential use for close-range 3D modeling. Remote Sensing 7, 10 (2015), 13070–13097. https://doi.org/10.3390/rs71013070

[10]

Thiago Buarque de Gusmão Lafayette, João Marcelo Xavier Natario Teixeira, and Alana Elza Fontes Da Gama. 2019. Hybrid Solution for Motion Capture with Kinect v2 to Different Biotypes Recognition. In XXVI Brazilian Congress on Biomedical Engineering, Rodrigo Costa-Felix, João Carlos Machado, and André Victor Alvarenga (Eds.). Springer Singapore, Singapore, 249–259.

[11]

Wei Lina and Jiangtao Ding. 2020. Behavior detection method of OpenPose combined with Yolo network. In 2020 International Conference on Communications, Information System and Computer Engineering (CISCE). IEEE, 326–330.

[12]

Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, 2019. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172(2019).

[13]

Roanna Lun and Wenbing Zhao. 2015. A Survey of Applications and Human Motion Recognition with Microsoft Kinect. Vol. 29. 1555008 pages. https://doi.org/10.1142/S0218001415550083

[14]

Jithin Mohan and S Ashok. 2018. A Reliable Robot Workspace Monitoring System Using Kinect v2. In 2018 International Conference on Recent Innovations in Electrical, Electronics & Communication Engineering (ICRIEECE). IEEE, 686–689.

[15]

Trinh Thi Doan Pham, Hai Thanh Nguyen, Sungmin Lee, and Chee Sun Won. 2016. Moving object detection with Kinect v2. In 2016 IEEE international conference on consumer electronics-Asia (ICCE-Asia). IEEE, 1–4.

[16]

Jitender Rai, R.C. Tripathi, and Nipun Gulati. 2020. A Comprehensive Survey of IT Sectors Affected by Covid-19. In 2020 9th International Conference System Modeling and Advancement in Research Trends (SMART). 52–54. https://doi.org/10.1109/SMART50582.2020.9337132

[17]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 779–788.

[18]

Mohammed Samir, Ehsan Golkar, and Ashrani Aizzuddin Abd Rahni. 2016. Comparison between the Kinect™ V1 and Kinect™ V2 for respiratory motion tracking. IEEE 2015 International Conference on Signal and Image Processing Applications, ICSIPA 2015 - Proceedings(2016), 150–155. https://doi.org/10.1109/ICSIPA.2015.7412180

[19]

Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby Sharp, Mark Finocchio, Richard Moore, Alex Kipman, and Andrew Blake. 2011. Real-time human pose recognition in parts from single depth images. In CVPR 2011. Ieee, 1297–1304.

Digital Library

[20]

S. K. Sriram and Nishant Sinha. 2020. Gestop: Customizable Gesture Control of Computer Systems. ACM International Conference Proceeding Series (2020), 405–409. https://doi.org/10.1145/3430984.3430993 arxiv:2010.13197

Digital Library

[21]

Bugra Tekin, Federica Bogo, and Marc Pollefeys. 2019. H+ o: Unified egocentric recognition of 3d hand-object poses and interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4511–4520.

[22]

Michal Tölgyessy, Martin Dekan, Ľuboš Chovanec, and Peter Hubinský. 2021. Evaluation of the azure kinect and its comparison to kinect v1 and kinect v2. Sensors (Switzerland) 21, 2 (2021), 1–25. https://doi.org/10.3390/s21020413

[23]

Aditya Viswakumar, Venkateswaran Rajagopalan, Tathagata Ray, and Chandu Parimi. 2019. Human gait analysis using OpenPose. In 2019 Fifth International Conference on Image Information Processing (ICIIP). IEEE, 310–314.

[24]

Oliver Wasenmüller and Didier Stricker. 2017. Comparison of kinect v1 and v2 depth images in terms of accuracy and precision. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10117 LNCS (2017), 34–45. https://doi.org/10.1007/978-3-319-54427-4_3

[25]

S. Zennaro, M. Munaro, S. Milani, P. Zanuttigh, A. Bernardi, S. Ghidoni, and E. Menegatti. 2015. Performance evaluation of the 1st and 2nd generation Kinect for multimedia applications. Proceedings - IEEE International Conference on Multimedia and Expo 2015-August(2015). https://doi.org/10.1109/ICME.2015.7177380

[26]

Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, and Matthias Grundmann. 2020. MediaPipe Hands: On-device Real-time Hand Tracking. (2020). arxiv:2006.10214http://arxiv.org/abs/2006.10214

Cited By

Liu TBao JZheng YLu YLiu TBao JZheng YLu Y(2025)Personnel Operation Behavior Detection and MonitoringIndustrial Intelligence: Methods and Applications10.1007/978-3-031-81477-8_7(189-224)Online publication date: 4-Feb-2025
https://doi.org/10.1007/978-3-031-81477-8_7
Hu ZZhang CWang XGe A(2024)Light-Adaptive Human Body Key Point Detection Algorithm Based on Multi-Source Information FusionSensors10.3390/s2410302124:10(3021)Online publication date: 10-May-2024
https://doi.org/10.3390/s24103021
Ashtiani OMaadugundu MKamal MPrabhakaran B(2024)Device-Agnostic Remote Range-of-motion Assessment using Data Abstraction2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR62202.2024.00041(221-226)Online publication date: 7-Aug-2024
https://doi.org/10.1109/MIPR62202.2024.00041
Show More Cited By

Index Terms

The Virtual Kinect
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Human-centered computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Real-time Hand Tracking Using Kinect
ICDSP '18: Proceedings of the 2nd International Conference on Digital Signal Processing

Real-time hand tracking is fundamental to human gesture recognition. However, due to the huge computation, previous studies are either off-line or limited to given poses. In order to satisfy the requirement of real-time hand tracking, in this paper we ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)

Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Kinect-based Taiwanese sign-language recognition system

Gesture-recognition is an important component for many intelligent human---computer interaction applications. For example, a realtime sign-language recognition system would detect and interpret hand gestures. Many vision-based sign-language recognition ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SVR '21: Proceedings of the 23rd Symposium on Virtual and Augmented Reality

October 2021

196 pages

ISBN:9781450395526

DOI:10.1145/3488162

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 January 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SVR'21

SVR'21: Symposium on Virtual and Augmented Reality

October 18 - 21, 2021

Virtual Event, Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
188
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)4

Reflects downloads up to 20 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu TBao JZheng YLu YLiu TBao JZheng YLu Y(2025)Personnel Operation Behavior Detection and MonitoringIndustrial Intelligence: Methods and Applications10.1007/978-3-031-81477-8_7(189-224)Online publication date: 4-Feb-2025
https://doi.org/10.1007/978-3-031-81477-8_7
Hu ZZhang CWang XGe A(2024)Light-Adaptive Human Body Key Point Detection Algorithm Based on Multi-Source Information FusionSensors10.3390/s2410302124:10(3021)Online publication date: 10-May-2024
https://doi.org/10.3390/s24103021
Ashtiani OMaadugundu MKamal MPrabhakaran B(2024)Device-Agnostic Remote Range-of-motion Assessment using Data Abstraction2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR62202.2024.00041(221-226)Online publication date: 7-Aug-2024
https://doi.org/10.1109/MIPR62202.2024.00041
Cóias ALee MBernardino ASmailagic A(2023)Skeleton Tracking Solutions for a Low-Cost Stroke Rehabilitation Support System2023 International Conference on Rehabilitation Robotics (ICORR)10.1109/ICORR58425.2023.10304749(1-6)Online publication date: 24-Sep-2023
https://doi.org/10.1109/ICORR58425.2023.10304749

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten