research-article

Open access

MoCaPose: Motion Capturing with Textile-integrated Capacitive Sensors in Loose-fitting Smart Garments

Authors:

Daniel Geissler,

Marc Faulhaber,

Clara Elisabeth Gleiss,

Esther Friederike Zahn,

Lala Shakti Swarup Ray,

Vitor Fortes Rey,

Paul LukowiczAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 7, Issue 1

Article No.: 41, Pages 1 - 40

https://doi.org/10.1145/3580883

Published: 28 March 2023 Publication History

Abstract

We present MoCaPose, a novel wearable motion capturing (MoCap) approach to continuously track the wearer's upper body's dynamic poses through multi-channel capacitive sensing integrated in fashionable, loose-fitting jackets. Unlike conventional wearable IMU MoCap based on inverse dynamics, MoCaPose decouples the sensor position from the pose system. MoCaPose uses a deep regressor to continuously predict the 3D upper body joints coordinates from 16-channel textile capacitive sensors, unbound by specific applications. The concept is implemented through two prototyping iterations to first solve the technical challenges, then establish the textile integration through fashion-technology co-design towards a design-centric smart garment. A 38-hour dataset of synchronized video and capacitive data from 21 participants was recorded for validation. The motion tracking result was validated on multiple levels from statistics (R2 ~ 0.91) and motion tracking metrics (MP JPE ~ 86mm) to the usability in pose and motion recognition (0.9 F1 for 10-class classification with unsupervised class discovery). The design guidelines impose few technical constraints, allowing the wearable system to be design-centric and usecase-specific. Overall, MoCaPose demonstrates that textile-based capacitive sensing with its unique advantages, can be a promising alternative for wearable motion tracking and other relevant wearable motion recognition applications.

Supplemental Material

ZIP File - zhou-1

Supplemental movie, appendix, image and software files for, MoCaPose: Motion Capturing with Textile-integrated Capacitive Sensors in Loose-fitting Smart Garments

Download
69.45 MB

References

[1]

2022. EasyMoCap - Make human motion capture easier. Github. https://github.com/zju3dv/EasyMocap

[2]

2022. Shieldex Bremen PW. https://www.shieldex.de/products/shieldex-bremen-pw/

[3]

Fatemeh Abyarjoo, Armando Barreto, Jonathan Cofino, and Francisco R Ortega. 2015. Implementing a sensor fusion algorithm for 3D orientation detection with inertial/magnetic sensors. In Innovations and advances in computing, informatics, systems sciences, networking and engineering. Springer, 305--310.

[4]

Talha Agcayazi, Murat A Yokus, Max Gordon, Tushar Ghosh, and Alper Bozkurt. 2017. A stitched textile-based capacitive respiration sensor. In 2017 IEEE SENSORS. IEEE, 1--3.

[5]

Karan Ahuja, Paul Streli, and Christian Holz. 2021. TouchPose: Hand Pose Prediction, Depth Estimation, and Touch Classification from Capacitive Images. In The 34th Annual ACM Symposium on User Interface Software and Technology. 997--1009.

Digital Library

[6]

Mazen Al Borno, Johanna O'Day, Vanessa Ibarra, James Dunne, Ajay Seth, Ayman Habib, Carmichael Ong, Jennifer Hicks, Scott Uhlrich, and Scott Delp. 2022. OpenSense: An open-source toolbox for inertial-measurement-unit-based measurement of lower extremity kinematics over long durations. Journal of neuroengineering and rehabilitation 19, 1 (2022), 1--11.

[7]

I Al-Nasri, Aaron David Price, Ana Luisa Trejos, and David M Walton. 2019. A commercially available capacitive stretch-sensitive sensor for measurement of rotational neck movement in healthy people: Proof of concept. In 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). IEEE, 163--168.

Digital Library

[8]

Anindya Das Antar, Masud Ahmed, and Md Atiqur Rahman Ahad. 2019. Challenges in sensor-based human activity recognition and a comparative analysis of benchmark datasets: a review. In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR). IEEE, 134--139.

[9]

Abdul Hakeem Anwer, Nishat Khan, Mohd Zahid Ansari, Sang-Soo Baek, Hoon Yi, Soeun Kim, Seung Man Noh, and Changyoon Jeong. 2022. Recent Advances in Touch Sensors for Flexible Wearable Devices. Sensors 22, 12 (2022), 4460.

[10]

Ozgur Atalay. 2018. Textile-based, interdigital, capacitive, soft-strain sensor for wearable applications. Materials 11, 5 (2018), 768.

[11]

Chaiyawan Auepanwiriyakul, Sigourney Waibel, Joanna Songa, Paul Bentley, and A Aldo Faisal. 2020. Accuracy and acceptability of wearable motion tracking for inpatient monitoring using smartwatches. Sensors 20, 24 (2020), 7313.

[12]

Christopher A Bailey, Thomas K Uchida, Julie Nantel, and Ryan B Graham. 2021. Validity and sensitivity of an inertial measurement unit-driven biomechanical model of motor variability for gait. Sensors 21, 22 (2021), 7690.

[13]

Hymalai Bello, Bo Zhou, Sungho Suh, and Paul Lukowicz. 2021. Mocapaci: Posture and gesture detection in loose garments using textile cables as capacitive antennas. In 2021 International Symposium on Wearable Computers. 78--83.

Digital Library

[14]

Hymalai Bello, Bo Zhou, Sungho Suh, Luis Alfredo Sanchez Marin, and Paul Lukowicz. 2022. Move With the Theremin: Body Posture and Gesture Recognition Using the Theremin in Loose-Garment With Embedded Textile Cables as Antennas. Frontiers in Computer Science (2022), 72.

[15]

Sizhen Bian and Paul Lukowicz. 2021. Capacitive sensing based on-board hand gesture recognition with TinyML. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 4--5.

Digital Library

[16]

Sizhen Bian and Paul Lukowicz. 2021. A systematic study of the influence of various user specific and environmental factors on wearable human body capacitance sensing. In EAI International Conference on Body Area Networks. Springer, 247--274.

[17]

Sizhen Bian, Vitor F Rey, Peter Hevesi, and Paul Lukowicz. 2019. Passive capacitive based approach for full body gym workout recognition and counting. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. IEEE, 1--10.

[18]

Sizhen Bian, Vitor F Rey, Junaid Younas, and Paul Lukowicz. 2019. Wrist-worn capacitive sensor for activity and physical collaboration recognition. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 261--266.

[19]

Kyle J Boddy, Joseph A Marsh, Alex Caravan, Kyle E Lindley, John O Scheffey, and Michael E O'Connell. 2019. Exploring wearable sensors as an alternative to marker-based motion capture in the pitching delivery. PeerJ 7 (2019), e6365.

[20]

Sameh Neili Boualia and Najoua Essoukri Ben Amara. 2019. Pose-based human activity recognition: a review. In 2019 15th international wireless communications & mobile computing conference (IWCMC). IEEE, 1468--1475.

[21]

Arianna Carnevale, Umile Giuseppe Longo, Emiliano Schena, Carlo Massaroni, Daniela Lo Presti, Alessandra Berton, Vincenzo Candela, and Vincenzo Denaro. 2019. Wearable systems for shoulder kinematics assessment: A systematic review. BMC musculoskeletal disorders 20, 1 (2019), 1--24.

[22]

Shaoxiang Chen, Ting Yao, and Yu-Gang Jiang. 2019. Deep Learning for Video Captioning: A Review. In IJCAI, Vol. 1. 2.

[23]

Jingyuan Cheng, Bo Zhou, Kai Kunze, Carl Christian Rheinländer, Sebastian Wille, Norbert Wehn, Jens Weppner, and Paul Lukowicz. 2013. Activity recognition and nutrition monitoring in every day situations with a textile capacitive neckband. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. 155--158.

Digital Library

[24]

Kunigunde Cherenack and Liesbeth Van Pieterson. 2012. Smart textiles: Challenges and opportunities. Journal of Applied Physics 112, 9 (2012), 091301.

[25]

P Chinmilli, Sangram Redkar, Wenlong Zhang, and Tom Sugar. 2017. A review on wearable inertial tracking based human gait analysis and control strategies of lower-limb exoskeletons. Int. Robot. Autom. J 3, 7 (2017), 00080.

[26]

Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, and Mohit Bansal. 2022. Fine-grained image captioning with clip reward. arXiv preprint arXiv:2205.13115 (2022).

[27]

Minho Choi and Sang Woo Kim. 2017. Driver's movement monitoring system using capacitive ECG sensors. In 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE). IEEE, 1--2.

[28]

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua R Smith, Matthew S Reynolds, Desney S Tan, and Shwetak N Patel. 2012. An ultra-low-power human body motion sensor using static electric field sensing. In Proceedings of the 2012 ACM conference on ubiquitous computing. 99--102.

Digital Library

[29]

Gabe Cohn, Daniel Morris, Shwetak N Patel, and Desney S Tan. 2011. Your noise is my command: sensing gestures using the body as an antenna. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 791--800.

Digital Library

[30]

Kaya de Barbaro. 2019. Automated sensing of daily activity: A new lens into development. Developmental psychobiology 61, 3 (2019), 444--464.

[31]

Junting Dong, Qi Fang, Wen Jiang, Yurou Yang, Hujun Bao, and Xiaowei Zhou. 2021. Fast and Robust Multi-Person 3D Pose Estimation and Tracking from Multiple Views. In T-PAMI.

[32]

Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. 2020. Counting out time: Class agnostic video repetition counting in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10387--10396.

[33]

Qi Fang, Qing Shuai, Junting Dong, Hujun Bao, and Xiaowei Zhou. 2021. Reconstructing 3D Human Pose by Watching Humans in the Mirror. In CVPR.

[34]

Gabriele Frediani, Federica Vannetti, Leonardo Bocchi, Giovanni Zonfrillo, and Federico Carpi. 2021. Monitoring Flexions and Torsions of the Trunk via Gyroscope-Calibrated Capacitive Elastomeric Wearable Sensors. Sensors 21, 20 (2021), 6706.

[35]

A Garinei and R Marsili. 2014. Development of a new capacitive matrix for a steering wheel's pressure distribution measurement. International Journal of Industrial Ergonomics 44, 1 (2014), 114--119.

[36]

Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan, and Cees GM Snoek. 2020. Actor-transformers for group activity recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 839--848.

[37]

John Ghattas and Danielle N Jarvis. 2021. Validity of inertial measurement units for tracking human motion: a systematic review. Sports Biomechanics (2021), 1--14.

[38]

Hristijan Gjoreski and Daniel Roggen. 2017. Unsupervised online activity discovery using temporal behaviour assumption. In Proceedings of the 2017 ACM International Symposium on Wearable Computers. 42--49.

Digital Library

[39]

Tobias Grosse-Puppendahl, Eugen Berlin, and Marko Borazio. 2012. Enhancing accelerometer-based activity recognition with capacitive proximity sensing. In International Joint Conference on Ambient Intelligence. Springer, 17--32.

[40]

Tobias Grosse-Puppendahl, Christian Holz, Gabe Cohn, Raphael Wimmer, Oskar Bechtold, Steve Hodges, Matthew S Reynolds, and Joshua R Smith. 2017. Finding common ground: A survey of capacitive sensing in human-computer interaction. In Proceedings of the 2017 CHI conference on human factors in computing systems. 3293--3315.

Digital Library

[41]

Harish Haresamudram, David V Anderson, and Thomas Plötz. 2019. On the role of features in human activity recognition. In Proceedings of the 23rd International symposium on wearable computers. 78--88.

Digital Library

[42]

Edward F Harris and Richard N Smith. 2009. Accounting for measurement error: a critical but often overlooked process. Archives of oral biology 54 (2009), S107-S117.

[43]

Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll. 2018. Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1--15.

Digital Library

[44]

Jihye Hwang, John Yang, and Nojun Kwak. 2020. Exploring Rare Pose in Human Pose Estimation. IEEE Access 8 (2020), 194964--194977.

[45]

Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2013. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence 36, 7 (2013), 1325--1339.

[46]

Yifeng Jiang, Yuting Ye, Deepak Gopinath, Jungdam Won, Alexander W Winkler, and C Karen Liu. 2022. Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs. arXiv preprint arXiv:2203.15720 (2022).

[47]

Haojian Jin, Zhijian Yang, Swarun Kumar, and Jason I Hong. 2018. Towards wearable everyday body-frame tracking using passive RFIDs. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1--23.

Digital Library

[48]

Manuel Kaufmann, Yi Zhao, Chengcheng Tang, Lingling Tao, Christopher Twigg, Jie Song, Robert Wang, and Otmar Hilliges. 2021. Em-pose: 3d human pose estimation from sparse electromagnetic trackers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 11510--11520.

[49]

Karly Kudrinko, Emile Flavin, Xiaodan Zhu, and Qingguo Li. 2020. Wearable sensor-based sign language recognition: A comprehensive review. IEEE Reviews in Biomedical Engineering 14 (2020), 82--97.

[50]

Kai Kunze and Paul Lukowicz. 2008. Dealing with sensor displacement in motion-based onbody activity recognition systems. In Proceedings of the 10th international conference on Ubiquitous computing. 20--29.

Digital Library

[51]

Chang June Lee and Jung Keun Lee. 2022. Inertial Motion Capture-Based Wearable Systems for Estimation of Joint Kinetics: A Systematic Review. Sensors 22, 7 (2022), 2507.

[52]

Frédéric Li, Kimiaki Shirahama, Muhammad Adeel Nisar, Xinyu Huang, and Marcin Grzegorzek. 2020. Deep transfer learning for time series data based on sensor modality classification. Sensors 20, 15 (2020), 4271.

[53]

Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. 2021. Ai choreographer: Music conditioned 3d dance generation with aist++. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13401--13412.

[54]

Siming Li, Ruiqing Li, Tianjiao Chen, and Xueliang Xiao. 2020. Highly sensitive and flexible capacitive pressure sensor enhanced by weaving of pyramidal concavities staggered in honeycomb matrix. IEEE Sensors Journal 20, 23 (2020), 14436--14443.

[55]

Sheng Li, Zhiqiang Tao, Kang Li, and Yun Fu. 2019. Visual to text: Survey of image and video captioning. IEEE Transactions on Emerging Topics in Computational Intelligence 3, 4 (2019), 297--312.

[56]

Han Liang, Yannan He, Chengfeng Zhao, Mutian Li, Jingya Wang, Jingyi Yu, and Lan Xu. 2022. HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions. arXiv preprint arXiv:2203.09287 (2022).

[57]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740--755.

[58]

Mengmeng Liu, Xiong Pu, Chunyan Jiang, Ting Liu, Xin Huang, Libo Chen, Chunhua Du, Jiangman Sun, Weiguo Hu, and Zhong Lin Wang. 2017. Large-area all-textile pressure sensors for monitoring human motion and physiological signals. Advanced materials 29, 41 (2017), 1703700.

[59]

Shiqiang Liu, Junchang Zhang, Yuzhong Zhang, and Rong Zhu. 2020. A wearable motion capture device able to detect dynamic motion of human limbs. Nature communications 11, 1 (2020), 1--12.

[60]

Shi Qiang Liu, Jun Chang Zhang, Guo Zhen Li, and Rong Zhu. 2020. A wearable flow-MIMU device for monitoring human dynamic motion. IEEE Transactions on Neural Systems and Rehabilitation Engineering 28, 3 (2020), 637--645.

[61]

Michael Lorenz, Gabriele Bleser, Takayuki Akiyama, Takehiro Niikura, Didier Stricker, and Bertram Taetz. 2022. Towards Artefact Aware Human Motion Capture using Inertial Sensors Integrated into Loose Clothing. In 2022 International Conference on Robotics and Automation (ICRA). IEEE, 1682--1688.

Digital Library

[62]

Zhuyu Ma, Yang Zhang, Kaiyi Zhang, Hua Deng, and Qiang Fu. 2022. Recent progress in flexible capacitive sensors: Structures and properties. Nano Materials Science (2022).

[63]

Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. 2019. AMASS: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision. 5442--5451.

[64]

Denys JC Matthies, Chamod Weerasinghe, Bodo Urban, and Suranga Nanayakkara. 2021. Capglasses: Untethered capacitive sensing with smart glasses. In Augmented Humans Conference 2021. 121--130.

[65]

Denys JC Matthies, Alex Woodall, and Bodo Urban. 2021. Prototyping Smart Eyewear with Capacitive Sensing for Facial and Head Gesture Detection. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 476--480.

[66]

Jonathan Samir Matthis and Aaron Cherian. 2022. FreeMoCap: A free, open source markerless motion capture system. https://github.com/freemocap/freemocap/

[67]

Denisa Qori McDonald, Richard Vallett, Erin Solovey, Geneviève Dion, and Ali Shokoufandeh. 2020. Knitted Sensors: Designs and Novel Approaches for Real-Time, Real-World Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1--25.

Digital Library

[68]

Zhaozong Meng, Mingxing Zhang, Changxin Guo, Qirui Fan, Hao Zhang, Nan Gao, and Zonghua Zhang. 2020. Recent progress in sensing and computing techniques for human activity recognition and motion analysis. Electronics 9, 9 (2020), 1357.

[69]

Matteo Menolotto, Dimitrios-Sokratis Komaris, Salvatore Tedesco, Brendan O'Flynn, and Michael Walsh. 2020. Motion capture technology in industrial applications: A systematic review. Sensors 20, 19 (2020), 5687.

[70]

Jieming Pan, Yuxuan Luo, Yida Li, Chen-Khong Tham, Chun-Huat Heng, and Aaron Voon-Yew Thean. 2020. A wireless multi-channel capacitive sensor system for efficient glove-based gesture recognition with AI at the edge. IEEE Transactions on Circuits and Systems II: Express Briefs 67, 9 (2020), 1624--1628.

[71]

Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 2019. 3D human pose estimation in video with temporal convolutions and semi-supervised training. In Conference on Computer Vision and Pattern Recognition (CVPR).

[72]

Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. 2021. Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. In CVPR.

[73]

Lukasz Piwek, David A Ellis, Sally Andrews, and Adam Joinson. 2016. The rise of consumer health wearables: promises and barriers. PLoS medicine 13, 2 (2016), e1001953.

[74]

Chiara Plizzari, Marco Cannici, and Matteo Matteucci. 2021. Skeleton-based action recognition via spatial and temporal transformer networks. Computer Vision and Image Understanding 208 (2021), 103219.

[75]

Franchino Porciuncula, Anna Virginia Roto, Deepak Kumar, Irene Davis, Serge Roy, Conor J Walsh, and Louis N Awad. 2018. Wearable movement sensors for rehabilitation: a focused review of technological and clinical advances. Pm&r 10, 9 (2018), S220-S232.

[76]

Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, Mustafa Emre Karagozler, Carsten Schwesig, and Karen E Robinson. 2016. Project Jacquard: interactive digital textiles at scale. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 4216--4227.

Digital Library

[77]

E Ramanujam, Thinagaran Perumal, and S Padmavathi. 2021. Human activity recognition with smartphone and wearable sensors using deep learning techniques: A review. IEEE Sensors Journal 21, 12 (2021), 13029--13040.

[78]

Manju Rana and Vikas Mittal. 2020. Wearable sensors for real-time kinematics analysis in sports: a review. IEEE Sensors Journal 21, 2 (2020), 1187--1207.

[79]

Christopher Reining, Friedrich Niemann, Fernando Moya Rueda, Gernot A Fink, and Michael ten Hompel. 2019. Human activity recognition for production and logistics---a systematic literature review. Information 10, 8 (2019), 245.

[80]

Yili Ren, Zi Wang, Sheng Tan, Yingying Chen, and Jie Yang. 2021. Winect: 3d human pose tracking for free-form activity using commodity wifi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1--29.

Digital Library

[81]

Douglas A Reynolds. 2009. Gaussian mixture models. Encyclopedia of biometrics 741, 659--663 (2009).

[82]

Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1--30.

Digital Library

[83]

Nikolaos Sarafianos, Bogdan Boteanu, Bogdan Ionescu, and Ioannis A Kakadiaris. 2016. 3d human pose estimation: A review of the literature and analysis of covariates. Computer Vision and Image Understanding 152 (2016), 1--20.

Digital Library

[84]

Fatemeh Serpush, Mohammad Bagher Menhaj, Behrooz Masoumi, and Babak Karasfi. 2022. Wearable Sensor-Based Human Activity Recognition in the Smart Healthcare System. Computational Intelligence and Neuroscience 2022 (2022).

[85]

Young-Eun Shin, Jeong-Eun Lee, Yoojeong Park, Sang-Ha Hwang, Han Gi Chae, and Hyunhyub Ko. 2018. Sewing machine stitching of polyvinylidene fluoride fibers: programmable textile patterns for wearable triboelectric sensors. Journal of Materials Chemistry A 6, 45 (2018), 22879--22888.

[86]

Monit Shah Singh, Vinaychandran Pondenkandath, Bo Zhou, Paul Lukowicz, and Marcus Liwickit. 2017. Transforming sensor data to the image domain for deep learning---An application to footstep detection. In 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, 2665--2672.

[87]

Patrick Slade, Ayman Habib, Jennifer L Hicks, and Scott L Delp. 2021. An open-source and wearable system for measuring 3D human motion in real-time. IEEE Transactions on Biomedical Engineering 69, 2 (2021), 678--688.

[88]

Wolfgang Teufl, Markus Miezal, Bertram Taetz, Michael Fröhlich, and Gabriele Bleser. 2019. Validity of inertial sensor based 3D joint kinematics of static and dynamic sport and physiotherapy specific movements. PloS one 14, 2 (2019), e0213064.

[89]

Texas Instruments 2015. FDC2x1x EMI-Resistant 28-Bit,12-Bit Capacitance-to-Digital Converter for Proximity and Level Sensing Applications. Texas Instruments.

[90]

Matthew Trumble, Andrew Gilbert, Charles Malleson, Adrian Hilton, and John Collomosse. 2017. Total capture: 3d human pose estimation fusing video and inertial sensors. In Proceedings of 28th British Machine Vision Conference. 1--13.

[91]

Shuhei Tsuchida, Satoru Fukayama, Masahiro Hamasaki, and Masataka Goto. 2019. AIST Dance Video Database: Multi-Genre, Multi-Dancer, and Multi-Camera Database for Dance Information Processing. In ISMIR, Vol. 1. 6.

[92]

Hanyue Tu, Chunyu Wang, and Wenjun Zeng. 2020. Voxelpose: Towards multi-camera 3d human pose estimation in wild environment. In European Conference on Computer Vision. Springer, 197--212.

Digital Library

[93]

Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. 2020. Scan: Learning to classify images without labels. In European conference on computer vision. Springer, 268--285.

Digital Library

[94]

Jinbao Wang, Shujie Tan, Xiantong Zhen, Shuo Xu, Feng Zheng, Zhenyu He, and Ling Shao. 2021. Deep 3D human pose estimation: A review. Computer Vision and Image Understanding 210 (2021), 103225.

Digital Library

[95]

Lin Wang, Hristijan Gjoreski, Mathias Ciliberto, Paula Lago, Kazuya Murao, Tsuyoshi Okita, and Daniel Roggen. 2021. Three-year review of the 2018-2020 SHL challenge on transportation and locomotion mode recognition from mobile sensors. Frontiers in Computer Science (2021).

[96]

Mathias Wilhelm, Jan-Peter Lechler, Daniel Krakowczyk, and Sahin Albayrak. 2020. Ring-based finger tracking using capacitive sensors and long short-term memory. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 551--555.

Digital Library

[97]

WK Wong, Filbert H Juwono, and Brendan Teng Thiam Khoo. 2021. Multi-features capacitive hand gesture recognition sensor: A machine learning approach. IEEE Sensors Journal 21, 6 (2021), 8441--8450.

[98]

Emil Woop, Esther Friederike Zahn, Rahel Flechtner, and Gesche Joost. 2020. Demonstrating a Modular Construction Toolkit for Interactive Textile Applications. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society. 1--4.

Digital Library

[99]

Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. 2019. Detectron2. https://github.com/facebookresearch/detectron2.

[100]

Chenggang Yan, Yunbin Tu, Xingzheng Wang, Yongbing Zhang, Xinhong Hao, Yongdong Zhang, and Qionghai Dai. 2019. STAT: Spatial-temporal attention mechanism for video captioning. IEEE transactions on multimedia 22, 1 (2019), 229--241.

[101]

Ruiyang Yin, Depeng Wang, Shufang Zhao, Zheng Lou, and Guozhen Shen. 2021. Wearable sensors-enabled human-machine interaction systems: from design to application. Advanced Functional Materials 31, 11 (2021), 2008936.

[102]

Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016. Image captioning with semantic attention. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4651--4659.

[103]

Qi Zhang, Yu Lu Wang, Yun Xia, Timothy Vernon Kirk, and Xiao Dong Chen. 2020. Textile-Only capacitive sensors with a lockstitch structure for facile integration in any areas of a fabric. ACS sensors 5, 6 (2020), 1535--1540.

[104]

Qi Zhang, Yu Lu Wang, Yun Xia, Peng Fei Zhang, Timothy V Kirk, and Xiao Dong Chen. 2019. Textile-only capacitive sensors for facile fabric integration without compromise of wearability. Advanced Materials Technologies 4, 10 (2019), 1900485.

[105]

Boyu Zhao, Zhijia Dong, and Honglian Cong. 2022. A wearable and fully-textile capacitive sensor based on flat-knitted spacing fabric for human motions detection. Sensors and Actuators A: Physical 340 (2022), 113558.

[106]

Enhao Zheng, Jingeng Mai, Yuxiang Liu, and Qining Wang. 2018. Forearm motion recognition with noncontact capacitive sensing. Frontiers in Neurorobotics 12 (2018), 47.

[107]

Huiyu Zhou, Thomas Stone, Huosheng Hu, and Nigel Harris. 2008. Use of multiple wearable inertial sensors in upper limb motion tracking. Medical engineering & physics 30, 1 (2008), 123--133.

Cited By

Ray LZhou BSuh SLukowicz P(2024)A comprehensive evaluation of marker-based, markerless methods for loose garment scenarios in varying camera configurationsFrontiers in Computer Science10.3389/fcomp.2024.13799256Online publication date: 5-Apr-2024
https://doi.org/10.3389/fcomp.2024.1379925
Geißler DBello HZahn EWoop EZhou BLukowicz PKarolus J(2024) Head 'n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free Interaction Proceedings of the ACM on Human-Computer Interaction10.1145/36765108:PACMHCI(1-20)Online publication date: 24-Sep-2024
https://doi.org/10.1145/3676510
Geißler DZhou BJavaheri HZahn EWoop ESchaffer SLukowicz PKarolus JKostakos VKay JHoang T(2024)Towards Non-Distracting Smartphone Interaction While Biking Using Capacitive Sensing as Input DeviceCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677544(181-185)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677544
Show More Cited By

Index Terms

MoCaPose: Motion Capturing with Textile-integrated Capacitive Sensors in Loose-fitting Smart Garments
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing design and evaluation methods

Recommendations

Capafoldable: Self-tracking Foldable Smart Textiles With Capacitive Sensing
UbiComp/ISWC '23 Adjunct: Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing

Folding is a unique structural technique to equip planar materials with motion or 3D mechanical properties. Textile-based capacitive sensing has shown to be sensitive to the geometry deformation and relative motion of conductive textiles. In this work, ...
Inferring Complex Textile Shape from an Integrated Carbon Black-infused Ecoflex-based Bend and Stretch Sensor Array
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers

We demonstrate how an array of custom-made strain and bend sensors could be integrated into a stretchable sleeve to infer the textile deformation. The angles and elongation measured by the sensors can be used by an optimisation-based algorithm to infer ...
Fabric as a Sensor: Towards Unobtrusive Sensing of Human Behavior with Triboelectric Textiles
SenSys '18: Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems

Smart apparel with embedded sensors have the potential to revolutionize human behavior sensing by leveraging everyday clothing as the sensing substrate. However, existing textile-based sensing techniques rely on tight-fitting garments to obtain ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 7, Issue 1

March 2023

1243 pages

EISSN:2474-9567

DOI:10.1145/3589760

Issue’s Table of Contents

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 March 2023

Published in IMWUT Volume 7, Issue 1

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

BMBF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
1,249
Total Downloads

Downloads (Last 12 months)719
Downloads (Last 6 weeks)115

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ray LZhou BSuh SLukowicz P(2024)A comprehensive evaluation of marker-based, markerless methods for loose garment scenarios in varying camera configurationsFrontiers in Computer Science10.3389/fcomp.2024.13799256Online publication date: 5-Apr-2024
https://doi.org/10.3389/fcomp.2024.1379925
Geißler DBello HZahn EWoop EZhou BLukowicz PKarolus J(2024) Head 'n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free Interaction Proceedings of the ACM on Human-Computer Interaction10.1145/36765108:PACMHCI(1-20)Online publication date: 24-Sep-2024
https://doi.org/10.1145/3676510
Geißler DZhou BJavaheri HZahn EWoop ESchaffer SLukowicz PKarolus JKostakos VKay JHoang T(2024)Towards Non-Distracting Smartphone Interaction While Biking Using Capacitive Sensing as Input DeviceCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677544(181-185)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677544
Zhao MXie FWu ZLiang ZCai X(2024)3D Human Pose Estimation Using Pressure Images on a Smart ChairProceedings of the 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition10.1145/3663976.3663978(1-7)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3663976.3663978
Bian SLiu MZhou BLukowicz PMagno M(2024)Body-Area Capacitive or Electric Field Sensing for Human Activity Recognition and Human-Computer InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435558:1(1-49)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643555
Wang CZheng SZhong LYu CLiang CWang YGao YLam TShi Y(2024)PepperPose: Full-Body Pose Estimation with a Companion RobotProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642231(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642231
Zuo CWang YZhan LGuo SYi XXu FQin Y(2024)Loose Inertial Poser: Motion Capture with IMU-attached Loose-Wear Jacket2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00215(2209-2219)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00215
Bello HSuh SGeißler DRay LZhou BLukowicz P(2024)Real-Time and on-the-Edge Multiple Channel Capacitive and Inertial Fusion-Based GloveBody Area Networks. Smart IoT and Big Data for Intelligent Health Management10.1007/978-3-031-72524-1_13(166-176)Online publication date: 27-Dec-2024
https://doi.org/10.1007/978-3-031-72524-1_13
Zuo CFang JGuo SQin YOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Self-adaptive motion tracking against on-body displacement of flexible sensorsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669501(77277-77289)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669501
Geißler DZahn EBello HRay LWoop EZhou BLukowicz PJoost G(2023)MoCa'Collection: Normalizing Dynamic Textile Geometry with Capacitive Sensing in Design Centric WearablesAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610779(276-280)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3610779
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents