Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3460421.3480418acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
short-paper
Open access

MoCapaci: Posture and gesture detection in loose garments using textile cables as capacitive antennas

Published: 21 September 2021 Publication History

Abstract

We present a wearable system to detect body postures and gestures that does not require sensors to be firmly fixed to the body or integrated into a tight-fitting garment. The sensing system can be used in a loose piece of clothing such as a coat/blazer. It is based on the well-known theremin musical instrument, which we have unobtrusively integrated into a standard men’s blazer using conductive textile antennas and OpenTheremin hardware as a prototype, the ”MoCaBlazer.” Fourteen participants with diverse body sizes and balanced gender distribution mimicked 20 arm/torso movements with the unbuttoned, single-sized blazer. State-of-the-art deep learning approaches were used to achieve average recognition accuracy results of 97.18% for leave one recording out and 86.25% for user independent recognition.

Supplementary Material

VTT File (3460421.3480418.vtt)
MP4 File (3460421.3480418.mp4)
Supplemental video

References

[1]
Zamir Ahmed Abro, Yi-Fan Zhang, Cheng-Yu Hong, Rafique Ahmed Lakho, and Nan-Liang Chen. 2018. Development of a smart garment for monitoring body postures based on FBG and flex sensing technologies. Sensors and Actuators A: Physical 272 (2018), 153–160.
[2]
Sizhen Bian, Vitor F Rey, Peter Hevesi, and Paul Lukowicz. 2019. Passive capacitive based approach for full body gym workout recognition and counting. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. IEEE, Kyoto, Japan, 1–10.
[3]
Sizhen Bian, Vitor F Rey, Junaid Younas, and Paul Lukowicz. 2019. Wrist-Worn Capacitive Sensor for Activity and Physical Collaboration Recognition. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, Kyoto, Japan, 261–266.
[4]
Ali Boyali, Manolya Kavakli, 2012. A robust and fast gesture recognition method for wearable sensing garments. In Proc. Int. Conf. Adv. Multimedia. International Academy, Research, and Industry Association (IARIA), Chamonix / Mont Blanc, France, 142–147.
[5]
Andreas Braun, Sebastian Frank, Martin Majewski, and Xiaofeng Wang. 2015. CapSeat: capacitive proximity sensing for automotive activity recognition. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Association for Computing Machinery, Nottingham, United Kingdom, 225–232.
[6]
Andreas Braun, Reiner Wichert, Arjan Kuijper, and Dieter W Fellner. 2015. Capacitive proximity sensing in smart environments. Journal of Ambient Intelligence and Smart Environments 7, 4(2015), 483–510.
[7]
Hammad Tanveer Butt, Manthan Pancholi, Mathias Musahl, Pramod Murthy, Maria Alejandra Sanchez, and Didier Stricker. 2019. Inertial Motion Capture Using Adaptive Sensor Fusion and Joint Angle Drift Correction. In 2019 22th International Conference on Information Fusion (FUSION). IEEE, Ottawa, ON, Canada, 1–8.
[8]
Youngsu Cha, Hojoon Kim, and Doik Kim. 2018. Flexible piezoelectric sensor-based gait recognition. Sensors 18, 2 (2018), 468.
[9]
Youngsu Cha, Kihyuk Nam, and Doik Kim. 2017. Patient posture monitoring system based on flexible sensors. Sensors 17, 3 (2017), 584.
[10]
Harish Chander, Reuben F Burch, Purva Talegaonkar, David Saucier, Tony Luczak, John E Ball, Alana Turner, Sachini NK Kodithuwakku Arachchige, Will Carroll, Brian K Smith, 2020. Wearable stretch sensors for human movement monitoring and fall detection in ergonomics. International journal of environmental research and public health 17, 10(2020), 3554.
[11]
Jingyuan Cheng, Oliver Amft, Gernot Bahle, and Paul Lukowicz. 2013. Designing sensitive wearable capacitive sensors for activity recognition. IEEE sensors journal 13, 10 (2013), 3935–3947.
[12]
Jingyuan Cheng, Oliver Amft, and Paul Lukowicz. 2010. Active capacitive sensing: Exploring a new wearable sensing modality for activity recognition. In International conference on pervasive computing. Springer, Berlin, Heidelberg, 319–336.
[13]
Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua R Smith, Matthew S Reynolds, Desney S Tan, and Shwetak N Patel. 2012. An ultra-low-power human body motion sensor using static electric field sensing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. Association for Computing Machinery, Pittsburgh, Pennsylvania, 99–102.
[14]
Gabe Cohn, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Humantenna: using the body as an antenna for real-time whole-body interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Austin, Texas, USA, 1901–1910.
[15]
Daniyar Enikeev and Svetlana Mustafina. 2020. Recognition of Sign Language Using Leap Motion Controller Data. In 2020 2nd International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA). IEEE, Lipetsk, Russia, 393–397.
[16]
Urs Gaudenz. 2016. Open.Theremin V3, GaudiLabs. GaudiLabs LLC. Retrieved June 10, 2021 from http://www.gaudi.ch/OpenTheremin/index.php/opentheremin-v3
[17]
Marian Haescher, Denys JC Matthies, Gerald Bieber, and Bodo Urban. 2015. Capwalk: A capacitive recognition of walking-based activities as a wearable assistive technology. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments. Association for Computing Machinery, Corfu, Greece, 1–8.
[18]
Holger Harms, Oliver Amft, Gerhard Tröster, and Daniel Roggen. 2008. Smash: A distributed sensing and processing garment for the classification of upper body postures. In Proceedings of the ICST 3rd international conference on Body area networks. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Tempe, Arizona, 1–8.
[19]
Paul Holleis, Albrecht Schmidt, Susanna Paasovaara, Arto Puikkonen, and Jonna Häkkilä. 2008. Evaluating capacitive touch input on clothes. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. Association for Computing Machinery, Amsterdam, The Netherlands, 81–90.
[20]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. PMLR, Lille, France, 448–456.
[21]
Pyeong-Gook Jung, Gukchan Lim, Seonghyok Kim, and Kyoungchul Kong. 2015. A wearable gesture recognition device for detecting muscular activities based on air-pressure sensors. IEEE Transactions on Industrial Informatics 11, 2 (2015), 485–494.
[22]
Julio Cezar Silveira Jacques Junior, Yağmur Güçlütürk, Marc Pérez, Umut Güçlü, Carlos Andujar, Xavier Baró, Hugo Jair Escalante, Isabelle Guyon, Marcel AJ Van Gerven, Rob Van Lier, 2019. First impressions: A survey on vision-based apparent personality trait analysis. IEEE Transactions on Affective Computing 1, 1 (2019), 1–1.
[23]
Muhammad US Khan, Assad Abbas, Mazhar Ali, Muhammad Jawad, and Samee U Khan. 2018. Convolutional neural networks as means to identify apposite sensor combination for human activity recognition. In 2018 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). IEEE, 45–50.
[24]
Muhammad US Khan, Assad Abbas, Mazhar Ali, Muhammad Jawad, and Samee U Khan. 2018. Convolutional neural networks as means to identify apposite sensor combination for human activity recognition. In 2018 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). IEEE, 45–50.
[25]
Ali Kiaghadi, Morgan Baima, Jeremy Gummeson, Trisha Andrew, and Deepak Ganesan. 2018. Fabric as a sensor: Towards unobtrusive sensing of human behavior with triboelectric textiles. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. Association for Computing Machinery, Shenzhen, China, 199–210.
[26]
Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arxiv:1412.6980 [cs.LG]
[27]
Yuya Koyama, Michiko Nishiyama, and Kazuhiro Watanabe. 2016. Gait monitoring for human activity recognition using perceptive shoe based on hetero-core fiber optics. In 2016 IEEE 5th Global Conference on Consumer Electronics. IEEE, Kyoto, Japan, 1–2.
[28]
Yuya Koyama, Michiko Nishiyama, and Kazuhiro Watanabe. 2018. Physical activity recognition using hetero-core optical fiber sensors embedded in a smart clothing. In 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE). IEEE, Nara, Japan, 71–72.
[29]
Arnaldo G Leal-Junior, Diana Ribeiro, Leticia M Avellar, Mariana Silveira, Camilo A Rodriguez Díaz, Anselmo Frizera-Neto, Wilfried Blanc, Eduardo Rocon, and Carlos Marques. 2020. Wearable and Fully-Portable Smart Garment for Mechanical Perturbation Detection With Nanoparticles Optical Fibers. IEEE Sensors Journal 21, 3 (2020), 2995–3003.
[30]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.
[31]
Junchan Li, Yu Wang, Pengfei Wang, Qing Bai, Yan Gao, Hongjuan Zhang, and Baoquan Jin. 2021. Pattern Recognition for Distributed Optical Fiber Vibration Sensing: A Review. IEEE Sensors Journal 21, 10 (2021), 11983–11998.
[32]
Qi Lin, Shuhua Peng, Yuezhong Wu, Jun Liu, Wen Hu, Mahbub Hassan, Aruna Seneviratne, and Chun H Wang. 2020. E-Jacket: Posture Detection with Loose-Fitting Garment using a Novel Strain Sensor. In 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, Sydney, NSW, Australia, 49–60.
[33]
Haoyan Liu, Enrique Sanchez, James Parkerson, and Alexander Nelson. 2019. Gesture Classification with Low-Cost Capacitive Sensor Array for Upper Extremity Rehabilitation. In 2019 IEEE SENSORS. IEEE, Montreal, QC, Canada, 1–4.
[34]
Gabriel Loke, Tural Khudiyev, Brian Wang, Stephanie Fu, Syamantak Payra, Yorai Shaoul, Johnny Fung, Ioannis Chatziveroglou, Pin-Wen Chou, Itamar Chinn, 2021. Digital electronics in fibres enable fabric-based machine-learning inference. Nature communications 12, 1 (2021), 1–9.
[35]
Mohammad Iman Mokhlespour Esfahani and Maury A Nussbaum. 2019. Classifying diverse physical activities using “Smart Garments”. Sensors 19, 14 (2019), 3133.
[36]
Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.
[37]
Bryce Osoinach. 2007. Proximity capacitive sensor technology for touch sensing applications. Freescale White Paper 12(2007).
[38]
Arash Pouryazdan, Robert J Prance, Helen Prance, and Daniel Roggen. 2016. Wearable electric potential sensing: a new modality sensing hair touch and restless leg movement. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing: adjunct. Association for Computing Machinery, Heidelberg, Germany, 846–850.
[39]
Rajarajan Ramalingame, Rim Barioul, Xupeng Li, Giuseppe Sanseverino, Dominik Krumm, Stephan Odenwald, and Olfa Kanoun. 2021. Wearable Smart Band for American Sign Language Recognition with Polymer Carbon Nanocomposite based Pressure Sensors. IEEE Sensors Letters 5, 6 (2021), 6001204.
[40]
Sanat Sarangi, Somya Sharma, and Bhushan Jagyasi. 2015. Agricultural activity recognition with smart-shirt and crop protocol. In 2015 IEEE Global Humanitarian Technology Conference (GHTC). IEEE, Seattle, WA, USA, 298–305.
[41]
Sungtae Shin, Han Ul Yoon, and Byungseok Yoo. 2021. Hand Gesture Recognition Using EGaIn-Silicone Soft Sensors. Sensors 21, 9 (2021), 3204.
[42]
Chamani Shiranthika, Nilantha Premakumara, Huei-Ling Chiu, Hooman Samani, Chathurangi Shyalika, and Chan-Yun Yang. 2020. Human Activity Recognition Using CNN & LSTM. In 2020 5th International Conference on Information Technology Research (ICITR). IEEE, 1–6.
[43]
Gurashish Singh, Alexander Nelson, Ryan Robucci, Chintan Patel, and Nilanjan Banerjee. 2015. Inviz: Low-power personalized gesture recognition using wearable textile capacitive sensor arrays. In 2015 IEEE international conference on pervasive computing and communications (PerCom). IEEE, St. Louis, MO, USA, 198–206.
[44]
Sophie Skach, Rebecca Stewart, and Patrick GT Healey. 2018. Smart arse: posture classification with textile sensors in trousers. In Proceedings of the 20th ACM International Conference on Multimodal Interaction. Association for Computing Machinery, Boulder, CO, USA, 116–124.
[45]
Kenneth D Skeldon, Lindsay M Reid, Viviene McInally, Brendan Dougan, and Craig Fulton. 1998. Physics of the Theremin. American Journal of Physics 66, 11 (1998), 945–955.
[46]
M Sornam, Kavitha Muthusubash, and V Vanitha. 2017. A survey on image classification and activity recognition using deep convolutional neural network architecture. In 2017 Ninth International Conference on Advanced Computing (ICoAC). IEEE, 121–126.
[47]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
[48]
Paul Stoffregen. 2014. FreqCount. Teensy. Retrieved June 10, 2021 from https://www.pjrc.com/teensy/td_libs_FreqCount.html
[49]
Paul Stoffregen. 2020. Teensy. Teensy. Retrieved June 10, 2021 from https://www.pjrc.com/store/teensy41.html
[50]
Saiganesh Swaminathan, Jonathon Fagert, Michael Rivera, Andrew Cao, Gierad Laput, Hae Young Noh, and Scott E Hudson. 2020. OptiStructures: Fabrication of Room-Scale Interactive Structures with Embedded Fiber Bragg Grating Optical Sensors and Displays. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 2 (2020), 1–21.
[51]
Nintendo Switch. 2020. Just Dance. Nintendo. Retrieved August 05, 2021 from https://www.youtube.com/watch?v=__Fx-cnrNoM
[52]
Liang Wang, Tao Gu, Xianping Tao, and Jian Lu. 2016. Toward a wearable RFID system for real-time activity recognition using radio patterns. IEEE Transactions on Mobile Computing 16, 1 (2016), 228–242.
[53]
Interactive Wear. 2021. TWC24004B. Interactive Wear. Retrieved June 10, 2021 from http://www.interactive-wear.com/solutions/10-textile-wires/45-textile-cables
[54]
Nintendo Wii. 2009. Nintendo Wii’s Rayman Raving Rabbids: TV Party - ShakeTV. Nintendo. Retrieved August 05, 2021 from https://www.youtube.com/watch?v=eoxjA6E1mDs
[55]
Mathias Wilhelm, Daniel Krakowczyk, Frank Trollmann, and Sahin Albayrak. 2015. eRing: multiple finger gesture recognition with one ring using an electric field. In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction. Association for Computing Machinery, Rostock, Germany, 1–6.
[56]
Raphael Wimmer, Matthias Kranz, Sebastian Boring, and Albrecht Schmidt. 2007. Captable and capshelf-unobtrusive activity recognition using networked capacitive sensors. In 2007 Fourth International Conference on Networked Sensing Systems. IEEE, Braunschweig, Germany, 85–88.
[57]
Yong Ye, Chiya Zhang, Chunlong He, Xi Wang, Jianjun Huang, and Jiahao Deng. 2020. A review on applications of capacitive displacement sensing for capacitive proximity sensor. IEEE Access 8(2020), 45325–45342.
[58]
Qihang Zeng, Wei Xu, Changyuan Yu, Na Zhang, and Cheungchuen Yu. 2018. Fiber-optic Activity Monitoring with Machine Learning. In Conference on Lasers and Electro-Optics/Pacific Rim. Optical Society of America, Hong Kong, China, W4K–5.
[59]
Yang Zhang, Chouchang Yang, Scott E Hudson, Chris Harrison, and Alanson Sample. 2018. Wall++ room-scale interactive and context-aware sensing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Montreal QC, Canada, 1–15.
[60]
Zhen Zhang, Kuo Yang, Jinwu Qian, and Lunwei Zhang. 2019. Real-time surface EMG pattern recognition for hand gestures based on an artificial neural network. Sensors 19, 14 (2019), 3170.
[61]
Enhao Zheng and Qining Wang. 2016. Noncontact capacitive sensing-based locomotion transition recognition for amputees with robotic transtibial prostheses. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25, 2(2016), 161–170.
[62]
Bo Zhou, Mathias Sundholm, Jingyuan Cheng, Heber Cruz, and Paul Lukowicz. 2017. Measuring muscle activities during gym exercises with textile pressure mapping sensors. Pervasive and Mobile Computing 38 (2017), 331–345.

Cited By

View all
  • (2024)A comprehensive evaluation of marker-based, markerless methods for loose garment scenarios in varying camera configurationsFrontiers in Computer Science10.3389/fcomp.2024.13799256Online publication date: 5-Apr-2024
  • (2024)Head 'n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free InteractionProceedings of the ACM on Human-Computer Interaction10.1145/36765108:MHCI(1-20)Online publication date: 24-Sep-2024
  • (2024)Functional Now, Wearable Later: Examining the Design Practices of Wearable TechnologistsProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676615(71-81)Online publication date: 5-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISWC '21: Proceedings of the 2021 ACM International Symposium on Wearable Computers
September 2021
220 pages
ISBN:9781450384629
DOI:10.1145/3460421
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Loose garment sensing
  2. activity recognition.
  3. capacitive sensing
  4. theremin

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Funding Sources

Conference

UbiComp '21

Acceptance Rates

Overall Acceptance Rate 38 of 196 submissions, 19%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)698
  • Downloads (Last 6 weeks)108
Reflects downloads up to 23 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A comprehensive evaluation of marker-based, markerless methods for loose garment scenarios in varying camera configurationsFrontiers in Computer Science10.3389/fcomp.2024.13799256Online publication date: 5-Apr-2024
  • (2024)Head 'n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free InteractionProceedings of the ACM on Human-Computer Interaction10.1145/36765108:MHCI(1-20)Online publication date: 24-Sep-2024
  • (2024)Functional Now, Wearable Later: Examining the Design Practices of Wearable TechnologistsProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676615(71-81)Online publication date: 5-Oct-2024
  • (2024)SeamPose: Repurposing Seams as Capacitive Sensors in a Shirt for Upper-Body Pose TrackingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676341(1-13)Online publication date: 13-Oct-2024
  • (2024)SeamSleeve: Robust Arm Movement Sensing through Powered StitchingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660726(1134-1147)Online publication date: 1-Jul-2024
  • (2024)Joey: Supporting Kangaroo Mother Care with Computational FabricsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661867(237-251)Online publication date: 3-Jun-2024
  • (2024)Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops59983.2024.10502797(364-365)Online publication date: 11-Mar-2024
  • (2024)Learn to Infer Human Poses Using a Full-Body Pressure-Sensing GarmentIEEE Sensors Journal10.1109/JSEN.2024.348522624:24(41357-41364)Online publication date: 15-Dec-2024
  • (2024)Embedding textile capacitive sensing into smart wearables as a versatile solution for human motion capturingScientific Reports10.1038/s41598-024-66165-z14:1Online publication date: 9-Jul-2024
  • (2024)TSAK: Two-Stage Semantic-Aware Knowledge Distillation for Efficient Wearable Modality and Model Optimization in Manufacturing LinesPattern Recognition10.1007/978-3-031-78389-0_14(201-216)Online publication date: 5-Dec-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media