Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

FingerDraw: Sub-wavelength Level Finger Motion Tracking with WiFi Signals

Published: 18 March 2020 Publication History

Abstract

This paper explores the possibility of tracking finger drawings in the air leveraging WiFi signals from commodity devices. Prior solutions typically require user to hold a wireless transmitter, or need proprietary wireless hardware. They can only recognize a small set of pre-defined hand gestures. This paper introduces FingerDraw, the first sub-wavelength level finger motion tracking system using commodity WiFi devices, without attaching any sensor to finger. FingerDraw can reconstruct finger drawing trajectory such as digits, alphabets, and symbols with the setting of one WiFi transmitter and two WiFi receivers. It uses a two-antenna receiver to sense the sub-wavelength scale displacement of finger motion in each direction. The theoretical underpinning of FingerDraw is our proposed CSI-quotient model, which uses the channel quotient between two antennas of the receiver to cancel out the noise in CSI amplitude and the random offsets in CSI phase, and quantifies the correlation between CSI value dynamics and object displacement. This channel quotient is sensitive to and enables us to detect small changes in In-phase and Quadrature parts of channel state information due to finger movement. Our experimental results show that the overall median tracking accuracy is 1.27 cm, and the recognition of drawing ten digits in the air achieves an average accuracy of over 93.0%.

Supplementary Material

wu (wu.zip)
Supplemental movie, appendix, image and software files for, FingerDraw: Sub-wavelength Level Finger Motion Tracking with WiFi Signals

References

[1]
Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. WiGest: A ubiquitous wifi-based gesture recognition system. In 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, 1472--1480.
[2]
Fadel Adib, Zachary Kabelac, Dina Katabi, and R Miller. 2014. WiTrack: motion tracking via radio reflections off the body. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI).
[3]
Mohammed Al-qaness and Fangmin Li. 2016. WiGeR: WiFi-based gesture recognition system. ISPRS International Journal of GeoInformation 5, 6 (2016), 92.
[4]
Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 90--102.
[5]
Douglas N. Arnold and Jonathan Rogness. 2008. Möbius transformations revealed. Notices Amer. Math. Soc. 55 (2008), 1226--1231.
[6]
Alex Butler, Shahram Izadi, and Steve Hodges. 2008. SideSight: multi-touch interaction around small devices. In Proceedings of the 21st annual ACM symposium on User interface software and technology. ACM, 201--204.
[7]
Liwei Chan, Rong-Hao Liang, Ming-Chang Tsai, Kai-Yin Cheng, Chao-Huai Su, Mike Y Chen, Wen-Huang Cheng, and Bing-Yu Chen. 2013. FingerPad: private and subtle interaction using fingertips. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 255--260.
[8]
Amy D Droitcour, Olga Boric-Lubecke, Victor M Lubecke, Jenshan Lin, and Gregory TA Kovacs. 2004. Range correlation and I/Q performance benefits in single-chip silicon Doppler radars for noncontact cardiopulmonary monitoring. IEEE Transactions on Microwave Theory and Techniques 52, 3 (2004), 838--848.
[9]
Makiko Funasaka, Yu Ishikawa, Masami Takata, and Kazuki Joe. 2015. Sign language recognition using leap motion controller. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA). The Steering Committee of The World Congress in Computer Science, 263.
[10]
Wei Gong and Jiangchuan Liu. 2018. SiFi: Pushing the Limit of Time-Based WiFi Localization Using a Single Commodity Access Point. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 10.
[11]
Google. [n.d.]. Project Soli. https://www.youtube.com/watch?v=0QNiZfSsPc0. Accessed Jan 15, 2020.
[12]
Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool release: gathering 802.11 n traces with channel state information. ACM SIGCOMM Computer Communication Review 41, 1 (2011), 53--53.
[13]
Wolf Kienzle and Ken Hinckley. 2014. LightRing: always-available 2D input on any surface. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 157--160.
[14]
David Kim, Otmar Hilliges, Shahram Izadi, Alex D Butler, Jiawen Chen, Iason Oikonomidis, and Patrick Olivier. 2012. Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 167--176.
[15]
Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, 269--282.
[16]
Sven Kratz and Michael Rohs. 2009. HoverFlow: expanding the design space of around-device interaction. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 4.
[17]
Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. 2016. Dynamic-music: accurate device-free indoor localization. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 196--207.
[18]
Xiang Li, Daqing Zhang, Qin Lv, Jie Xiong, Shengjie Li, Yue Zhang, and Hong Mei. 2017. IndoTrack: Device-free indoor human tracking with commodity Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 72.
[19]
Xiang Li, Daqing Zhang, Jie Xiong, Yue Zhang, Shengjie Li, Yasha Wang, and Hong Mei. 2018. Training-Free Human Vitality Monitoring Using Commodity Wi-Fi Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 121.
[20]
Jian Liu, Yan Wang, Yingying Chen, Jie Yang, Xu Chen, and Jerry Cheng. 2015. Tracking vital signs during sleep leveraging off-the-shelf wifi. In Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 267--276.
[21]
Xuefeng Liu, Jiannong Cao, Shaojie Tang, and Jiaqi Wen. 2014. Wi-Sleep: Contactless sleep monitoring via WiFi signals. In Real-Time Systems Symposium (RTSS), 2014 IEEE. IEEE, 346--355.
[22]
Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, and Woosub Jung. 2018. SignFi: Sign Language Recognition Using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 23.
[23]
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1515--1525.
[24]
Sameera Palipana, David Rojas, Piyush Agrawal, and Dirk Pesch. 2018. FallDeFi: Ubiquitous Fall Detection using Commodity Wi-Fi Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 155.
[25]
Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 27--38.
[26]
Kun Qian, Chenshu Wu, Zheng Yang, Yunhao Liu, and Kyle Jamieson. 2017. Widar: Decimeter-level passive tracking via velocity monitoring with commodity Wi-Fi. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 6.
[27]
Kun Qian, Chenshu Wu, Zimu Zhou, Yue Zheng, Zheng Yang, and Yunhao Liu. 2017. Inferring motion direction using commodity wi-fi for interactive exergames. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1961--1972.
[28]
Muhammad Shahzad and Shaohu Zhang. 2018. Augmenting User Identification with WiFi Based Gesture Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 134.
[29]
Chao Sun, Tianzhu Zhang, and Changsheng Xu. 2015. Latent support vector machine modeling for sign language recognition with Kinect. ACM Transactions on Intelligent Systems and Technology (TIST) 6, 2 (2015), 20.
[30]
Ke Sun, Ting Zhao, Wei Wang, and Lei Xie. 2018. VSkin: Sensing Touch Gestures on Surfaces of Mobile Devices Using Acoustic Signals. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. ACM, 591--605.
[31]
Li Sun, Souvik Sen, Dimitrios Koutsonikolas, and Kyu-Han Kim. 2015. Widraw: Enabling hands-free drawing in the air on commodity wifi devices. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 77--89.
[32]
Sheng Tan and Jie Yang. 2016. WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 201--210.
[33]
Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-Level Localization with a Single WiFi Access Point. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Vol. 16. 165--178.
[34]
Raghav H Venkatnarayan, Griffin Page, and Muhammad Shahzad. 2018. Multi-User Gesture Recognition Using WiFi. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 401--413.
[35]
Aditya Virmani and Muhammad Shahzad. 2017. Position and orientation agnostic gesture recognition using wifi. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 252--264.
[36]
Chuyu Wang, Jian Liu, Yingying Chen, Hongbo Liu, Lei Xie, Wei Wang, Bingbing He, and Sanglu Lu. 2018. Multi-Touch in the Air: Device-Free Finger Tracking and Gesture Recognition via COTS RFID. In Proceedings of the Conference on Computer Communications (INFOCOM). IEEE, 1691--1699.
[37]
Hao Wang, Daqing Zhang, Junyi Ma, Yasha Wang, Yuxiang Wang, Dan Wu, Tao Gu, and Bing Xie. 2016. Human respiration detection with commodity wifi devices: do user location and body orientation matter?. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 25--36.
[38]
Hao Wang, Daqing Zhang, Yasha Wang, Junyi Ma, Yuxiang Wang, and Shengjie Li. 2017. RT-Fall: A real-time and contactless fall detection system with commodity WiFi devices. IEEE Transactions on Mobile Computing 16, 2 (2017), 511--526.
[39]
Jue Wang, Deepak Vasisht, and Dina Katabi. 2014. RF-IDraw: virtual touch screen in the air using RF signals. In ACM SIGCOMM Computer Communication Review, Vol. 44. ACM, 235--246.
[40]
Wei Wang, Alex X Liu, and Muhammad Shahzad. 2016. Gait recognition using wifi signals. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 363--373.
[41]
Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st annual international conference on mobile computing and networking. ACM, 65--76.
[42]
Wei Wang, Alex X Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 82--94.
[43]
Chenshu Wu, Zheng Yang, Zimu Zhou, Xuefeng Liu, Yunhao Liu, and Jiannong Cao. 2015. Non-invasive detection of moving and stationary human with wifi. IEEE Journal on Selected Areas in Communications 33, 11 (2015), 2329--2342.
[44]
Dan Wu, Daqing Zhang, Chenren Xu, Yasha Wang, and Hao Wang. 2016. WiDir: walking direction estimation using wireless signals. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, 351--362.
[45]
Yaxiong Xie, Zhenjiang Li, and Mo Li. 2015. Precise Power Delay Profiling with Commodity WiFi. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom '15). ACM, 53--64.
[46]
Tong Xin, Bin Guo, Zhu Wang, Pei Wang, Jacqueline Chi Kei Lam, Victor Li, and Zhiwen Yu. 2018. FreeSense: A Robust Approach for Indoor Human Detection Using Wi-Fi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 143.
[47]
Zheng Yang, Zimu Zhou, and Yunhao Liu. 2013. From RSSI to CSI: Indoor localization via channel response. ACM Computing Surveys (CSUR) 46, 2 (2013), 25.
[48]
Nan Yu, Wei Wang, Alex X Liu, and Lingtao Kong. 2018. QGesture: Quantifying Gesture Distance and Direction with WiFi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 51.
[49]
Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 15--28.
[50]
Youwei Zeng, Dan Wu, Ruiyang Gao, Tao Gu, and Daqing Zhang. 2018. FullBreathe: Full Human Respiration Detection Exploiting Complementarity of CSI Phase and Amplitude of WiFi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 148.
[51]
Daqing Zhang, Hao Wang, and Dan Wu. 2017. Toward centimeter-scale human activity sensing with Wi-Fi signals. Computer 50, 1 (2017), 48--57.
[52]
Fusang Zhang, Daqing Zhang, Jie Xiong, Hao Wang, Kai Niu, Beihong Jin, and Yuxiang Wang. 2018. From Fresnel Diffraction Model to Fine-grained Human Respiration Sensing with Commodity Wi-Fi Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 53.
[53]
Ouyang Zhang and Kannan Srinivasan. 2016. Mudra: User-friendly Fine-grained Gesture Recognition using WiFi Signals. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies. ACM, 83--96.
[54]
Chen Zhao, Ke-Yu Chen, Md Tanvir Islam Aumi, Shwetak Patel, and Matthew S Reynolds. 2014. SideSwipe: detecting in-air gestures around mobile devices using actual GSM signal. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 527--534.
[55]
Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 313--325.

Cited By

View all
  • (2024)SoundScroll: Robust Finger Slide Detection Using Friction Sound and Wrist-Worn MicrophonesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676614(63-70)Online publication date: 5-Oct-2024
  • (2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
  • (2024)WiProfile: Unlocking Diffraction Effects for Sub-Centimeter Target Profiling Using Commodity WiFi DevicesProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649355(185-199)Online publication date: 29-May-2024
  • Show More Cited By

Index Terms

  1. FingerDraw: Sub-wavelength Level Finger Motion Tracking with WiFi Signals

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 4, Issue 1
    March 2020
    1006 pages
    EISSN:2474-9567
    DOI:10.1145/3388993
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 March 2020
    Published in IMWUT Volume 4, Issue 1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Channel Quotient
    2. Channel state information (CSI)
    3. Finger-draw gesture
    4. WiFi

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • EU CHIST-ERA RadioSense Project
    • National Key Research and Development Plan
    • Peking University Information Technology Institute (Tianjin Binhai)
    • Australian Research Council (ARC) Discovery Project

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)266
    • Downloads (Last 6 weeks)24
    Reflects downloads up to 15 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)SoundScroll: Robust Finger Slide Detection Using Friction Sound and Wrist-Worn MicrophonesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676614(63-70)Online publication date: 5-Oct-2024
    • (2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
    • (2024)WiProfile: Unlocking Diffraction Effects for Sub-Centimeter Target Profiling Using Commodity WiFi DevicesProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649355(185-199)Online publication date: 29-May-2024
    • (2024)WiFi gesture recognition method based on data enhancementFourth International Conference on Machine Learning and Computer Application (ICMLCA 2023)10.1117/12.3029330(127)Online publication date: 22-May-2024
    • (2024)On the Design and Performance of QRD-Based Beamforming Feedback for Wi-Fi SensingIEEE Transactions on Wireless Communications10.1109/TWC.2023.332534523:5(5261-5271)Online publication date: May-2024
    • (2024)Understanding the Diffraction Model in Static Multipath-Rich Environments for WiFi Sensing System DesignIEEE Transactions on Mobile Computing10.1109/TMC.2024.337770823:11(10393-10410)Online publication date: Nov-2024
    • (2024)Wi-Fi-Based Human Activity Recognition for Continuous, Whole-Room Monitoring of Motor Functions in Parkinson’s DiseaseIEEE Open Journal of Antennas and Propagation10.1109/OJAP.2024.33931175:3(788-799)Online publication date: Jun-2024
    • (2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.232385840:8(1799-1831)Online publication date: 22-Mar-2024
    • (2024)WiCARPervasive and Mobile Computing10.1016/j.pmcj.2024.101963103:COnline publication date: 1-Oct-2024
    • (2024)DA-HARPervasive and Mobile Computing10.1016/j.pmcj.2023.10185096:COnline publication date: 4-Mar-2024
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media