research-article

Push the Limit of Device-Free Acoustic Sensing on Commercial Mobile Devices

Authors:

Wei LouAuthors Info & Claims

IEEE INFOCOM 2021 - IEEE Conference on Computer Communications

Pages 1 - 10

https://doi.org/10.1109/INFOCOM42981.2021.9488703

Published: 10 May 2021 Publication History

Abstract

Device-free acoustic sensing has obsessed with renovating human-computer interaction techniques for all-sized mobile devices in various applications. Recent advances have explored sound signals in different methods to achieve highly accurate and efficient tracking and recognition. However, accuracies of most approaches remain bottlenecked by the limited sampling rate and narrow bandwidth, leading to restrictions and inconvenience in applications. To bridge over the aforementioned daunting barriers, we propose PDF, a novel ultrasound-based device-free tracking scheme that can distinctly improve the resolution of fine-grained sensing to submillimetre level. In its heart lies an original Phase Difference based approach to derive time delay of the reflected Frequency-Modulated Continuous Wave (FMCW), thus precisely inferring absolute distance, catering to interaction needs of tinier perception with lower delay. The distance resolution of PDF is only related to the speed of actions and chirp duration. We implement a prototype with effective denoising methods all in the time domain on smartphones. The evaluation results show that PDF achieves accuracies of 2.5 mm, 3.6 mm, and 2.1 mm in distance change, absolute distance change, and trajectory tracking error respectively. PDF is also valid in recognizing 2 mm or even tinier micro-movements, which paves the way for more delicate sensing work.

References

[1]

J. Lien, N. Gillian, M. E. Karagozler, P. Amihood, C. Schwesig, E. Olson, H. Raja, and I. Poupyrev, “Soli: Ubiquitous gesture sensing with millimeter wave radar,” Trans. on Graphics, vol. 35, no. 4, 2016.

[2]

T. Wei and X. Zhang, “mtrack: High-precision passive tracking using millimeter wave radios,” in Proc. of ACM Mobicom, 2015.

[3]

F. Adib, Z. Kabelac, D. Katabi, and R. C. Miller, “3d tracking via body radio reflections,” in Proc. of USENIX NSDI, 2014.

[4]

K. Joshi, D. Bharadia, M. Kotaru, and S. Katti, “Wideo: Fine-grained device-free motion tracing using rf backscatter,” in Proc. of USENIX NSDI, 2015.

[5]

L. Sun, S. Sen, D. Koutsonikolas, and K.-H. Kim, “Widraw: Enabling hands-free drawing in the air on commodity wifi devices,” in Proc. of ACM MobiCom, 2015.

[6]

J. Wang, D. Vasisht, and D. Katabi, “Rf-idraw: Virtual touch screen in the air using rf signals,” in Proc. of ACM SIGCOMM, 2014.

[7]

L. Motion, “Leap motion,” Leap Motion, 2018. [Online]. Available: https://www.leapmotion.com/

[8]

Microsoft, “Microsoft kinect,” Microsoft, 2019. [Online]. Available: https://developer.microsoft.com/en-us/windows/kinect

[9]

C. Zhang, J. Tabor, J. Zhang, and X. Zhang, “Extending mobile interaction through near-field visible light sensing,” in Proc. of ACM MobiCom, 2015.

[10]

S. Tan and J. Yang, “Wifinger: Leveraging commodity wifi for fine-grained finger gesture recognition,” in Proc. of ACM MobiHoc, 2016.

[11]

K. Ali, A. X. Liu, W. Wang, and M. Shahzad, “Keystroke recognition using wifi signals,” in Proc. of ACM MobiCom, 2015.

[12]

W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, “Understanding and modeling of wifi signal based human activity recognition,” in Proc. of ACM MobiCom, 2015.

[13]

W. Wang, A. X. Liu, and K. Sun, “Device-free gesture tracking using acoustic signals,” in Proc. of ACM MobiCom, 2016.

[14]

S. Yun, Y.-C. Chen, H. Zheng, L. Qiu, and W. Mao, “Strata: Fine-grained acoustic-based device-free tracking,” in Proc. of ACM MobiSys, 2017.

[15]

X. Wang, R. Huang, and S. Mao, “Sonarbeat: Sonar phase for breathing beat monitoring with smartphones,” in Proc. of IEEE ICCCN, 2017.

[16]

B. Zhou, M. Elbadry, R. Gao, and F. Ye, “Battracker: High precision infrastructure-free mobile device tracking in indoor environments,” in Proc. of ACM SenSys, 2017.

[17]

R. Nandakumar, V. Iyer, D. Tan, and S. Gollakota, “Fingerio: Using active sonar for fine-grained finger tracking,” in Proc. of ACM CHI, 2016.

[18]

T. Wang, D. Zhang, Y. Zheng, T. Gu, X. Zhou, and B. Dorizzi, “C-fmcw based contactless respiration detection using acoustic signal,” Proc. of ACM IMWUT, vol. 1, 2017.

[19]

H. Chen, F. Li, and Y. Wang, “Echotrack: Acoustic device-free hand tracking on smart phones,” in Proc. of IEEE Infocom, 2017.

[20]

B. Zhou, M. Elbadry, R. Gao, and F. Ye, “Batmapper: Acoustic sensing based indoor floor plan construction using smartphones,” in Proc. of ACM MobiSys, 2017.

[21]

S. Pradhan, G. Baig, W. Mao, L. Qiu, G. Chen, and B. Yang, “Smartphone-based acoustic indoor space mapping,” Proc. of ACM IMWUT, vol. 2, 2018.

[22]

K. Qian, C. Wu, F. Xiao, Y. Zheng, Y. Zhang, Z. Yang, and Y. Liu, “Acousticcardiogram: Monitoring heartbeats using acoustic signals on smart devices,” in Proc. of IEEE Infocom, 2018.

[23]

Y. Wang, J. Shen, and Y. Zheng, “Push the limit of acoustic gesture recognition,” in Proc. of IEEE Infocom, 2020.

[24]

L. Lu, J. Yu, Y. Chen, and Y. Wang, “Vocallock: Sensing vocal tract for passphrase-independent user authentication leveraging acoustic signals on smartphones,” Proc. of ACM IMWUT, vol. 4, 2020.

[25]

K. Ling, H. Dai, Y. Liu, and A. X. Liu, “Ultragesture: Fine-grained gesture sensing and recognition,” in Proc. of IEEE SECON, 2018.

[26]

Z. Xiao, T. Chen, Y. Liu, and Z. Li, “Mobile phones know your keystrokes through the sounds from finger’s tapping on the screen,” in Proc. of IEEE ICDCS, 2020.

[27]

H. Du, P. Li, H. Zhou, W. Gong, G. Luo, and P. Yang, “Wordrecorder: Accurate acoustic-based handwriting recognition using deep learning,” in Proc. of IEEE Infocom, 2018.

[28]

Z. Zhang, D. Chu, X. Chen, and T. Moscibroda, “Swordfight: Enabling a new class of phone-to-phone action games on commodity phones,” in Proc. of ACM MobiSys, 2012.

[29]

S. Yun, Y.-C. Chen, and L. Qiu, “Turning a mobile device into a mouse in the air,” in Proc. of ACM MobiSys, 2015.

[30]

W. Mao, J. He, and L. Qiu, “Cat: High-precision acoustic motion tracking,” in Proc. of ACM MobiCom, 2016.

[31]

Y. Liu, W. Zhang, Y. Yang, W. Fang, F. Qin, and X. Dai, “Pamt: Phase-based acoustic motion tracking in multipath fading environments,” in Proc. of IEEE Infocom, 2019.

[32]

A. Wang and S. Gollakota, “Millisonic: Pushing the limits of acoustic motion tracking,” in Proc. of ACM CHI, 2019.

[33]

H. Zhang, W. Du, P. Zhou, M. Li, and P. Mohapatra, “Dopenc: Acoustic-based encounter profiling using smartphones,” in Proc. of ACM MobiCom, 2016.

[34]

M. Zhou, Q. Wang, J. Yang, Q. Li, F. Xiao, Z. Wang, and X. Chen, “Patternlistener: Cracking android pattern lock using acoustic signals,” in Proc. of ACM CCS, 2018.

[35]

A. S. Rathore, W. Zhu, A. Daiyan, C. Xu, K. Wang, F. Lin, K. Ren, and W. Xu, “Sonicprint: A generally adoptable and secure fingerprint biometrics in smart devices,” in Proc. of ACM MobiSys, 2020.

[36]

H. Dai, W. Wang, A. X. Liu, K. Ling, and J. Sun, “Speech based human authentication on smartphones,” in Proc. of IEEE SECON, 2019.

[37]

J. Tan, C. Nguyen, and X. Wang, “Silenttalk: Lip reading through ultrasonic sensing on mobile phones,” in Proc. of IEEE Infocom, 2017.

[38]

L. Lu, J. Yu, Y. Chen, H. Liu, Y. Zhu, L. Kong, and M. Li, “Lip readingbased user authentication through acoustic sensing on smartphones,” IEEE/ACM Trans. on Networking, vol. 27, no. 1, Feb 2019.

[39]

J. Tan, X. Wang, C.-T. Nguyen, and Y. Shi, “Silentkey: A new authentication framework through ultrasonic-based lip reading,” Proc. of ACM IMWUT, vol. 2, 2018.

[40]

B. Zhou, J. Lohokare, R. Gao, and F. Ye, “Echoprint: Two-factor authentication using acoustics and vision on smartphones,” in Proc. of ACM MobiCom, 2018.

[41]

X. Xu, J. Yu, Y. Chen, Y. Zhu, L. Kong, and M. Li, “Breathlistener: Fine-grained breathing monitoring in driving environments utilizing acoustic signals,” in Proc. of ACM MobiSys, 2019.

[42]

R. Nandakumar, S. Gollakota, and N. Watson, “Contactless sleep apnea detection on smartphones,” in Proc. of ACM MobiSys, 2015.

[43]

H. Kim, A. Byanjankar, Y. Liu, Y. Shu, and I. Shin, “Ubitap: Leveraging acoustic dispersion for ubiquitous touch interface on solid surfaces,” in Proc. of ACM SenSys, 2018.

[44]

M. Chen, P. Yang, J. Xiong, M. Zhang, Y. Lee, C. Xiang, and C. Tian, “Your table can be an input panel: Acoustic-based device-free interaction recognition,” Proc. of ACM IMWUT, vol. 3, Mar. 2019.

[45]

M. Chen, J. Lin, Y. Zou, R. Ruby, and K. Wu, “Silentsign: Device-free handwritten signature verification through acoustic sensing,” in Proc. of IEEE PerCom, 2020.

[46]

Y. Zou, Q. Yang, R. Ruby, Y. Han, S. Wu, M. Li, and K. Wu, “Echowrite: An acoustic-based finger input system without training,” in Proc. of IEEE ICDCS, 2019.

[47]

Y. Zou, Q. Yang, Y. Han, D. Wang, J. Cao, and K. Wu, “Acoudigits: Enabling users to input digits in the air,” in Proc. of IEEE PerCom, 2019.

[48]

L. Lu, J. Yu, Y. Chen, Y. Zhu, X. Xu, G. Xue, and M. Li, “Keylistener: Inferring keystrokes on qwerty keyboard of touch screen through acoustic signals,” in Proc. of IEEE Infocom, 2019.

[49]

K. Sun, T. Zhao, W. Wang, and L. Xie, “Vskin: Sensing touch gestures on surfaces of mobile devices using acoustic signals,” in Proc. of ACM MobiCom, 2018.

[50]

C. Wolff, “Radar basics,” Radar basics, 2018. [Online]. Available: https://www.radartutorial.eu/02.basics/pubs/FMCW-Radar.en.pdf

[51]

H. Zhu, Y. Zhang, Z. Liu, S. Chang, and Y. Chen, “Hyperear: Indoor remote object finding with a single phone,” in Proc. of IEEE ICDCS, 2019.

[52]

N. Yu, W. Wang, A. X. Liu, and L. Kong, “Qgesture: Quantifying gesture distance and direction with wifi signals,” Proc. of ACM IMWUT, vol. 2, 2018.

[53]

Y.-C. Tung, D. Bui, and K. G. Shin, “Cross-platform support for rapid development of mobile acoustic sensing applications,” in Proc. of ACM MobiSys, 2018.

[54]

W. Mao, M. Wang, and L. Qiu, “Aim: Acoustic imaging on a mobile,” in Proc. of ACM MobiSys, 2018.

[55]

F. Adib, H. Mao, Z. Kabelac, D. Katabi, and R. C. Miller, “Smart homes that monitor breathing and heart rate,” in Proc. of ACM CHI, 2015.

Cited By

Zhang JWang J(2024)FusionTrack: Towards Accurate Device-free Acoustic Motion Tracking with Signal FusionACM Transactions on Sensor Networks10.1145/365466620:3(1-30)Online publication date: 30-Mar-2024
https://dl.acm.org/doi/10.1145/3654666
Wu YChen SMeng XTong XLiu XXie XQu WOkoshi TKo JLiKamWa R(2024)Enabling 6D Pose Tracking on Your Acoustic DevicesProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661875(15-28)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661875
Wang SZhong LFu YChen LRen JZhang Y(2024)UFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435468:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643546
Show More Cited By

Index Terms

Push the Limit of Device-Free Acoustic Sensing on Commercial Mobile Devices
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Signal processing systems
2. Human-centered computing

Index terms have been assigned to the content through auto-classification.

Recommendations

PD-FMCW: Push the Limit of Device-Free Acoustic Sensing Using Phase Difference in FMCW
Device-free acoustic sensing has obsessed with renovating human-computer interaction techniques for all-sized mobile devices in various applications. Recent advances have explored sound signals in different methods to achieve highly accurate and efficient ...
Accurate Device-Free Localization with Little Human Cost
SmartObjects '15: Proceedings of the 1st International Workshop on Experiences with the Design and Implementation of Smart Objects

The device-free localization (DFL), i.e., localizing target without requiring target attached any devices, is attractive. Current localization methods, however, query a significant degree of pre-deployment effort, such as the transceivers' locations, ...
Robust Acoustic Self-Localization of Mobile Devices

Self-localization of smart portable devices serves as foundation for several novel applications. This work proposes a set of algorithms that enable a mobile device to passively determine its position relative to a known reference with centimeter ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

IEEE INFOCOM 2021 - IEEE Conference on Computer Communications

May 2021

2503 pages

Copyright © 2021.

Publisher

IEEE Press

Publication History

Published: 10 May 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang JWang J(2024)FusionTrack: Towards Accurate Device-free Acoustic Motion Tracking with Signal FusionACM Transactions on Sensor Networks10.1145/365466620:3(1-30)Online publication date: 30-Mar-2024
https://dl.acm.org/doi/10.1145/3654666
Wu YChen SMeng XTong XLiu XXie XQu WOkoshi TKo JLiKamWa R(2024)Enabling 6D Pose Tracking on Your Acoustic DevicesProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661875(15-28)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661875
Wang SZhong LFu YChen LRen JZhang Y(2024)UFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435468:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643546
Wang XYang ZWang WDai HShi SGu Q(2024)UltraCLR: Contrastive Representation Learning Framework for Ultrasound-based SensingACM Transactions on Sensor Networks10.1145/359749820:4(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3597498
Yang QZheng YEskicioglu RHuang PPatwari N(2023)AquaHelper: Underwater SOS Transmission and Detection in Swimming PoolsProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems10.1145/3625687.3625816(294-307)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3625687.3625816
Wang ZWang YTian MShen J(2023)HearFireProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35695006:4(1-25)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569500

View Options

View options

Figures

Tables

Media

View Table of Conten