Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3458864.3467885acmconferencesArticle/Chapter ViewAbstractPublication PagesmobisysConference Proceedingsconference-collections
research-article
Public Access

ITrackU: tracking a pen-like instrument via UWB-IMU fusion

Published: 24 June 2021 Publication History

Abstract

High-precision tracking of a pen-like instrument's movements is desirable in a wide range of fields spanning education, robotics, and art, to name a few. The key challenge in doing so stems from the impracticality of embedding electronics in the tip of such instruments (a pen, marker, scalpel, etc.) as well as the difficulties in instrumenting the surface that it works on. In this paper, we present ITrackU, a movement digitization system that does not require modifications to the surface or the tracked instrument's tip. ITrackU fuses locations obtained using ultra-wideband radios (UWB), with an inertial and magnetic unit (IMU) and a pressure sensor, yielding multidimensional improvements in accuracy, range, cost, and robustness, over existing works. ITrackU embeds a micro-transmitter at the base of a pen which creates a trackable beacon, that is localized from the corners of a writing surface. Fused with inertial motion sensor and a pressure sensor, ITrackU enables accurate tracking. Our prototype of ITrackU covers a large 2.5m × 2m area, while obtaining around 2.9mm median error. We demonstrate the accuracy of our system by drawing numerous shapes and characters on a whiteboard, and compare them against a touchscreen and a camera-based ground-truthing system. Finally, the produced stream of digitized data is minuscule in volume, when compared with a video of the whiteboard, which saves both network bandwidth and storage space.

References

[1]
Ieee standard for low-rate wireless networks. IEEE Std 802.15.4-2015 (Revision of IEEE Std 802.15.4-2011), pages 1--709, 2016.
[2]
Magnetic calibration algorithms, 2016. https://www.nxp.com/docs/en/application-note/AN5019.pdf.
[3]
4-wire analog resistive touch screen datasheet, 2017. https://media.digikey.com/pdf/Data%20Sheets/NKK%20PDFs/FT_Series_4-Wire_Ds_Oct_2017.pdf.
[4]
Decawave user manual, 2017. https://www.decawave.com/sites/default/files/resources/dw1000_user_manual_2.11.pdf.
[5]
Microsoft surface pen, 2017. https://news.microsoft.com/uploads/2017/05/SurfacePenFS.pdf.
[6]
Apple pencil, 2018. https://www.apple.com/apple-pencil/.
[7]
S pen, 2018. https://www.samsung.com/global/galaxy/galaxy-note10/s-pen/.
[8]
MAVL: Multiresolution analysis of voice localization. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21), Boston, MA, April 2021. USENIX Association.
[9]
Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 3d tracking via body radio reflections. In 11th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 14), pages 317--329, 2014.
[10]
Fadel Adib and Dina Katabi. See through walls with wifi! In Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM, pages 75--86, 2013.
[11]
Yang Bai, Li Lu, Jerry Cheng, Jian Liu, Yingying Chen, and Jiadi Yu. Acoustic-based sensing and applications: A survey. Computer Networks, 181:107447, 2020.
[12]
Gaoshuai Cao, Kuang Yuan, Jie Xiong, Panlong Yang, Yubo Yan, Hao Zhou, and Xiang-Yang Li. Earphonetrack: involving earphones into the ecosystem of acoustic motion tracking. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pages 95--108, 2020.
[13]
Yifeng Cao, Ashutosh Dhekne, and Mostafa Ammar. 6fit-a-part: A protocol for physical distancing on a custom wearable device. In 2020 IEEE 28th International Conference on Network Protocols (ICNP), pages 1--12. IEEE, 2020.
[14]
Boxlight Corporation. Mimiocapture ink recorder datasheet, 2016. https://mimio.boxlight.com/wp-content/uploads/2017/03/MimioCapture_SellSheet.pdf.
[15]
Decawave. Transmit power calibration and management, 2019. https://www.decawave.com/wp-content/uploads/2019/07/APS023_Part-1_Transmit_Power_Calibration__Management.pdf.
[16]
Ashutosh Dhekne, Ayon Chakraborty, Karthikeyan Sundaresan, and Sampath Rangarajan. Trackio: tracking first responders inside-out. In 16th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 19), pages 751--764, 2019.
[17]
Tobias Feigl, Sebastian Kram, Philipp Woller, Ramiz H Siddiqui, Michael Philippsen, and Christopher Mutschler. Rnn-aided human velocity estimation from a single imu. Sensors, 20(13):3656, 2020.
[18]
Mahanth Gowda, Ashutosh Dhekne, Sheng Shen, Romit Roy Choudhury, Lei Yang, Suresh Golwalkar, and Alexander Essanian. Bringing iot to sports analytics. In 14th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 17), pages 499--513, 2017.
[19]
Mahanth Gowda, Justin Manweiler, Ashutosh Dhekne, Romit Roy Choudhury, and Justin D Weisz. Tracking drone orientation with multiple gps receivers. In Proceedings of the 22nd annual international conference on mobile computing and networking, pages 280--293, 2016.
[20]
Bernhard Großwindhager, Michael Rath, Josef Kulmer, Mustafa S Bakr, Carlo Alberto Boano, Klaus Witrisal, and Kay Römer. Salma: Uwb-based single-anchor localization system using multipath assistance. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pages 132--144, 2018.
[21]
Pengfei Guo, Haitao Qiu, Yunchun Yang, and Zhang Ren. The soft iron and hard iron calibration method using extended kalman filter for attitude and heading reference system. In 2008 IEEE/ION Position, Location and Navigation Symposium, pages 1167--1174. IEEE, 2008.
[22]
Jose A. Gutierrez, Edgar H. Callaway, and Raymond L. Barrett. IEEE 802.15.4 Low-Rate Wireless Personal Area Networks: Enabling Wireless Sensor Networks. IEEE, 2003.
[23]
Evan G Hemingway and Oliver M O'Reilly. Perspectives on euler angle singularities, gimbal lock, and the orthogonality of applied forces and applied moments. Multibody System Dynamics, 44(1):31--56, 2018.
[24]
KC Ho and Wenwei Xu. An accurate algebraic solution for moving source location using tdoa and fdoa measurements. IEEE Transactions on Signal Processing, 52(9):2453--2463, 2004.
[25]
Ngoc-Huynh Ho, Phuc Huu Truong, and Gu-Min Jeong. Step-detection and adaptive step-length estimation for pedestrian dead-reckoning at various walking speeds using a smartphone. Sensors, 16(9):1423, 2016.
[26]
Fang Hongzhao. 60 GHz RSS localization with omni-directional and horn antennas. PhD thesis, 2010.
[27]
Carl Q Howard, Colin H Hansen, and Anthony C Zander. A review of current ultrasound exposure limits. The Journal of Occupational Health and Safety of Australia and New Zealand, 21(3):253--257, 2005.
[28]
Wenchao Huang, Yan Xiong, Xiang-Yang Li, Hao Lin, Xufei Mao, Panlong Yang, and Yunhao Liu. Shake and walk: Acoustic direction finding and fine-grained indoor localization using smartphones. In IEEE INFOCOM 2014-IEEE Conference on Computer Communications, pages 370--378. IEEE, 2014.
[29]
Chengkun Jiang, Junchen Guo, Yuan He, Meng Jin, Shuai Li, and Yunhao Liu. mmvib: micrometer-level vibration measurement with mmwave radar. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, pages 1--13, 2020.
[30]
Benjamin Kempke, Pat Pannuto, Bradford Campbell, and Prabal Dutta. Surepoint: Exploiting ultra wideband flooding and diversity to provide robust, scalable, high-fidelity indoor localization. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, pages 137--149, 2016.
[31]
Wei Li and Jinling Wang. Effective adaptive kalman filter for memsimu/magnetometers integrated attitude and heading reference systems. The Journal of Navigation, 66(1):99--113, 2013.
[32]
Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen, and Fan Ye. Push the limit of wifi based localization for smartphones. In Proceedings of the 18th annual international conference on Mobile computing and networking, pages 305--316, 2012.
[33]
Yang Liu, Zhenjiang Li, Zhidan Liu, and Kaishun Wu. Real-time arm skeleton tracking and gesture inference tolerant to missing wearable sensors. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services, pages 287--299, 2019.
[34]
Robert Mahony, Tarek Hamel, and Jean-Michel Pflimlin. Nonlinear complementary filters on the special orthogonal group. IEEE Transactions on automatic control, 53(5):1203--1218, 2008.
[35]
Wenguang Mao, Jian He, and Lili Qiu. Cat: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, pages 69--81, 2016.
[36]
Microsoft. Change handwritten ink to shapes, text, or math in powerpoint for microsoft 365, 2020. https://support.microsoft.com/en-us/topic/change-handwritten-ink-to-shapes-text-or-math-in-powerpoint-for-microsoft-365-0740dec3-6291-4c1f-8baa-011d18449919.
[37]
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 1515--1525, 2016.
[38]
Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. Risq: Recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services, pages 149--161, 2014.
[39]
Neal Patwari, Lara Brewer, Quinn Tate, Ossi Kaltiokallio, and Maurizio Bocca. Breathfinding: A wireless network that monitors and locates breathing in a home. IEEE Journal of Selected Topics in Signal Processing, 8(1):30--42, 2013.
[40]
Urs Ramer. An iterative procedure for the polygonal approximation of plane curves. Computer graphics and image processing, 1(3):244--256, 1972.
[41]
Nirupam Roy, Haitham Hassanieh, and Romit Roy Choudhury. Backdoor: Making microphones hear inaudible sounds. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, pages 2--14, 2017.
[42]
Nirupam Roy, Sheng Shen, Haitham Hassanieh, and Romit Roy Choudhury. Inaudible voice commands: The long-range attack and defense. In 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18), pages 547--560, 2018.
[43]
Longfei Shangguan and Kyle Jamieson. Leveraging electromagnetic polarization in a two-antenna whiteboard in the air. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, pages 443--456, 2016.
[44]
Sheng Shen, He Wang, and Romit Roy Choudhury. I am a smartwatch and i can track my user's arm. In Proceedings of the 14th annual international conference on Mobile systems, applications, and services, pages 85--96, 2016.
[45]
Ke Sun, Wei Wang, Alex X Liu, and Haipeng Dai. Depth aware finger tapping on virtual displays. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, pages 283--295, 2018.
[46]
Ke Sun, Ting Zhao, Wei Wang, and Lei Xie. Vskin: Sensing touch gestures on surfaces of mobile devices using acoustic signals. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking, pages 591--605, 2018.
[47]
Emily Tate. Sensonor stim3000, June 2020. https://www.sensonor.com/products/inertial-measurement-units/stim300/.
[48]
David Tse and Pramod Viswanath. Fundamentals of wireless communication. Cambridge university press, 2005.
[49]
He Wang, Ted Tsung-Te Lai, and Romit Roy Choudhury. Mole: Motion leaks through smartwatch sensors. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 155--166, 2015.
[50]
He Wang, Souvik Sen, Ahmed Elgohary, Moustafa Farid, Moustafa Youssef, and Romit Roy Choudhury. No need to war-drive: Unsupervised indoor localization. In Proceedings of the 10th international conference on Mobile systems, applications, and services, pages 197--210, 2012.
[51]
Jue Wang, Deepak Vasisht, and Dina Katabi. Rf-idraw: virtual touch screen in the air using rf signals. ACM SIGCOMM Computer Communication Review, 44(4):235--246, 2014.
[52]
Wei Wang, Alex X Liu, and Ke Sun. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, pages 82--94, 2016.
[53]
Teng Wei and Xinyu Zhang. mtrack: High-precision passive tracking using millimeter wave radios. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 117--129, 2015.
[54]
Dan Wu, Ruiyang Gao, Youwei Zeng, Jinyi Liu, Leye Wang, Tao Gu, and Daqing Zhang. Fingerdraw: Sub-wavelength level finger motion tracking with wifi signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1):1--27, 2020.
[55]
Ning Xiao, Panlong Yang, Xiang-Yang Li, Yanyong Zhang, Yubo Yan, and Hao Zhou. Milliback: Real-time plug-n-play millimeter level tracking using wireless backscattering. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(3):1--23, 2019.
[56]
Chao Xu, Parth H Pathak, and Prasant Mohapatra. Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, pages 9--14, 2015.
[57]
Chenren Xu, Bernhard Firner, Robert S Moore, Yanyong Zhang, Wade Trappe, Richard Howard, Feixiong Zhang, and Ning An. Scpl: Indoor device-free multi-subject counting and localization using radio signal strength. In Proceedings of the 12th international conference on Information processing in sensor networks, pages 79--90, 2013.
[58]
Lei Yang, Yekui Chen, Xiang-Yang Li, Chaowei Xiao, Mo Li, and Yunhao Liu. Tagoram: Real-time tracking of mobile rfid tags to high precision using cots devices. In Proceedings of the 20th annual international conference on Mobile computing and networking, pages 237--248, 2014.
[59]
Seanglidet Yean, Bu Sung Lee, Chai Kiat Yeo, and Chan Hua Vun. Algorithm for 3d orientation estimation based on kalman filter and gradient descent. In 2016 IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pages 1--6. IEEE, 2016.
[60]
Huanpu Yin, Anfu Zhou, Guangyuan Su, Bo Chen, Liang Liu, and Huadong Ma. Learning to recognize handwriting input with acoustic features. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(2):1--26, 2020.
[61]
Tuo Yu, Haiming Jin, and Klara Nahrstedt. Mobile devices based eavesdropping of handwriting. IEEE Transactions on Mobile Computing, 2019.
[62]
Sangki Yun, Yi-Chao Chen, and Lili Qiu. Turning a mobile device into a mouse in the air. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, pages 15--29, 2015.
[63]
Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th annual international conference on mobile systems, applications, and services, pages 15--28, 2017.
[64]
Bing Zhou, Jay Lohokare, Ruipeng Gao, and Fan Ye. Echoprint: Two-factor authentication using acoustics and vision on smartphones. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking, pages 321--336, 2018.

Cited By

View all
  • (2024)Push the Limit of Highly Accurate Ranging on Commercial UWB DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596028:2(1-27)Online publication date: 15-May-2024
  • (2024)UTrack3D: 3D Tracking Using Ultra-wideband (UWB) RadiosProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661881(345-358)Online publication date: 3-Jun-2024
  • (2024)Enabling 6D Pose Tracking on Your Acoustic DevicesProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661875(15-28)Online publication date: 3-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MobiSys '21: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services
June 2021
528 pages
ISBN:9781450384438
DOI:10.1145/3458864
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 June 2021

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

Conference

MobiSys '21
Sponsor:

Acceptance Rates

MobiSys '21 Paper Acceptance Rate 36 of 166 submissions, 22%;
Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)563
  • Downloads (Last 6 weeks)55
Reflects downloads up to 04 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Push the Limit of Highly Accurate Ranging on Commercial UWB DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596028:2(1-27)Online publication date: 15-May-2024
  • (2024)UTrack3D: 3D Tracking Using Ultra-wideband (UWB) RadiosProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661881(345-358)Online publication date: 3-Jun-2024
  • (2024)Enabling 6D Pose Tracking on Your Acoustic DevicesProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661875(15-28)Online publication date: 3-Jun-2024
  • (2024)ScribeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314117:4(1-31)Online publication date: 12-Jan-2024
  • (2024)FreqMAE: Frequency-Aware Masked Autoencoder for Multi-Modal IoT SensingProceedings of the ACM Web Conference 202410.1145/3589334.3645346(2795-2806)Online publication date: 13-May-2024
  • (2024)Spectrum Sensing Everywhere: Wide-Band Spectrum Sensing With Low-Cost UWB NodesIEEE/ACM Transactions on Networking10.1109/TNET.2023.334297732:3(2112-2127)Online publication date: Jun-2024
  • (2024)Dynamic Feasible Region-Based IMU/UWB Fusion Method for Indoor PositioningIEEE Sensors Journal10.1109/JSEN.2024.339878924:13(21447-21457)Online publication date: 1-Jul-2024
  • (2024)HeadTrack: Real-Time Human–Computer Interaction via Wireless EarphonesIEEE Journal on Selected Areas in Communications10.1109/JSAC.2023.334538142:4(990-1002)Online publication date: Apr-2024
  • (2023)Emma: An accurate, efficient, and multi-modality strategy for autonomous vehicle angle predictionIntelligent and Converged Networks10.23919/ICN.2023.00044:1(41-49)Online publication date: Mar-2023
  • (2023)XRLoc: Accurate UWB Localization to Realize XR DeploymentsProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems10.1145/3625687.3625810(459-473)Online publication date: 12-Nov-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

ePub

View this article in ePub.

ePub

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media