research-article

RF-ray: Joint RF and Linguistics Domain Learning for Object Recognition

Authors:

Yihong GongAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 3

Article No.: 96, Pages 1 - 24

https://doi.org/10.1145/3478115

Published: 14 September 2021 Publication History

Abstract

This paper presents a non-invasive design, namely RF-ray, to recognize the shape and material of an object simultaneously. RF-ray puts the object approximate to an RFID tag array, and explores the propagation effect as well as coupling effect between RFIDs and the object for sensing. In contrast to prior proposals, RF-ray is capable to recognize unseen objects, including unseen shape-material pairs and unseen materials within a certain container. To make it real, RF-ray introduces a sensing capability enhancement module and leverages a two-branch neural network for shape profiling and material identification respectively. Furthermore, we incorporate a Zero-Shot Learning based embedding module that incorporates the well-learned linguistic features to generalize RF-ray to recognize unseen materials. We build a prototype of RF-ray using commodity RFID devices. Comprehensive real-world experiments demonstrate our system can achieve high object recognition performance.

References

[1]

[n.d.]. DHS Awards $200k for AI-Based Proof of Concept for TSA. https://www.hstoday.us/industry/dhs-awards-200k-for-ai-based-proof-of-concept-for-tsa/. Government Technology & Services Coalition's Homeland Security, 2019.

[2]

[n.d.]. word2vec. https://code.google.com/archive/p/word2vec/. Google Code, 2013.

[3]

Siddharth Biswal, Joshua Kulas, Haoqi Sun, Balaji Goparaju, M Brandon Westover, Matt T Bianchi, and Jimeng Sun. 2017. SLEEPNET: Automated Sleep Staging System via Deep Learning. arXiv preprint arXiv:1707.08262 (2017).

[4]

Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. 2018. Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks. In Proceedings of IEEE CVPR.

[5]

Ashutosh Dhekne, Mahanth Gowda, Yixuan Zhao, Haitham Hassanieh, and Romit Roy Choudhury. 2018. Liquid: A Wireless Liquid Identifier. In Proceedings of ACM MobiSys.

Digital Library

[6]

Han Ding, Jinsong Han, Chen Qian, Fu Xiao, Ge Wang, Nan Yang, Wei Xi, and Jian Xiao. 2018. Trio: Utilizing tag interference for refined localization of passive RFID. In Proceedings of IEEE INFOCOM.

Digital Library

[7]

Han Ding, Longfei Shangguan, Zheng Yang, Jinsong Han, Zimu Zhou, Panlong Yang, Wei Xi, and Jizhong Zhao. 2015. FEMO: A Platform for Free-weight Exercise Monitoring with RFIDs. In Proceedings of ACM SenSys.

Digital Library

[8]

Shuya Ding, Zhe Chen, Tianyue Zheng, and Jun Luo. 2020. RF-net: A Unified Meta-learning Framework for RF-enabled One-shot Human Activity Recognition. In Proceedings of ACM SenSys.

Digital Library

[9]

Mingming Fan and Khai N Truong. 2015. SoQr: Sonically Quantifying the Content Level Inside Containers. In Proceedings of ACM UbiComp.

Digital Library

[10]

Chuang Gan, Ming Lin, Yi Yang, Yueting Zhuang, and Alexander G Hauptmann. 2015. Exploring Semantic Inter-class Relationships (SIR) for Zero-Shot Action Recognition. In Proceedings of AAAI.

[11]

Unsoo Ha, Junshan Leng, Alaa Khaddaj, and Fadel Adib. 2020. Food and Liquid Sensing in Practical Environments using RFIDs. In Proceedings of USENIX NSDI.

[12]

Unsoo Ha, Yunfei Ma, Zexuan Zhong, Tzu-Ming Hsu, and Fadel Adib. 2018. Learning Food Quality and Safety from Wireless Stickers. In Proceedings of HotNets.

Digital Library

[13]

Jiahui Hou, Xiang-Yang Li, Peide Zhu, Zefan Wang, Yu Wang, Jianwei Qian, and Panlong Yang. 2019. Signspeaker: A Real-time, High-precision Smartwatch-based Sign Language Translator. In Proceedings of ACM MobiCom.

Digital Library

[14]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-Excitation Networks. In Proceedings of the IEEE CVPR.

[15]

Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, et al. 2018. Towards Environment Independent Device Free Human Activity Recognition. In Proceedings of ACM MobiCom.

Digital Library

[16]

Kiran Joshi, Steven Hong, and Sachin Katti. 2013. Pinpoint: Localizing Interfering Radios. In Proceedings of USENIX NSDI.

[17]

Jimmy Lei Ba, Kevin Swersky, Sanja Fidler, et al. 2015. Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions. In Proceedings of IEEE ICCV.

Digital Library

[18]

Kevin J Liang, John B Sigman, Gregory P Spell, Dan Strellis, William Chang, Felix Liu, Tejas Mehta, and Lawrence Carin. 2019. Toward Automatic Threat Recognition for Airport X-ray Baggage Screening with Deep Convolutional Object Detection. arXiv preprint arXiv:1912.06329 (2019).

[19]

Yang Liu, Zhenjiang Li, Zhidan Liu, and Kaishun Wu. 2019. Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors. In Proceedings of ACM MobiSys.

Digital Library

[20]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of Springer MICCAI.

[21]

Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, Inception-resnet and the Impact of Residual Connections on Learning. In Proceedings of AAAI.

[22]

John L Volakis. 2007. Antenna Engineering Handbook. McGraw-Hill Education.

[23]

Chuyu Wang, Jian Liu, Yingying Chen, Hongbo Liu, Lei Xie, Wei Wang, Bingbing He, and Sanglu Lu. 2018. Multi-Touch in the Air: Device-Free Finger Tracking and Gesture Recognition via COTS RFID. In Proceedings of IEEE INFOCOM.

Digital Library

[24]

Ge Wang, Haofan Cai, Chen Qian, Jinsong Han, Xin Li, Han Ding, and Jizhong Zhao. 2018. Towards replay-resilient RFID authentication. In Proceedings of ACM MobiCom.

Digital Library

[25]

Ge Wang, Jinsong Han, Chen Qian, Wei Xi, Han Ding, Zhiping Jiang, and Jizhong Zhao. 2018. Verifiable Smart Packaging with Passive RFID. IEEE Transactions on Mobile Computing 18, 5 (2018), 1217--1230.

[26]

Ju Wang, Liqiong Chang, Shourya Aggarwal, Omid Abari, and Srinivasan Keshav. 2020. Soil moisture sensing with commodity RFID systems. In Proceedings of ACM MobiSys.

Digital Library

[27]

Ju Wang, Jie Xiong, Xiaojiang Chen, Hongbo Jiang, Rajesh Krishna Balan, and Dingyi Fang. 2017. TagScan: Simultaneous Target Imaging and Material Identification with Commodity RFID Devices. In Proceedings of ACM MobiCom.

Digital Library

[28]

Yanwen Wang, Jiaxing Shen, and Yuanqing Zheng. 2020. Push the Limit of Acoustic Gesture Recognition. (2020).

[29]

Binbin Xie, Jie Xiong, Xiaojiang Chen, Eugene Chai, Liyao Li, Zhanyong Tang, and Dingyi Fang. 2019. Tagtag: Material Sensing with Commodity RFID. In Proceedings of ACM SenSys.

Digital Library

[30]

Chenhan Xu, Zhengxiong Li, Hanbin Zhang, Aditya Singh Rathore, Huining Li, Chen Song, Kun Wang, and Wenyao Xu. 2019. WaveEar: Exploring a MMwave-based Noise-resistant Speech Sensing for Voice-user Interface. In Proceedings of ACM MobiCom.

Digital Library

[31]

Yinggang Yu, Dong Wang, Run Zhao, and Qian Zhang. 2019. RFID based Real-time Recognition of Ongoing Gesture with Adversarial Learning. In Proceedings of ACM Sensys.

Digital Library

[32]

Diana Zhang, Jingxian Wang, Junsu Jang, Junbo Zhang, and Swarun Kumar. 2019. On the Feasibility of Wi-Fi Based Material Sensing. In Proceedings of ACM MobiCom.

Digital Library

[33]

Cui Zhao, Zhenjiang Li, Ting Liu, Han Ding, Jinsong Han, Wei Xi, and Ruowei Gui. 2019. RF-Mehndi: A Fingertip Profiled RF Identifier. In Proceedings of IEEE INFOCOM.

Digital Library

[34]

Mingmin Zhao, Yonglong Tian, Hang Zhao, Mohammad Abu Alsheikh, Tianhong Li, Rumen Hristov, Zachary Kabelac, Dina Katabi, and Antonio Torralba. 2018. RF-based 3D Skeletons. In Proceedings of ACM SIGCOMM.

Digital Library

[35]

Yi Zhu, Yang Long, Yu Guan, Shawn Newsam, and Ling Shao. 2018. Towards Universal Representation for Unseen Action Recognition. In Proceedings of IEEE CVPR.

Cited By

Hu JJiang HLiu DXiao ZDustdar SLiu J(2024)A Wireless Self-Service System for Library Using Commodity RFID DevicesIEEE Internet of Things Journal10.1109/JIOT.2023.330146211:3(4998-5010)Online publication date: 1-Feb-2024
https://doi.org/10.1109/JIOT.2023.3301462
He TNiu QLiu N(2023)GC-LocProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694956:4(1-27)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569495
Zhao YWang SZhang L(2023)RSSI Fluctuation Analysis for RFID Tagged and Untagged Liquid Object2023 4th International Conference on Information Science, Parallel and Distributed Systems (ISPDS)10.1109/ISPDS58840.2023.10235597(662-665)Online publication date: 14-Jul-2023
https://doi.org/10.1109/ISPDS58840.2023.10235597
Show More Cited By

Index Terms

RF-ray: Joint RF and Linguistics Domain Learning for Object Recognition
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
2. Networks
  1. Network services

Recommendations

Zero-Shot Object Recognition Using Semantic Label Vectors
CRV '15: Proceedings of the 2015 12th Conference on Computer and Robot Vision

We consider the problem of zero-shot recognition of object categories from images. Given a set of object categories (called "known classes") with training images, our goal is to learn a system to recognize another non-overlapping set of object ...
On Novel Object Recognition: A Unified Framework for Discriminability and Adaptability
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

The rich and accessible labeled data fueled the revolutionary successes of deep learning in object recognition. However, recognizing objects of novel classes with limited supervision information provided, i.e., Novel Object Recognition (NOR), remains a ...
Zero-shot classification by transferring knowledge and preserving data structure

In practical object recognition tasks, one often encounters a problem to recognize some unseen objects, which are some new categories without labeled images at training stage. For solving the challenging problem, zero-shot learning has been studied ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 3

Sept 2021

1443 pages

EISSN:2474-9567

DOI:10.1145/3486621

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Published in IMWUT Volume 5, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
367
Total Downloads

Downloads (Last 12 months)73
Downloads (Last 6 weeks)6

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu JJiang HLiu DXiao ZDustdar SLiu J(2024)A Wireless Self-Service System for Library Using Commodity RFID DevicesIEEE Internet of Things Journal10.1109/JIOT.2023.330146211:3(4998-5010)Online publication date: 1-Feb-2024
https://doi.org/10.1109/JIOT.2023.3301462
He TNiu QLiu N(2023)GC-LocProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694956:4(1-27)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569495
Zhao YWang SZhang L(2023)RSSI Fluctuation Analysis for RFID Tagged and Untagged Liquid Object2023 4th International Conference on Information Science, Parallel and Distributed Systems (ISPDS)10.1109/ISPDS58840.2023.10235597(662-665)Online publication date: 14-Jul-2023
https://doi.org/10.1109/ISPDS58840.2023.10235597
Röddiger TClarke CBreitling PSchneegans TZhao HGellersen HBeigl M(2022)Sensing with EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503146:3(1-57)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550314
Lyu WZhang KGuo BHong ZYang GWang GYang YLiu YZhang DAl Hasan MXiong L(2022)Towards Fair Workload Assessment via Homogeneous Order Grouping in Last-mile DeliveryProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557132(3361-3370)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557132
Ding HZhao CWang GZhao KXi WZhao J(2022)Utilizing Tag Interference for Refined Localization of Passive RFIDIEEE Internet of Things Journal10.1109/JIOT.2021.31379689:14(12656-12672)Online publication date: 15-Jul-2022
https://doi.org/10.1109/JIOT.2021.3137968

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents