Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3290605.3300778acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Enabling Identification and Behavioral Sensing in Homes using Radio Reflections

Published: 02 May 2019 Publication History

Abstract

Understanding users' behavior at home is central to behavioral research. For example, social researchers are interested in studying domestic abuse, and healthcare professionals are interested in caregiver-patient interaction. Today, such studies rely on diaries and questionnaires, which are subjective, erroneous, and hard to sustain in longitudinal studies. We introduce Marko, a system that automatically collects behavior-related data, without asking people to write diaries or wear sensors. Marko transmits a low power wireless signal and analyses its reflections from the environment. It maps those reflections to how users interact with the environment (e.g., access to medication cabinet) and with each other (e.g., watch TV together). It provides novel algorithms for identifying who-does-what, and bootstrapping the system in new homes without asking users for new annotations. We evaluate Marko with a one-month deployment in six homes, and demonstrate its value for studying couple relationships and caregiver-patient interaction.

Supplementary Material

ZIP File (paper548.zip)
A single PDF file that includes analyses of how the labeling and identification network operate and evaluation of our design choices.

References

[1]
2018. mbientlab MetaMotion sensors. Retrieved September 20, 2018 from https://mbientlab.com
[2]
2018. Walabot. Retrieved September 20, 2018 from https://walabot. com/
[3]
Fadel Adib, Chen-Yu Hsu, Hongzi Mao, Dina Katabi, and Frédo Durand. 2015. Capturing the human figure through a wall. ACM Transactions on Graphics (TOG) 34, 6 (2015), 219.
[4]
Fadel Adib, Zachary Kabelac, and Dina Katabi. 2015. Multi-person localization via RF body reflections. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). 279--292.
[5]
Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C Miller. 2015. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, 837--846.
[6]
Nadav Aharony, Wei Pan, Cory Ip, Inas Khayal, and Alex Pentland. 2011. The social fMRI: measuring, understanding, and designing social mechanisms in the real world. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 445--454.
[7]
Chuankai An, Tianxing Li, Zhao Tian, Andrew T Campbell, and Xia Zhou. 2015. Visible light knows who you are. In Proceedings of the 2nd International Workshop on Visible Light Communications Systems. ACM, 39--44.
[8]
Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5--32.
[9]
Yuanying Chen, Wei Dong, Yi Gao, Xue Liu, and Tao Gu. 2017. Rapid: A Multimodal and Device-free Approach Using Noise Estimation for Robust Person Identification. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 41.
[10]
James Clawson, Jessica A. Pater, Andrew D. Miller, Elizabeth D. Mynatt, and Lena Mamykina. 2015. No Longer Wearing: Investigating the Abandonment of Personal Health-tracking Technologies on Craigslist. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). ACM, New York, NY, USA, 647--658.
[11]
Federal Communications Commission. 2018. Rules and Regulations for Title 47.
[12]
Sunny Consolvo, David W McDonald, Tammy Toscos, Mike Y Chen, Jon Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand, Ryan Libby, et al. 2008. Activity sensing in the wild: a field trial of ubifit garden. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1797--1806.
[13]
Ann C Crouter, Maureen Perry-Jenkins, Ted L Huston, and Duane W Crawford. 1989. The influence of work-induced psychological states on behavior at home. Basic and Applied Social Psychology 10, 3 (1989), 273--292.
[14]
Daniela Fialová, Eva Topinková, Giovanni Gambassi, Harriet FinneSoveri, Pálmi V Jónsson, Iain Carpenter, Marianne Schroll, Graziano Onder, Liv Wergeland Sørbye, Cordula Wagner, et al. 2005. Potentially inappropriate medication use among elderly home care patients in Europe. Jama 293, 11 (2005), 1348--1358.
[15]
Gerda G Fillenbaum. 1985. Screening the elderly: A brief instrumental activities of daily living measure. Journal of the American Geriatrics Society 33, 10 (1985), 698--706.
[16]
Glenn Fox, Shaun Garland, Andrew Keibel, and Leslie Saxon. 2017. Why People Stick With or Abandon Wearable Devices. https://catalyst. nejm.org/stay-abandon-wearable-devices/
[17]
Hulya Gokalp and Malcolm Clarke. 2013. Monitoring activities of daily living of the elderly and the potential for its use in telecare and telehealth: a review. TELEMEDICINE and e-HEALTH 19, 12 (2013), 910--923.
[18]
Tobias Grosse-Puppendahl, Xavier Dellangnol, Christian Hatzfeld, Biying Fu, Mario Kupnik, Arjan Kuijper, Matthias R Hastall, James Scott, and Marco Gruteser. 2016. Platypus: Indoor Localization and Identification Through Sensing of Electric Potential Changes in Human Bodies. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 17--30.
[19]
Abdelsalam Helal, Diane J Cook, and Mark Schmalz. 2009. Smart home-based health platform for behavioral monitoring and alteration of diabetes patients. Journal of diabetes science and technology 3, 1 (2009), 141--148.
[20]
Timothy W Hnat, Erin Griffiths, Ray Dawson, and Kamin Whitehouse. 2012. Doorjamb: unobtrusive room-level tracking of people in homes using doorway sensors. In Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems. ACM, 309--322.
[21]
Sandra L Hofferth and John F Sandberg. 2001. How American children spend their time. Journal of Marriage and Family 63, 2 (2001), 295--308.
[22]
Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric user identification on mobile devices using the capacitive touchscreen to scan body parts. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, 3011--3014.
[23]
Feng Hong, Xiang Wang, Yanni Yang, Yuan Zong, Yuliang Zhang, and Zhongwen Guo. 2016. WFID: Passive Device-free Human Identification Using WiFi Signal. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ACM, 47--56.
[24]
Chen-Yu Hsu, Aayush Ahuja, Shichao Yue, Rumen Hristov, Zachary Kabelac, and Dina Katabi. 2017. Zero-Effort In-Home Sleep and Insomnia Monitoring using Radio Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 59.
[25]
Chen-Yu Hsu, Yuchen Liu, Zachary Kabelac, Rumen Hristov, Dina Katabi, and Christine Liu. 2017. Extracting Gait Velocity and Stride Length from Surrounding Radio Signals. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2116--2126.
[26]
Remus Ilies, Kelly M Schwind, David T Wagner, Michael D Johnson, D Scott DeRue, and Daniel R Ilgen. 2007. When can employees have a family life? The effects of daily workload and affect on work-family conflict and social behaviors at home. Journal of Applied Psychology 92, 5 (2007), 1368.
[27]
Kiran Joshi, Dinesh Bharadia, Manikanta Kotaru, and Sachin Katti. 2015. WiDeo: fine-grained device-free motion tracing using RF backscatter. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation. USENIX Association, 189--204.
[28]
Avinash Kalyanaraman, Dezhi Hong, Elahe Soltanaghaei, and Kamin Whitehouse. 2017. Forma Track: Tracking People based on Body Shape. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 61.
[29]
Sidney Katz. 1983. Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. Journal of the American Geriatrics Society 31, 12 (1983), 721--727.
[30]
Nacer Khalil, Driss Benhaddou, Omprakash Gnawali, and Jaspal Subhlok. 2016. Nonintrusive Occupant Identification by Sensing Body Shape and Movement. In Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments. ACM, 1--10.
[31]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[32]
Nicholas D Lane, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew T Campbell. 2010. A survey of mobile phone sensing. IEEE Communications magazine 48, 9 (2010).
[33]
Amanda Lazar, Christian Koehler, Joshua Tanenbaum, and David H. Nguyen. 2015. Why We Use and Abandon Smart Devices. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). ACM, New York, NY, USA, 635--646.
[34]
D Ledger and D McCaffrey. 2014. Inside wearables: How the science of human behavior change offers the secret to long-Term engagement. 93 (01 2014), 36--45.
[35]
Lily Lee and W Eric L Grimson. 2002. Gait analysis for recognition and classification. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on. IEEE, 155--162.
[36]
Hanchuan Li, Peijin Zhang, Samer Al Moubayed, Shwetak N Patel, and Alanson P Sample. 2016. Id-match: A hybrid computer vision and rfid system for recognizing individuals in groups. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4933--4944.
[37]
Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. 2016. Dynamic-music: accurate device-free indoor localization. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 196--207.
[38]
Wen-Hau Liau, Chao-Lin Wu, and Li-Chen Fu. 2008. Inhabitants tracking system in a cluttered home environment via floor load sensors. IEEE Transactions on Automation Science and Engineering 5, 1 (2008), 10--20.
[39]
Andreas Möller, Matthias Kranz, Barbara Schmid, Luis Roalter, and Stefan Diewald. 2013. Investigating self-reporting behavior in longterm studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2931--2940.
[40]
Kristiina Moller, Philip C Hwang, and Birgitta Wickberg. 2008. Couple relationship and transition to parenthood: Does workload at home matter? Journal of reproductive and infant psychology 26, 1 (2008), 57--68.
[41]
Robert J Orr and Gregory D Abowd. 2000. The smart floor: A mechanism for natural user identification and tracking. In CHI'00 extended abstracts on Human factors in computing systems. ACM, 275--276.
[42]
O. M. Parkhi, A. Vedaldi, and A. Zisserman. 2015. Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC).
[43]
European Parliament and Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union L119 (2016), 1--88.
[44]
Gregory S Pettit and John E Bates. 1989. Family interaction patterns and children's behavior problems from infancy to 4 years. Developmental psychology 25, 3 (1989), 413.
[45]
Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 27--38.
[46]
Kiran K Rachuri, Mirco Musolesi, Cecilia Mascolo, Peter J Rentfrow, Chris Longworth, and Andrius Aucinas. 2010. EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In Proceedings of the 12th ACM international conference on Ubiquitous computing. ACM, 281--290.
[47]
Tauhidur Rahman, Alexander T Adams, Ruth Vinisha Ravichandran, Mi Zhang, Shwetak N Patel, Julie A Kientz, and Tanzeem Choudhury. 2015. Dopplesleep: A contactless unobtrusive sleep sensing system using short-range doppler radar. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 39--50.
[48]
Stefan Schneegass, Youssef Oualil, and Andreas Bulling. 2016. SkullConduct: Biometric user identification on eyewear computers using bone conduction through the skull. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1379--1384.
[49]
Yu-Lin Shen and Chow-Shing Shin. 2009. Distributed sensing floor for an intelligent environment. IEEE Sensors Journal 9, 12 (2009), 1673-- 1678.
[50]
Cong Shi, Jian Liu, Hongbo Liu, and Yingying Chen. 2017. Smart User Authentication through Actuation of Daily Activities Leveraging WiFienabled IoT. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 5.
[51]
Patrick Shih, Kyungsik Han, Erika Shehan Poole, Mary Beth Rosson, and John Carroll. 2015. Use and adoption challenges of wearable activity trackers. (03 2015).
[52]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[53]
Kaycee M Sink, Karen F Holden, and Kristine Yaffe. 2005. Pharmacological treatment of neuropsychiatric symptoms of dementia: a review of the evidence. Jama 293, 5 (2005), 596--608.
[54]
Robert Steele, Amanda Lo, Chris Secombe, and Yuk Kuen Wong. 2009. Elderly persons' perception and acceptance of using wireless sensor networks to assist healthcare. International journal of medical informatics 78, 12 (2009), 788--801.
[55]
Andreas E Stuck, Harriet U Aronow, Andrea Steiner, Cathy A Alessi, Christophe J Büla, Marcia N Gold, Karen E Yuhas, Rosane Nisenbaum, Laurence Z Rubenstein, and John C Beck. 1995. A trial of annual in-home comprehensive geriatric assessments for elderly people living in the community. New England Journal of Medicine 333, 18 (1995), 1184--1189.
[56]
Rui Wang, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia Zhou, Dror Ben-Zeev, and Andrew T Campbell. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing. ACM, 3--14.
[57]
Wei Wang, Alex X Liu, and Muhammad Shahzad. 2016. Gait recognition using wifi signals. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 363-- 373.
[58]
Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st annual international conference on mobile computing and networking. ACM, 65--76.
[59]
Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, 617--628.
[60]
Rick Wash, Emilee Rader, and Chris Fennell. 2017. Can People SelfReport Security Accurately?: Agreement Between Self-Report and Behavioral Measures. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2228--2232.
[61]
Luke Weston. 2014. Simple, low-cost FMCW radar. Retrieved September 20, 2018 from https://hackaday.io/project/ 1682-simple-low-cost-fmcw-radar#menu-description
[62]
Yunze Zeng, Parth H Pathak, and Prasant Mohapatra. 2016. WiWho: wifi-based person identification in smart spaces. In Proceedings of the 15th International Conference on Information Processing in Sensor Networks. IEEE Press, 4.
[63]
Yang Zhang, Chouchang Jack Yang, Scott E Hudson, Chris Harrison, and Alanson Sample. 2018. Wall++: Room-Scale Interactive and Context-Aware Sensing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 273.
[64]
Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. 2018. Throughwall human pose estimation using radio signals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7356-- 7365.

Cited By

View all
  • (2025)Advanced Techniques for Human‐Centric Sensing in Environmental MonitoringNetworked Sensing Systems10.1002/9781394310890.ch4(101-120)Online publication date: 2-Feb-2025
  • (2024)SiWiS: Fine-grained Human Detection Using Single WiFi DeviceProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690703(1439-1454)Online publication date: 4-Dec-2024
  • (2024)PmTrackProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314337:4(1-30)Online publication date: 12-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
May 2019
9077 pages
ISBN:9781450359702
DOI:10.1145/3290605
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. behavioral sensing
  2. passive rf sensing
  3. smart environments
  4. user identification
  5. wireless sensing

Qualifiers

  • Research-article

Conference

CHI '19
Sponsor:

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)87
  • Downloads (Last 6 weeks)4
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Advanced Techniques for Human‐Centric Sensing in Environmental MonitoringNetworked Sensing Systems10.1002/9781394310890.ch4(101-120)Online publication date: 2-Feb-2025
  • (2024)SiWiS: Fine-grained Human Detection Using Single WiFi DeviceProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690703(1439-1454)Online publication date: 4-Dec-2024
  • (2024)PmTrackProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314337:4(1-30)Online publication date: 12-Jan-2024
  • (2024)Tagnoo: Enabling Smart Room-Scale Environments with RFID-Augmented PlywoodProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642356(1-18)Online publication date: 11-May-2024
  • (2024)RadarFormer: End-to-End Human Perception With Through-Wall Radar and TransformersIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.331403135:12(18285-18299)Online publication date: Dec-2024
  • (2024)MobiRFPose: Portable RF-Based 3D Human Pose CameraIEEE Transactions on Multimedia10.1109/TMM.2023.331497926(3715-3727)Online publication date: 2024
  • (2024)RPM: RF-Based Pose MachinesIEEE Transactions on Multimedia10.1109/TMM.2023.326837626(637-649)Online publication date: 2024
  • (2024)WiRITE: General and Practical Wi-Fi Based Hand-Writing RecognitionIEEE Transactions on Mobile Computing10.1109/TMC.2023.326598823:4(2943-2957)Online publication date: Apr-2024
  • (2024)WiFi-Based Human Sensing With Deep Learning: Recent Advances, Challenges, and OpportunitiesIEEE Open Journal of the Communications Society10.1109/OJCOMS.2024.34115295(3595-3623)Online publication date: 2024
  • (2024)A Wireless Self-Service System for Library Using Commodity RFID DevicesIEEE Internet of Things Journal10.1109/JIOT.2023.330146211:3(4998-5010)Online publication date: 1-Feb-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media