research-article

Object hallmarks: identifying object users using wearable wrist sensors

Authors:

Kamin WhitehouseAuthors Info & Claims

UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 51 - 61

https://doi.org/10.1145/2750858.2804263

Published: 07 September 2015 Publication History

Abstract

In order for objects to perform personalized or contextual functions based on identity, they must solve what we call the object user identification problem: understanding who is actually using them. In this paper, we propose a new technique that uses data from wearable wrist sensors to perform object user identification. We hypothesize that objects have unique hallmarks that are imprinted in the hand gestures of its users. By detecting the presence of an object's hallmark in the wrist sensor data, we can identify who used the object. We evaluate this concept with a smart home application: recognizing who is using an object or appliance in a multi-person home by combining smart meter data and wearables. We conduct three different studies with 10 participants: 1) a study with scripted object use 2) a study with high-level tasked activities and unscripted object use, and 3) a 5-day in-situ study. These studies indicate that our approach performs object user identification with an average accuracy of 85--90%.

References

[1]

Android Wear Orientation Sensor. http://developer.android.com/reference/android/hardware/Sensor.html#TYPE_ROTATION_VECTOR.

[2]

Fitbit Aria: Smarter scale. Better Results. https://www.fitbit.com/aria.

[3]

Batra, N., Kelly, J., Parson, O., Dutta, H., Knottenbelt, W., Rogers, A., Singh, A., and Srivastava, M. Nilmtk: An open source toolkit for non-intrusive load monitoring. In Proceedings of the 5th international conference on Future energy systems, ACM (2014), 265--276.

Digital Library

[4]

Berlin, E., Liu, J., Van Laerhoven, K., and Schiele, B. Coming to grips with the objects we grasp: detecting interactions with efficient wrist-worn sensors. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, ACM (2010), 57--64.

Digital Library

[5]

Chang, K.-h., Hightower, J., and Kveton, B. Inferring identity using accelerometers in television remote controls. In Pervasive Computing. Springer, 2009, 151--167.

Digital Library

[6]

Cheng, Y., Chen, K., Zhang, B., Liang, C.-J. M., Jiang, X., and Zhao, F. Accurate real-time occupant energy-footprinting in commercial buildings. In Proceedings of the Fourth ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings, ACM (2012), 115--122.

Digital Library

[7]

Garmin Navigation System. http://www.garmin.com/en-US.

[8]

Hamilton, M. T., Healy, G. N., Dunstan, D. W., Zderic, T. W., and Owen, N. Too little exercise and too much sitting: inactivity physiology and the need for new recommendations on sedentary behavior. Current cardiovascular risk reports 2, 4 (2008), 292--298.

[9]

Hart, G. W. Nonintrusive appliance load monitoring. Proceedings of the IEEE 80, 12 (1992), 1870--1891.

[10]

Hay, S., and Rice, A. The case for apportionment. In Proceedings of the First ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings, BuildSys '09, ACM (2009), 13--18.

Digital Library

[11]

Ho, B.-J., Kao, H.-L. C., Chen, N.-C., You, C.-W., Chu, H.-H., and Chen, M.-S. Heatprobe: a thermal-based power meter for accounting disaggregated electricity usage. In Proceedings of the 13th international conference on Ubiquitous computing, ACM (2011), 55--64.

Digital Library

[12]

UX90-002M: HOBO UX90 Light On/Off Logger with Extended Memory. http://www.onsetcomp.com/products/data-loggers/ux90-002m.

[13]

Hodges, M. R., and Pollack, M. E. An object-use fingerprint: The use of electronic sensors for human identification. In UbiComp 2007: Ubiquitous Computing. Springer, 2007, 289--303.

Digital Library

[14]

Kim, Y., Schmid, T., Charbiwala, Z. M., and Srivastava, M. B. Viridiscope: design and implementation of a fine grained power monitoring system for homes. In Proceedings of the 11th international conference on Ubiquitous computing, ACM (2009), 245--254.

Digital Library

[15]

Lee, S., Ahn, D., Lee, S., Ha, R., and Cha, H. Personalized energy auditor: Estimating personal electricity usage. In Pervasive Computing and Communications (PerCom), 2014 IEEE International Conference on, IEEE (2014), 44--49.

[16]

Leeb, S. B., Shaw, S. R., and Kirtley Jr, J. L. Transient event detection in spectral envelope estimates for nonintrusive load monitoring. Power Delivery, IEEE Transactions on 10, 3 (1995), 1200--1210.

[17]

LG G Watch. http://www.lg.com/us/smartwatch/g-watch-r.

[18]

Marceau, M. L., and Zmeureanu, R. Nonintrusive load disaggregation computer program to estimate the energy consumption of major end uses in residential buildings. Energy Conversion and Management 41, 13 (2000), 1389--1403.

[19]

Nest Thermostat. https://nest.com/thermostat/.

[20]

Philipose, M., Fishkin, K. P., Perkowitz, M., Patterson, D. J., Fox, D., Kautz, H., and Hahnel, D. Inferring activities from interactions with objects. Pervasive Computing, IEEE 3, 4 (2004), 50--57.

Digital Library

[21]

Ranjan, J., Griffiths, E., and Whitehouse, K. Discerning electrical and water usage by individuals in homes. In Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings, ACM (2014), 20--29.

Digital Library

[22]

Ranjan, J., Yao, Y., and Whitehouse, K. An rf doormat for tracking people's room locations. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, ACM (2013), 797--800.

Digital Library

[23]

Ruzzelli, A. G., Nicolas, C., Schoofs, A., and O'Hare, G. M. Real-time recognition and profiling of appliances through a single electricity sensor. In Sensor Mesh and Ad Hoc Communications and Networks (SECON), 2010 7th Annual IEEE Communications Society Conference on, IEEE (2010), 1--9.

[24]

Google's Self Driving Car. http://www.google.com/selfdrivingcar/.

[25]

UX90-001: HOBO UX90 State Logger. http://www.onsetcomp.com/products/data-loggers/ux90-001.

[26]

Wolff, M. Behavioral biometric identification on mobile devices. In Foundations of Augmented Cognition. Springer, 2013, 783--791.

[27]

Zoha, A., Gluhak, A., Imran, M. A., and Rajasegarar, S. Non-intrusive load monitoring approaches for disaggregated energy sensing: A survey. Sensors 12, 12 (2012), 16838--16866.

Cited By

Salehzadeh Niksirat KVelykoivanenko LZufferey NCherubini MHuguenin KHumbert M(2024)Wearable Activity Trackers: A Survey on Utility, Privacy, and SecurityACM Computing Surveys10.1145/364509156:7(1-40)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1145/3645091
Zlatintsi AFilntisis PEfthymiou NGaroufis CRetsinas GSounapoglou TMaglogiannis ITsanakas PSmyrnis NMaragos P(2024)Person Identification and Relapse Detection From Continuous Recordings of Biosignals Challenge: Overview and ResultsIEEE Open Journal of Signal Processing10.1109/OJSP.2024.33763005(641-651)Online publication date: 2024
https://doi.org/10.1109/OJSP.2024.3376300
Selvaraj VAl-Amin MYu XTao WMin S(2024)Real-time action localization of manual assembly operations using deep learning and augmented inference state machinesJournal of Manufacturing Systems10.1016/j.jmsy.2023.12.00772(504-518)Online publication date: Mar-2024
https://doi.org/10.1016/j.jmsy.2023.12.007
Show More Cited By

Index Terms

Object hallmarks: identifying object users using wearable wrist sensors
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Technical Function Evaluation of Two Smart Wearables and Data Analysis Methods for Step Counts
Augmented Cognition
Abstract
Smart wearable devices that capture physical activity data are increasingly used for health research and show potential for augmented cognition. These devices must be tested to understand their function before use in research and everyday life. ...
Sensor requirements for activity recognition on smart watches
PETRA '13: Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments

The new generation of watches is smart. Smart watches are connected to the internet and provide sensor functionality that allows an enhanced human-computer-interaction. Smart watches provide a gesture interaction and a permanent monitoring of physical ...
Fall detection and activity identification using wearable and hand-held devices

Human motion data captured from wearable devices such as smart watches can be utilized for activity recognition and emergency event detection, especially in the case of elderly or disabled people living independently in their homes. The output of such ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2015

1302 pages

ISBN:9781450335744

DOI:10.1145/2750858

General Chairs:
Kenji Mase
Nagoya University, JP
,
Marc Langheinrich
Università della Svizzera italiana (USI), CH
,
Daniel Gatica-Perez
IDIAP-EPFL, CH
,
Program Chairs:
Hans Gellersen
Lancaster University, UK
,
Tanzeem Choudhury
Cornell University
,
Koji Yatani
Univerisity of Tokyo, JP

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Yahoo! Japan: Yahoo! Japan
SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
FX Palo Alto Laboratory, Inc.: FX Palo Alto Laboratory, Inc.
Google Inc.
ACM: Association for Computing Machinery
Rakuten Institute of Technology: Rakuten Institute of Technology
Microsoft: Microsoft
Bell Labs: Bell Laboratories
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
KDDI
Panasonic: Panasonic Corporation
NTT DoCoMo
Telefónica: Telefónica
ISTC-PC: Intel Science and Technology Center for Pervasive Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

UbiComp '15

Sponsor:

Yahoo! Japan
SIGMOBILE
FX Palo Alto Laboratory, Inc.
ACM
Rakuten Institute of Technology
Microsoft
Bell Labs
SIGCHI
Panasonic
Telefónica
ISTC-PC

UbiComp '15: The 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 7 - 11, 2015

Osaka, Japan

Acceptance Rates

UbiComp '15 Paper Acceptance Rate 101 of 394 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

UBICOMP '24

Sponsor:
sigchi
sigchi

The 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 5 - 9, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
974
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Salehzadeh Niksirat KVelykoivanenko LZufferey NCherubini MHuguenin KHumbert M(2024)Wearable Activity Trackers: A Survey on Utility, Privacy, and SecurityACM Computing Surveys10.1145/364509156:7(1-40)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1145/3645091
Zlatintsi AFilntisis PEfthymiou NGaroufis CRetsinas GSounapoglou TMaglogiannis ITsanakas PSmyrnis NMaragos P(2024)Person Identification and Relapse Detection From Continuous Recordings of Biosignals Challenge: Overview and ResultsIEEE Open Journal of Signal Processing10.1109/OJSP.2024.33763005(641-651)Online publication date: 2024
https://doi.org/10.1109/OJSP.2024.3376300
Selvaraj VAl-Amin MYu XTao WMin S(2024)Real-time action localization of manual assembly operations using deep learning and augmented inference state machinesJournal of Manufacturing Systems10.1016/j.jmsy.2023.12.00772(504-518)Online publication date: Mar-2024
https://doi.org/10.1016/j.jmsy.2023.12.007
Zhao GZhou ZHuang YNayak AGong WZhou H(2023)ALSensing: Human Activity Recognition using WiFi based on Active Learning2023 IEEE Wireless Communications and Networking Conference (WCNC)10.1109/WCNC55385.2023.10119036(1-6)Online publication date: Mar-2023
https://doi.org/10.1109/WCNC55385.2023.10119036
Yu JKong HKong LYu JKong HKong L(2023)Gesture-Independent User Authentication Using WiFiWiFi signal-based user authentication10.1007/978-981-99-5914-3_3(33-56)Online publication date: 17-Oct-2023
https://doi.org/10.1007/978-981-99-5914-3_3
Yu JKong HKong LYu JKong HKong L(2023)Finger Gesture-Based Continuous User Authentication Using WiFiWiFi signal-based user authentication10.1007/978-981-99-5914-3_2(5-32)Online publication date: 17-Oct-2023
https://doi.org/10.1007/978-981-99-5914-3_2
Nassi BShams JRokach LElovici Y(2022)Virtual Breathalyzer: Towards the Detection of Intoxication Using Motion Sensors of Commercial Wearable DevicesSensors10.3390/s2209358022:9(3580)Online publication date: 8-May-2022
https://doi.org/10.3390/s22093580
Wu CLi XZuo FLuo LDu XDi JZeng Q(2022)Use It-No Need to Shake It!Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503226:3(1-25)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550322
Wu CLi XLuo LZeng QBulusu NAryafar EBalasubramanian ASong J(2022)G2AuthProceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services10.1145/3498361.3538941(84-98)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3498361.3538941
Sharp JWu CZeng Q(2022)Authentication for drone delivery through a novel way of using face biometricsProceedings of the 28th Annual International Conference on Mobile Computing And Networking10.1145/3495243.3560550(609-622)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.1145/3495243.3560550
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents