research-article

Public Access

Towards recognizing person-object interactions using a single wrist wearable device

Authors:

Juhi Ranjan,

Kamin WhitehouseAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Pages 722 - 731

https://doi.org/10.1145/2968219.2968279

Published: 12 September 2016 Publication History

PDF eReader

Abstract

Activity recognition (AR) is an important part of context-aware applications. In this paper, we focus on an indirect AR method: by sensing the objects that a person is using. Objects that provide functional utility to their user, also indicate the type of activity that their user is doing. For example, the use of a hair dryer indicates that its user is grooming their hair. In this paper, we discuss an approach to sense the objects that the person interacts with, using only a single wearable device on the wrist of the person. Wearable devices typically have an IMU sensor, which can sense several aspects of the person's hand gestures, such as acceleration, and orientation. We collect a dataset of 17 different object interaction gestures using 5 participants in a test home. We evaluate the object gestures using supervised and unsupervised machine learning approaches. Our study reveals that we can recognize object interactions with 83-91% accuracy in the supervised approach, and 58-66% accuracy in the unsupervised approach.

References

[1]

Android 2015. Android Wear Orientation Sensor. (2015). http://developer.android.com/reference/android/hardware/Sensor.html#TYPE_ROTATION_VECTOR.

Google Scholar

[2]

Ling Bao and Stephen S Intille. 2004. Activity recognition from user-annotated acceleration data. In International Conference on Pervasive Computing. Springer, 1--17.

Crossref

Google Scholar

[3]

Eugen Berlin, Jun Liu, Kristof Van Laerhoven, and Bernt Schiele. 2010. Coming to grips with the objects we grasp: detecting interactions with efficient wrist-worn sensors. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction. ACM, 57--64.

Digital Library

Google Scholar

[4]

Keng-hao Chang, Jeffrey Hightower, and Branislav Kveton. 2009. Inferring identity using accelerometers in television remote controls. In Pervasive Computing. Springer, 151--167.

Digital Library

Google Scholar

[5]

Yun Cheng, Kaifei Chen, Ben Zhang, Chieh-Jan Mike Liang, Xiaofan Jiang, and Feng Zhao. 2012. Accurate real-time occupant energy-footprinting in commercial buildings. In Proceedings of the Fourth ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings. ACM, 115--122.

Digital Library

Google Scholar

[6]

Manuel Fernández-Deado, Eva Cernadas, Senén Barro, and Dinani Amorim. 2014. Do We Need Hundreds of Classifiers to Solve Real World Classification Problems? The Journal of Machine Learning Research 15, 1 (2014), 3133--3181.

Digital Library

Google Scholar

[7]

Mark R Hodges and Martha E Pollack. 2007. An object-use fingerprint: The use of electronic sensors for human identification. In UbiComp 2007: Ubiquitous Computing. Springer, 289--303.

Digital Library

Google Scholar

[8]

Seungwoo Lee, Daye Ahn, Sukjun Lee, Rhan Ha, and Hojung Cha. 2014. Personalized energy auditor: Estimating personal electricity usage. In Pervasive Computing and Communications (PerCom), 2014 IEEE International Conference on. IEEE, 44--49.

Google Scholar

[9]

Matthai Philipose, Kenneth P Fishkin, Mike Perkowitz, Donald J Patterson, Dieter Fox, Henry Kautz, and Dirk Hahnel. 2004. Inferring activities from interactions with objects. Pervasive Computing, IEEE 3, 4 (2004), 50--57.

Digital Library

Google Scholar

[10]

Juhi Ranjan, Erin Griffiths, and Kamin Whitehouse. 2014. Discerning electrical and water usage by individuals in homes. In Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings. ACM, 20--29.

Digital Library

Google Scholar

[11]

Juhi Ranjan and Kamin Whitehouse. 2015a. Object hallmarks: Identifying object users using wearable wrist sensors. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 51--61.

Digital Library

Google Scholar

[12]

Juhi Ranjan and Kamin Whitehouse. 2015b. Rethinking the fusion of technology and clinical practices in functional behavior analysis for the elderly. In Human Behavior Understanding. Springer, 52--65.

Digital Library

Google Scholar

[13]

Matt Wolff. 2013. Behavioral Biometric Identification on Mobile Devices. In Foundations of Augmented Cognition. Springer, 783--791.

Google Scholar

Cited By

View all

Al-Saedi ABoeva VCasalicchio EExner P(2022)Context-Aware Edge-Based AI Models for Wireless Sensor Networks—An OverviewSensors10.3390/s2215554422:15(5544)Online publication date: 25-Jul-2022
https://doi.org/10.3390/s22155544
Sturgess JEberz SSluganovic IMartinovic I(2022)Inferring User Height and Improving Impersonation Attacks in Mobile Payments using a Smartwatch2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767287(775-780)Online publication date: 21-Mar-2022
https://doi.org/10.1109/PerComWorkshops53856.2022.9767287
Rosa SPatane ALu CTrigoni N(2019)Semantic Place Understanding for Human–Robot Coexistence—Toward Intelligent WorkplacesIEEE Transactions on Human-Machine Systems10.1109/THMS.2018.287507949:2(160-170)Online publication date: Apr-2019
https://doi.org/10.1109/THMS.2018.2875079
Show More Cited By

Index Terms

Towards recognizing person-object interactions using a single wrist wearable device
1. Human-centered computing

Recommendations

Viewpoint Integration for Hand-Based Recognition of Social Interactions from a First-Person View
ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Wearable devices are becoming part of everyday life, from first-person cameras (GoPro, Google Glass), to smart watches (Apple Watch), to activity trackers (FitBit). These devices are often equipped with advanced sensors that gather data about the wearer ...
Recognising activities of daily living from patterns of object use

To provide assistance and support to the elderly disabled and cognitively impaired, the recognition of their activities of daily living (ADL) must be accurate and precise with regards to the object use for the activity situations. Current ...
Evaluation of a Wrist-Based Wearable Fall Detection Method
Hybrid Artificial Intelligent Systems
Abstract
Fall detection represents an important issue when dealing with Ambient Assisted Living for the elder. The vast majority of fall detection approaches have been developed for healthy and relatively young people. Moreover, plenty of these approaches ...

Comments

Information & Contributors

Information

Published In

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

September 2016

1807 pages

ISBN:9781450344623

DOI:10.1145/2968219

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
433
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)16

Reflects downloads up to 22 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Al-Saedi ABoeva VCasalicchio EExner P(2022)Context-Aware Edge-Based AI Models for Wireless Sensor Networks—An OverviewSensors10.3390/s2215554422:15(5544)Online publication date: 25-Jul-2022
https://doi.org/10.3390/s22155544
Sturgess JEberz SSluganovic IMartinovic I(2022)Inferring User Height and Improving Impersonation Attacks in Mobile Payments using a Smartwatch2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767287(775-780)Online publication date: 21-Mar-2022
https://doi.org/10.1109/PerComWorkshops53856.2022.9767287
Rosa SPatane ALu CTrigoni N(2019)Semantic Place Understanding for Human–Robot Coexistence—Toward Intelligent WorkplacesIEEE Transactions on Human-Machine Systems10.1109/THMS.2018.287507949:2(160-170)Online publication date: Apr-2019
https://doi.org/10.1109/THMS.2018.2875079
Rosa SPatanè ALu XTrigoni N(2018)CommonSenseProceedings of the 1st International Workshop on Internet of People, Assistive Robots and Things10.1145/3215525.3215526(1-6)Online publication date: 10-Jun-2018
https://dl.acm.org/doi/10.1145/3215525.3215526
Fang ZYuan JMagnenat-Thalmann N(2018)Understanding Human-Object Interaction in RGB-D videos for Human Robot InteractionProceedings of Computer Graphics International 201810.1145/3208159.3208192(163-167)Online publication date: 11-Jun-2018
https://dl.acm.org/doi/10.1145/3208159.3208192
Faraci FPapandrea MPuiatti AAgustoni SGiulivi SDrApuzzo VGiordano SRighi FBarberis OThommen ERossini E(2018)AutoPlay: a smart toys-kit for an objective analysis of children ludic behavior and development2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA)10.1109/MeMeA.2018.8438636(1-6)Online publication date: Jun-2018
https://doi.org/10.1109/MeMeA.2018.8438636
Civitarese GBettini C(2017)Monitoring objects manipulations to detect abnormal behaviors2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)10.1109/PERCOMW.2017.7917594(388-393)Online publication date: Mar-2017
https://doi.org/10.1109/PERCOMW.2017.7917594

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Viewpoint Integration for Hand-Based Recognition of Social Interactions from a First-Person View

Recognising activities of daily living from patterns of object use

Evaluation of a Wrist-Based Wearable Fall Detection Method