research-article

Daily activity recognition combining gaze motion and visual features

Authors:

Andreas DengelAuthors Info & Claims

UbiComp '14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication

Pages 1103 - 1111

https://doi.org/10.1145/2638728.2641691

Published: 13 September 2014 Publication History

Abstract

Recognition of user activities is a key issue for context-aware computing. We present a method for recognition of user daily activities using gaze motion features and image-based visual features. Gaze motion features dominate for inferring the user's egocentric context whereas image-based visual features dominate for recognition of the environments and the target objects. The experimental results show the fusion of those different type of features improves performance of user daily activity recognition.

References

[1]

Biedert, Ralf, Buscher, Georg, and Dengel, Andreas. The eyeBook: Using Eye Tracking to Enhance the Reading Experience. Informatik-Spektrum, 33, 3 (Sep 2009), 272--281.

[2]

Bolt, Richard A. Eyes at the interface. In Proceedings of the 1982 Conference on Human Factors in Computing Systems (1982), ACM, 360--362.

Digital Library

[3]

Bulling, Andreas, Ward, Jamie, Gellersen, Hans, and Töster, Gerhard. Eye movement analysis for activity recognition using electrooculography. IEEE transactions on pattern analysis and machine intelligence, 33, 4 (2011), 741--53.

Digital Library

[4]

Chang, Chih-Chung and Lin, Chih-Jen. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2 (2011), 27:1--27:27.

Digital Library

[5]

Csurka, Gabriella and Dance, C. Visual categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision, ECCV (2004), 1--22.

[6]

Fathi, Alireza, Li, Yin, and Rehg, James M. Learning to Recognize Daily Actions Using Gaze. In Proceedings of the 12th European Conference on Computer Vision - Volume Part I (2012), Springer-Verlag, 314--327.

Digital Library

[7]

Hipiny, Irwandi Mohamad and Mayol-Cuevas, Walterio. Recognising Egocentric Activities from Gaze Regions with Multiple-Voting Bag of Words. CSTR-12-003 (2012), 1--15.

[8]

Hong, Jong-yi, Suh, Eui-ho, and Kim, Sung-Jin. Context-aware systems: A literature review and classification. Expert Systems with Applications, 36, 4 (May 2009), 8509--8522.

Digital Library

[9]

Iqbal, Shamsi T and Bailey, Brian P. Using Eye Gaze Patterns to Identify User Tasks. In: The Grace Hopper Celebration of Women in Computing (2004).

[10]

Moghimi, Mohammad, Azagra, Pablo, Montesano, Luis et al. Experiments on an RGB-D Wearable Vision System for Egocentric Activity Recognition. CVPR Workshop on Egocentric (First-person) Vision (2014).

[11]

Nowak, Eric. Sampling Strategies for Bag-of-Features. Proceedings of the 9th European Conference on Computer Vision - Volume Part IV (2006), 490--503.

Digital Library

[12]

Ogaki, Keisuke, Kitani, Kris M., Sugano, Yusuke, and Sato, Yoichi. Coupling eye-motion and ego-motion features for first-person activity recognition. 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Jun 2012), 1--7.

[13]

Sukthankar, R. PCA-SIFT: a more distinctive representation for local image descriptors. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., 2 (2004), 506--513.

Digital Library

[14]

Toyama, Takumi, Kieninger, Thomas, Shafait, Faisal, and Dengel, Andreas. Gaze guided object recognition using a head-mounted eye tracker. In Proc. of the Symposium on Eye Tracking Research and Applications (New York, NY, USA 2012), ACM, 91--98.

Digital Library

[15]

Wu, TF, Lin, CJ, and Weng, RC. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research, 5 (2004), 975--1005.

Digital Library

Cited By

Stiber MBohus DAndrist S(2024)"Uh, This One?": Leveraging Behavioral Signals for Detecting Confusion during Physical TasksProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685727(194-203)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685727
Barz MSonntag D(2021)Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human GazeSensors10.3390/s2112414321:12(4143)Online publication date: 16-Jun-2021
https://doi.org/10.3390/s21124143
Cartas ARadeva PDimiccoli M(2020)Activities of Daily Living Monitoring via a Wearable Camera: Toward Real-World ApplicationsIEEE Access10.1109/ACCESS.2020.29903338(77344-77363)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2990333
Show More Cited By

Index Terms

Daily activity recognition combining gaze motion and visual features
1. Human-centered computing

Recommendations

Combining Low and Mid-Level Gaze Features for Desktop Activity Recognition

Human activity recognition (HAR) is an important research area due to its potential for building context-aware interactive systems. Though movement-based activity recognition is an established area of research, recognising sedentary activities remains an ...
Visual features for ego-centric activity recognition: a survey
WearSys '18: Proceedings of the 4th ACM Workshop on Wearable Systems and Applications

Wearable cameras, which are becoming common mobile sensing platforms to capture the environment surrounding a person, can also be used to infer activities of the wearer. In this paper we critically discuss features for ego-centric activity recognition ...
Complex activity recognition using context-driven activity theory and activity signatures

In pervasive and ubiquitous computing systems, human activity recognition has immense potential in a large number of application domains. Current activity recognition techniques (i) do not handle variations in sequence, concurrency and interleaving of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication

September 2014

1409 pages

ISBN:9781450330473

DOI:10.1145/2638728

General Chairs:
AJ Brush
Microsoft Research
,
Adrian Friday
Lancaster University, UK
,
Program Chairs:
Julie Kientz
University of Washington
,
James Scott
Microsoft Research, UK
,
Junehwa Song
Korea Advanced Institute of Science and Technology (KAIST), KR

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '14

Sponsor:

UbiComp '14: The 2014 ACM Conference on Ubiquitous Computing

September 13 - 17, 2014

Washington, Seattle

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
272
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stiber MBohus DAndrist S(2024)"Uh, This One?": Leveraging Behavioral Signals for Detecting Confusion during Physical TasksProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685727(194-203)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685727
Barz MSonntag D(2021)Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human GazeSensors10.3390/s2112414321:12(4143)Online publication date: 16-Jun-2021
https://doi.org/10.3390/s21124143
Cartas ARadeva PDimiccoli M(2020)Activities of Daily Living Monitoring via a Wearable Camera: Toward Real-World ApplicationsIEEE Access10.1109/ACCESS.2020.29903338(77344-77363)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2990333
Steil JHuang MBulling ASharif BKrejtz K(2018)Fixation detection for head-mounted eye tracking based on visual similarity of gaze targetsProceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications10.1145/3204493.3204538(1-9)Online publication date: 14-Jun-2018
https://dl.acm.org/doi/10.1145/3204493.3204538
Augereau OSanches CKise KKunze K(2018)Wordometer Systems for Everyday LifeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31616011:4(1-21)Online publication date: 8-Jan-2018
https://dl.acm.org/doi/10.1145/3161601
Nguyen TNebel JFlorez-Revuelta F(2016)Recognition of Activities of Daily Living with Egocentric Vision: A ReviewSensors10.3390/s1601007216:1(72)Online publication date: 7-Jan-2016
https://doi.org/10.3390/s16010072
Adiba ATanaka NMiyake J(2016)An Adjustable Gaze Tracking System and Its Application for Automatic Discrimination of Interest ObjectsIEEE/ASME Transactions on Mechatronics10.1109/TMECH.2015.247052221:2(973-979)Online publication date: Apr-2016
https://doi.org/10.1109/TMECH.2015.2470522
Steil JBulling AMase KLangheinrich MGatica-Perez DGellersen HChoudhury TYatani K(2015)Discovery of everyday human activities from long-term visual behaviour using topic modelsProceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/2750858.2807520(75-85)Online publication date: 7-Sep-2015
https://dl.acm.org/doi/10.1145/2750858.2807520
Braunagel CKasneci EStolzmann WRosenstiel W(2015)Driver-Activity Recognition in the Context of Conditionally Autonomous DrivingProceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems10.1109/ITSC.2015.268(1652-1657)Online publication date: 15-Sep-2015
https://dl.acm.org/doi/10.1109/ITSC.2015.268

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten