Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3643489.3661115acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
research-article
Open access

EAGLE: Eyegaze-Assisted Guidance and Learning Evaluation for Lifeloging Retrieval

Published: 18 June 2024 Publication History

Abstract

In this work, we focus on bridging the gap between how expert users and novice users use lifelog information retrieval systems. Our proposed system aims to achieve two main objectives: 1) incorporating implicit interactions such as eye movements into the search process, and 2) using an automated search flow to support novice and experienced users based on a minimal effort principle. Based on user interactions with the search system, we propose an algorithm that rearranges and modifies how information is displayed and organized based on user eye movements, which aims to optimize display results. Our results on previous LSC topics show that leveraging user input data in the search process is beneficial and profitable that future human-centric systems should exploit for improved performance.

References

[1]
Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2023. Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 7--12.
[2]
Ralf Biedert, Georg Buscher, and Andreas R. Dengel. 2010. The eyeBook - Using Eye Tracking to Enhance the Reading Experience. Informatik-Spektrum 33 (2010), 272--281.
[3]
Tong Cha and Sebastian Maier. 2012. Eye gaze assisted human-computer interaction in a hand gesture controlled multi-display environment. In Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction (Santa Monica, California) (Gaze-In '12). Association for Computing Machinery, New York, NY, USA, Article 13, 3 pages.
[4]
Cathal Gurrin, Liting Zhou, Graham Healy, Bailer. Werner, Duc-Tien Dang-Nguyen, Steve Hodges, Björn Þór Jónsson, Jakub Lokoč, Luca Rossetto, Minh-Triet Tran, and Klaus Schöffmann. 2024. Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24. International Conference on Multimedia Retrieval (ICMR'24).
[5]
Nhat Hoang-Xuan, Thang-Long Nguyen-Ho, Cathal Gurrin, and Minh-Triet Tran. 2023. Lifelog Discovery Assistant: Suggesting Prompts and Indexing Event Sequences for FIRST at LSC 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 47--52.
[6]
Maria Tysse Hordvik, Julie Sophie Teilstad Østby, Manoj Kesavulu, Thao-Nhu Nguyen, Tu-Khiem Le, and Duc-Tien Dang-Nguyen. 2023. LifeLens: Transforming Lifelog Search with Innovative UX/UI Design. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 1--6.
[7]
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv:2301.12597 [cs.CV]
[8]
Bhanuka Mahanama, Yasith Jayawardana, Sundararaman Rengarajan, Gavindya Jayawardena, Leanne Chukoskie, Joseph Snider, and Sampath Jayarathna. [n. d.]. Eye Movement and Pupil Measures: A Review. Frontiers in Computer Science 3 ([n. d.]).
[9]
Thao-Nhu Nguyen, Tu-Khiem Le, Van-Tu Ninh, Cathal Gurrin, Minh-Triet Tran, Thanh Binh Nguyen, Graham Healy, Annalina Caputo, and Sinead Smyth. 2023. E-LifeSeeker: An interactive lifelog search engine for lsc'23. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 13--17.
[10]
Thang-Long Nguyen-Ho, Tien-Thanh Nguyen-Dang, Gia-Huy Vuong, Van-Son Ho, Xuan-Dang Thai, Minh-Khoi Pham, Tu-Khiem Le, Van-Tu Ninh, and Minh-Triet Tran. 2023. Automatic Sub-Task Focus: LifeInsight's Contribution to NTCIR-17 Lifelog-5. (No Title) (2023), none.
[11]
Van-Tu Ninh, Tu-Khiem Le, Liting Zhou, Luca Piras, Michael Riegler, Pål Halvorsen, Minh-Triet Tran, Mathias Lux, Cathal Gurrin, and Duc-Tien Dang-Nguyen. 2020. Overview of ImageCLEF Lifelog 2020:Lifelog Moment Retrieval and Sport Performance Lifelog. In CLEF2020 Working Notes (CEUR Workshop Proceedings). CEUR-WS.org <http://ceur-ws.org>, Thessaloniki, Greece.
[12]
Georgios Th. Papadopoulos, Konstantinos C. Apostolakis, and Petros Daras. 2014. Gaze-Based Relevance Feedback for Realizing Region-Based Image Retrieval. IEEE Transactions on Multimedia 16, 2 (2014), 440--454.
[13]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. CoRR abs/2103.00020 (2021).
[14]
Yashas Rai and Patrick Le Callet. 2018. Chapter 3 - Visual attention, visual salience, and perceived interest in multimedia applications. In Academic Press Library in Signal Processing, Volume 6, Rama Chellappa and Sergios Theodoridis (Eds.). Academic Press, 113--161.
[15]
Ricardo Ribeiro, Luísa Amaral, Wei Ye, Alina Trifan, António JR Neves, and Pedro Iglésias. 2023. MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 18--23.
[16]
Klaus Schoeffmann. 2023. lifexplore at the lifelog search challenge 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 53--58.
[17]
Thomas Serre. 2013. Hierarchical Models of the Visual System. Springer New York, New York, NY, 1--12.
[18]
Ly-Duyen Tran, Manh-Duy Nguyen, Binh Nguyen, Hyowon Lee, Liting Zhou, and Cathal Gurrin. 2022. E-Myscéal: Embedding-based Interactive Lifelog Retrieval System for LSC'22. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 32--37.
[19]
Quang-Linh Tran, Ly-Duyen Tran, Binh Nguyen, and Cathal Gurrin. 2023. MemoriEase: An Interactive Lifelog Retrieval System for LSC'23. In Proceedings of the 6th Annual ACM Lifelog Search Challenge. 30--35.
[20]
Tom Tullis and Bill Albert. 2013. Chapter 7 - Behavioral and Physiological Metrics. In Measuring the User Experience (Second Edition) (second edition ed.), Tom Tullis and Bill Albert (Eds.). Morgan Kaufmann, Boston, 163--186.
[21]
Zhou, Liting and Gurrin, Cathal and Dang-Nguyen, Duc-Tien and Healy, Graham, and Lyu, ChenYang and Ji, Tianbo and Wang, Longyue and Hideo Joho and Tran, Ly-Duyen and Naushad Alam. 2023. Overview of the NTCIR-17 Lifelog-5 Task. In Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies (NTCIR-17). Tokyo, Japan.

Cited By

View all
  • (2024)Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24Proceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658891(1334-1335)Online publication date: 30-May-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge
June 2024
128 pages
ISBN:9798400705502
DOI:10.1145/3643489
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2024

Check for updates

Author Tags

  1. lifelog
  2. retrieval systems
  3. eye-tracking
  4. semantic embedding
  5. vector database

Qualifiers

  • Research-article

Funding Sources

Conference

LSC '24
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)51
  • Downloads (Last 6 weeks)28
Reflects downloads up to 01 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24Proceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658891(1334-1335)Online publication date: 30-May-2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media