MEMORIA: A Memory Enhancement and MOment RetrIeval Application at the LSC2024
Authors: 
Alexandre Gago, Bernardo Kaluza, Eva Bartolomeu, Josefa Pandeirada, Ricardo Ribeiro, António J. R. NevesAuthors Info & Claims
Alexandre Gago, Bernardo Kaluza, Eva Bartolomeu, Josefa Pandeirada, Ricardo Ribeiro, António J. R. NevesAuthors Info & ClaimsPages 99 - 104
Abstract
The emergence of lifelogging, recording, storing and organizing personal data, underscores the need for efficient digital systems. Lifelog Search Challenge (LSC) promotes a space for discussing the challenges in developing these systems. MEMORIA, in its third version for LSC'24, stands out mainly for its renewed user interface, which aims for more intuitive and accessible navigation, offering a better user experience. Adding more layers and sub-layers has improved event segmentation based on hierarchical events. Another new feature is searching using an image and redesigning the home page, which offers a general summary of the user's lifelogs. Event navigation has been simplified, making navigating on a single page possible. Enhanced personalized search capabilities are made possible by a new labels page that offers a consolidated view of the system's annotations. Also, the image retrieval page has been updated with filters and informative pop-ups. Using Large Language Models (LLMs) to annotate images and events is a significant development in progress that could provide richer and more contextual descriptions. An event classification feature was created to retrieve pertinent memories automatically. The new album feature allows users to organize and explore image collections, while event search will enable users to find more specific moments using text queries. These new features are expected to make MEMORIA a relevant tool in lifelogging, with innovative solutions to the complex challenge of capturing, organizing, and accessing digital memories.
References
[1]
Naushad Alam, Yvette Graham, and Cathal Gurrin. 2023. Memento 3.0: An Enhanced Lifelog Search Engine for LSC'23. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA, 41--46.
[2]
Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2022. Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 43--47.
[3]
Youngmin Baek, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. 2019. Character Region Awareness for Text Detection.
[4]
Cathal Gurrin, Graham Healy, Liting Zhou, Björn Þór Jónsson, Duc Tien Dang Nguyen, Jakub Lokoc, Luca Rossetto, Minh-Triet Tran, Wolfgang Hürst, Werner Bailer, and Klaus Schoeffmann. 2024. Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24. In Proc. International Conference on Multimedia Retrieval (ICMR'24) (Phuket, Thailand) (ICMR '24). New York, NY, USA.
[5]
Olivier Jeunehomme1 and Arnaud D'Argembeau. 2020. Event segmentation and the temporal compression of experience in episodic memory. Psychological Research 84 (2020), 481--490.
[6]
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. [n. d.]. Mistral 7B. ([n. d.]).
[7]
Ron Mokady, Amir Hertz, and Amit H. Bermano. 2021. ClipCap: CLIP Prefix for Image Captioning.
[8]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision.
[9]
Ricardo Ribeiro, Luísa Amaral, Wei Ye, Alina Trifan, António J. R. Neves, and Pedro Iglésias. 2023. MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA, 18--23.
[10]
Ricardo Ribeiro, Alina Trifan, and António J R Neves. 2022. Lifelog Retrieval From Daily Digital Data: Narrative Review. JMIR Mhealth Uhealth 10, 5 (2 May 2022), e30517.
[11]
Klaus Schoeffmann. 2023. LifeXplore at the Lifelog Search Challenge 2023. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA, 53--58.
[12]
Ly Duyen Tran, Binh Nguyen, Liting Zhou, and Cathal Gurrin. 2023. MyEachtra: Event-Based Interactive Lifelog Retrieval System for LSC'23. In Proceedings of the 6th Annual ACM Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA, 24--29.
[13]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.
[14]
Jialian Wu, Jianfeng Wang, Zhengyuan Yang, Zhe Gan, Zicheng Liu, Junsong Yuan, and Lijuan Wang. 2022. GRiT: A Generative Region-to-text Transformer for Object Understanding. arXiv preprint arXiv:2212.00280 (2022).
[15]
Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2022. Zero-Shot Video Question Answering via Frozen Bidirectional Language Models. arXiv:2206.08155 [cs.CV]
Index Terms
- MEMORIA: A Memory Enhancement and MOment RetrIeval Application at the LSC2024
Recommendations
MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search ChallengeThe continuous collection and storage of personal data, denoted Lifelogging, has gained popularity in recent years as a means of monitoring and improving personal health. One important aspect of lifelogging is the collection and analysis of image data, ...
MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022
LSC '22: Proceedings of the 5th Annual on Lifelog Search ChallengeResearch on retrieving data and analyzing lifelogs revealed to be a very complex task, and the interdisciplinary challenges to be tackled have boosted increasing attention from the scientific community in information retrieval and lifelogging. The ...
Impact of Blind Image Quality Assessment on the Retrieval of Lifelog Images
IMuR '22: Proceedings of the 2nd International Workshop on Interactive Multimedia RetrievalThe use of personal lifelogs can be beneficial to improve the quality of our life, as they can serve as tools for memory augmentation or for providing support to people with memory issues. In visual lifelogs, data are captured by cameras in the form of ...
Comments
Information & Contributors
Information
Published In
June 2024
128 pages
ISBN:9798400705502
DOI:10.1145/3643489
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
LSC '24
Sponsor:
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 47Total Downloads
- Downloads (Last 12 months)47
- Downloads (Last 6 weeks)47
Reflects downloads up to 26 Jul 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in