research-article

Searching Temporally Distant Activities in Lifelog Data With PraK Tool V2

Authors:

Zuzana Vopálková,

Jakub LokocAuthors Info & Claims

LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge

Pages 111 - 116

https://doi.org/10.1145/3643489.3661131

Published: 18 June 2024 Publication History

Abstract

We present a new version of our interactive cross-modal search system PraK that was updated for the needs of lifelog search challenge data and tasks. The architecture of PraK allows to hide lifelog data specific details behind already existing interface of video data services. Hence, the already existing application service designed for the Video Browser Showdown could be tested for the visual part of the lifelog dataset. Nevertheless, we add more features to address search challenges specific for lifelog data. In the new version, the interface of the services is slightly updated to incorporate meta-data based filtering as well as to support temporally distant activity queries. Another major update of the system is shift of state management from the web frontend to the application backend, which leads to more centralized architecture but allows to deploy PraK components in a more efficient way.

References

[1]

Naushad Alam, Yvette Graham, and Cathal Gurrin. 2023. Memento 3.0: An Enhanced Lifelog Search Engine for LSC'23. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[2]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2023. Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[3]

Jim Gemmell, Gordon Bell, and Roger Lueder. 2006. MyLifeBits: A Personal Database for Everything. Commun. ACM 49, 1 (Jan 2006), 88--95.

Digital Library

[4]

Cathal Gurrin, Hideo Joho, Frank Hopfgartner, Liting Zhou, Van-Tu Ninh, Tu-Khiem Le, Rami Albatal, Duc-Tien Dang-Nguyen, and Graham Healy. 2019. Advances in Lifelog Data Organisation and Retrieval at the NTCIR-14 Lifelog-3 Task. In NII Testbeds and Community for Information Access Research, Makoto P Kato, Yiqun Liu, Noriko Kando, and Charles L A Clarke (Eds.). Springer International Publishing, Cham, 16--28.

[5]

Cathal Gurrin, Björn Þór Jónsson, Duc-Tien Dang-Nguyen, Graham Healy, Jakub Lokoč, Liting Zhou, Luca Rossetto, Minh-Triet Tran, Wolfgang Hürst, Werner Bailer, and Klaus Schoeffmann. 2023. Introduction to the Sixth Annual Lifelog Search Challenge, LSC'23. In Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, ICMR 2023, Thessaloniki, Greece, June 12-15, 2023. ACM, 678--679.

Digital Library

[6]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, et al. 2019. Comparing approaches to interactive lifelog search at the lifelog search challenge (LSC2018). ITE Transactions on Media Technology and Applications 7, 2 (2019), 46--59.

[7]

Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Foundations and Trends® in Information Retrieval 8, 1 (2014), 1--125.

Digital Library

[8]

Cathal Gurrin, Liting Zhou, Graham Healy, Bailer. Werner, Duc-Tien Dang-Nguyen, Steve Hodges, Björn Þór Jónsson, Jakub Lokoč, Luca Rossetto, Minh-Triet Tran, and Klaus Schöffmann. 2024. Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24. International Conference on Multimedia Retrieval (ICMR'24).

Digital Library

[9]

Silvan Heller. 2023. Temporal Multimodal Video and Lifelog Retrieval. PhD Thesis. https://dbis.dmi.unibas.ch/publications/2023/PhD-SilvanHeller-23/.

[10]

Nhat Hoang-Xuan, Thang-Long Nguyen-Ho, Cathal Gurrin, and Minh-Triet Tran. 2023. Lifelog Discovery Assistant: Suggesting Prompts and Indexing Event Sequences for FIRST at LSC 2023. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[11]

Maria Tysse Hordvik, Julie Sophie Teilstad Østby, Tu-Khiem Le Manoj Kesavulu, Thao-Nhu Nguyen, and Duc-Tien Dang-Nguyen. 2023. LifeLens: Transforming Lifelog Search with Innovative UX/UI Design. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[12]

Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. 2021. OpenCLIP.

[13]

Miroslav Kratochvíl, František Mejzlík, Patrik Veselý, Tomáš Souček, and Jakub Lokoč. 2020. SOMHunter: Lightweight Video Search System with SOM-Guided Relevance Feedback. In MM '20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020, Chang Wen Chen, Rita Cucchiara, Xian-Sheng Hua, Guo-Jun Qi, Elisa Ricci, Zhengyou Zhang, and Roger Zimmermann (Eds.). ACM, 4481--4484.

Digital Library

[14]

Jakub Lokoč, František Mejzlík, Tomáš Souček, Patrik Dokoupil, and Ladislav Peška. 2022. Video Search with Context-Aware Ranker and Relevance Feedback. In MultiMedia Modeling. Springer International Publishing, Cham, 505--510.

[15]

Jakub Lokoč, Stelios Andreadis, Werner Bailer, Aaron Duane, Cathal Gurrin, Zhixin Ma, Nicola Messina, Thao-Nhu Nguyen, Ladislav Peška, Luca Rossetto, Loris Sauter, Konstantin Schall, Klaus Schoeffmann, Omar Shahbaz Khan, Florian Spiess, Lucia Vadicamo, and Stefanos Vrochidis. 2023. Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS. Multim. Syst. 29, 6 (2023), 3481--3504.

Digital Library

[16]

Jakub Lokoč, Gregor Kovalčík, Tomáš Souček, Jaroslav Moravec, and Přemysl Čech. 2019. A Framework for Effective Known-item Search in Video. In Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21-25, 2019, Laurent Amsaleg, Benoit Huet, Martha A. Larson, Guillaume Gravier, Hayley Hung, Chong-Wah Ngo, and Wei Tsang Ooi (Eds.). ACM, 1777--1785.

Digital Library

[17]

Jakub Lokoč and Ladislav Peška. 2023. A Study of a Cross-modal Interactive Search Tool Using CLIP and Temporal Fusion. In MultiMedia Modeling - 29th International Conference, MMM 2023, Bergen, Norway, January 9-12, 2023, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 13833), Duc-Tien Dang-Nguyen, Cathal Gurrin, Martha A. Larson, Alan F. Smeaton, Stevan Rudinac, Minh-Son Dao, Christoph Trattner, and Phoebe Chen (Eds.). Springer, 397--408.

Digital Library

[18]

Jakub Lokoč, Zuzana Vopálková, Michael Stroh, Raphael Buchmueller, and Udo Schlegel. 2024. PraK Tool: An Interactive Search Tool Based on Video Data Services. In MultiMedia Modeling - 30th International Conference, MMM 2024, Amsterdam, The Netherlands, January 29 - February 2, 2024, Proceedings, Part IV (Lecture Notes in Computer Science, Vol. 14557), Stevan Rudinac, Alan Hanjalic, Cynthia C. S. Liem, Marcel Worring, Björn Þór Jónsson, Bei Liu, and Yoko Yamakata (Eds.). Springer, 340--346.

Digital Library

[19]

Thao-Nhu Nguyen, Tu-Khiem Le, Van-Tu Ninh, Cathal Gurrin, Minh-Triet Tran, Thanh Binh Nguyen, Graham Healy, Annalina Caputo, and Sinead Smyth. 2023. E-LifeSeeker: An Interactive Lifelog Search Engine for LSC'23. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[20]

Tien-Thanh Nguyen-Dang, Xuan-Dang Thai, Gia-Huy Vuong, Van-Son Ho, Minh-Triet Tran, Van-Tu Ninh, Minh-Khoi Pham, Tu-Khiem Le, and Graham Healy. 2023. LifeInsight: An Interactive Lifelog Retrieval System with Comprehensive Spatial Insights and Query Assistance. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[21]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748--8763. http://proceedings.mlr.press/v139/radford21a.html

[22]

Ricardo Ribeiro, Luísa Amaral, Wei Ye, Alina Trifan, and António J. R. Neves. 2023. MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[23]

Luca Rossetto, Ralph Gasser, Loris Sauter, Abraham Bernstein, and Heiko Schuldt. 2021. A System for Interactive Multimedia Retrieval Evaluations. In International Conference on Multimedia Modeling. Springer, 385--390.

Digital Library

[24]

Luca Rossetto, Oana Inel, Svenja Lange, Florian Ruosch, Ruijie Wang, and Abraham Bernstein. 2023. Multi-Mode Clustering for Graph-Based Lifelog Retrieval. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[25]

Klaus Schoeffmann. 2023. lifeXplore at the Lifelog Search Challenge 2023. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[26]

Florian Spiess, Ralph Gasser, Heiko Schuldt, and Luca Rossetto. 2023. The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid System. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[27]

Ly Duyen Tran, Binh Nguyen, Liting Zhou, and Cathal Gurrin. 2023. MyEachtra: Event-based Interactive Lifelog Retrieval System for LSC'23. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[28]

Ly-Duyen Tran, Manh-Duy Nguyen, Duc-Tien Dang-Nguyen, Silvan Heller, Florian Spiess, Jakub Lokoč, Ladislav Peška, Thao-Nhu Nguyen, Omar Shahbaz Khan, Aaron Duane, Björn þór Jónsson, Luca Rossetto, An-Zi Yen, Ahmed Alateeq, Naushad Alam, Minh-Triet Tran, Graham Healy, Klaus Schoeffmann, and Cathal Gurrin. 2023. Comparing Interactive Retrieval Approaches at the Lifelog Search Challenge 2021. IEEE Access 11 (2023), 30982--30995.

[29]

Ly-Duyen Tran, Manh-Duy Nguyen, Binh Nguyen, Hyowon Lee, Liting Zhou, and Cathal Gurrin. 2022. E-Myscéal: Embedding-Based Interactive Lifelog Retrieval System for LSC'22. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 32--37.

Digital Library

[30]

Quang-Linh Tran, Ly-Duyen Tran, Binh Nguyen, and Cathal Gurrin. 2023. MemoriEase: An Interactive Lifelog Retrieval System for LSC'23. In Proceedings of the 6th Annual on Lifelog Search Challenge (Thessaloniki, Greece) (LSC '23). Association for Computing Machinery, New York, NY, USA.

Digital Library

Cited By

Gurrin CZhou LHealy GBailer WDang Nguyen DHodges SJónsson BLokoč JRossetto LTran MSchöffmann KGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24Proceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658891(1334-1335)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658891

Index Terms

Searching Temporally Distant Activities in Lifelog Data With PraK Tool V2
1. Human-centered computing
  1. Interaction design
    1. Empirical studies in interaction design
2. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Search interfaces

Recommendations

Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22
ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval

For the fifth time since 2018, the Lifelog Search Challenge (LSC) facilitated a benchmarking exercise to compare interactive search systems designed for multimodal lifelogs. LSC'22 attracted nine participating research groups who developed interactive ...
Introduction to the Sixth Annual Lifelog Search Challenge, LSC’23
ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

For the sixth time since 2018, the Lifelog Search Challenge (LSC) was organized as a comparative benchmarking exercise for various interactive lifelog search systems. The goal of this international competition is to test system capabilities to access ...
Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24
ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval

For the seventh time since 2018, the Lifelog Search Challenge (LSC) benchmarked interactive lifelog search systems in a live challenge. The LSC goal is to comparatively evaluate system capabilities to access large multimodal lifelogs comprising hundreds ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge

June 2024

128 pages

ISBN:9798400705502

DOI:10.1145/3643489

Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

GACR

Conference

LSC '24

Sponsor:

SIGMM

LSC '24: 7th Annual ACM Workshop on the Lifelog Search Challenge

June 10, 2024

Phuket, Thailand

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
25
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)3

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gurrin CZhou LHealy GBailer WDang Nguyen DHodges SJónsson BLokoč JRossetto LTran MSchöffmann KGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24Proceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658891(1334-1335)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658891

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents