research-article

Open access

Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs

Authors:

Cathal GurrinAuthors Info & Claims

LSC '23: Proceedings of the 6th Annual ACM Lifelog Search Challenge

Pages 7 - 12

https://doi.org/10.1145/3592573.3593097

Published: 12 June 2023 Publication History

All formats PDF

Abstract

In this paper, we introduce Voxento 4.0 – an interactive voice-based retrieval system for lifelogs which has been developed to participate in the sixth Lifelog Search Challenge LSC’23, at ACM ICMR’23. Voxento has participated three times in the LSC editions and achieved the rank of 4th in LSC21 and 5th in LSC22 respectively. In this version, Voxento 4.0, we have focused on improving the previous system’s interface, voice interaction and retrieval functionality. The current version has implemented some processing and cleaning of the dataset and employs the CLIP model to extract image features. In addition, the system’s interface was redesigned for better visualisation of the elements and the images for effective interaction. This improvement in the interface will help to support voice interaction in future work. The interface developments include logging voice interaction and images displayed, submitted, selected and starred to enhance user experience with the system. The voice interaction part has also been enhanced in the workflow of the voice lifecycle interaction and with additional voice commands.

References

[1]

Naushad Alam, Yvette Graham, and Cathal Gurrin. 2022. Memento 2.0: An Improved Lifelog Search Engine for LSC’22. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22). Association for Computing Machinery, New York, NY, USA, 2–7. https://doi.org/10.1145/3512729.3533006

Digital Library

[2]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2020. Voxento: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC’20) (Dublin, Ireland). Association for Computing Machinery, New York, NY, USA, 77–81. https://doi.org/10.1145/3379172.3391728

Digital Library

[3]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2021. Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 65–70. https://doi.org/10.1145/3463948.3469071

Digital Library

[4]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2022. Voxento 3.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22), Vol. 1. Association for Computing Machinery, New York, NY, USA, 43–47. https://doi.org/10.1145/3463948.3469071

Digital Library

[5]

Aaron Duane and Bjorn THORNór Jónsson. 2021. ViRMA: Virtual Reality Multimedia Analytics at LSC 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 29–34. https://doi.org/10.1145/3463948.3469067

Digital Library

[6]

Jim Gemmell, Gordon Bell, and Roger Lueder. 2006. MyLifeBits: A personal database for everything. Commun. ACM 49, 1 (2006), 88–95. https://doi.org/10.1145/1107458.1107460

Digital Library

[7]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc Tien Dang Nguyen, Michael Riegler, Luca Piras, Minh-Triet Tran, Jakub Lokoč, and Wolfgang Hürst. 2019. [Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018). ITE Transactions on Media Technology and Applications 7 (04 2019), 46–59. https://doi.org/10.3169/mta.7.46

[8]

Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal big data. Vol. 8. Now Publishers. 1–125 pages. https://doi.org/10.1561/1500000033

Digital Library

[9]

Cathal Gurrin, Liting Zhou, Graham Healy, Björn Þór Jónsson, Duc-Tien Dang-Nguyen, Jakub Lokoć, Minh-Triet Tran, Wolfgang Hürst, Luca Rossetto, and Klaus Schöffmann. 2022. Introduction to the Fifth Annual Lifelog Search Challenge, LSC’22. In Proceedings of the 2022 International Conference on Multimedia Retrieval (Newark, NJ, USA) (ICMR ’22). Association for Computing Machinery, New York, NY, USA, 685–687. https://doi.org/10.1145/3512527.3531439

Digital Library

[10]

Cathal Gurrin, Björn Por Þór Jónsson, Duc-Tien Dang-Nguyen, Graham Healy, Jakub Lokoč, Liting Zhou, Luca Rossetto, Minh-Triet Tran, Wolfgang Hürst, and Klaus Schöffmann. 2023. Introduction to the Sixth Annual Lifelog Search Challenge, LSC’23. In Proc. International Conference on Multimedia Retrieval (ICMR’23) (Thessaloniki, Greece) (ICMR ’23). Association for Computing Machinery, New York, NY, USA.

Digital Library

[11]

Silvan Heller, Luca Rossetto, Loris Sauter, and Heiko Schuldt. 2022. vitrivr at the Lifelog Search Challenge 2022. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22). Association for Computing Machinery, New York, NY, USA, 27–31. https://doi.org/10.1145/3512729.3533003

Digital Library

[12]

Nhat Hoang-Xuan, Hoang-Phuc Trang-Trung, E-Ro Nguyen, Thanh-Cong Le, Mai-Khiem Tran, Tu-Khiem Le, Van-Tu Ninh, Cathal Gurrin, and Minh-Triet Tran. 2022. Flexible Interactive Retrieval SysTem 3.0 for Visual Lifelog Exploration at LSC 2022. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22). Association for Computing Machinery, New York, NY, USA, 20–26. https://doi.org/10.1145/3512729.3533013

Digital Library

[13]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data 7, 3 (2019), 535–547.

[14]

Emil Knudsen, Thomas Holstein Qvortrup, Omar Shahbaz Khan, and Björn THORNór Jónsson. 2021. XQC at the Lifelog Search Challenge 2021: Interactive Learning on a Mobile Device. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 89–93. https://doi.org/10.1145/3463948.3469063

Digital Library

[15]

Andreas Leibetseder, Daniela Stefanics, and Klaus Schoeffmann. 2022. lifeXplore at the Lifelog Search Challenge 2022. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22). Association for Computing Machinery, New York, NY, USA, 48–52. https://doi.org/10.1145/3512729.3533005

Digital Library

[16]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arxiv:2103.00020http://arxiv.org/abs/2103.00020

[17]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. (2022). arxiv:2212.04356https://github.com/openai http://arxiv.org/abs/2212.04356

[18]

Florian Spiess and Heiko Schuldt. 2022. Multimodal Interactive Lifelog Retrieval with vitrivr-VR. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22), Vol. 1. Association for Computing Machinery, New York, NY, USA, 38–42. https://doi.org/10.1145/3512729.3533008

Digital Library

[19]

Ly-Duyen Tran, Manh-Duy Nguyen, Binh Nguyen, Hyowon Lee, Liting Zhou, and Cathal Gurrin. 2022. E-Myscéal: Embedding-based Interactive Lifelog Retrieval System for LSC’22. In Proceedings of the 5th Annual Lifelog Search Challenge (LSC ’22). Association for Computing Machinery, New York, NY, USA, 32–37. https://doi.org/10.1145/3512729.3533012

Digital Library

Cited By

Vopálková ZYaghob JStroh MSchlegel ULokoc J(2024)Searching Temporally Distant Activities in Lifelog Data With PraK Tool V2Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661131(111-116)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661131
Alateeq ARoantree MGurrin C(2024)Voxento-Pro: An Advanced Voice Lifelog Retrieval Interaction for Multimodal LifelogsProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661130(105-110)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661130
Tran LNguyen TGurrin CZhou L(2024)MyEachtraX: Lifelog Question Answering on MobileProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661128(93-98)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661128
Show More Cited By

Index Terms

Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs

Recommendations

Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog
LSC '22: Proceedings of the 5th Annual on Lifelog Search Challenge

Voxento is an interactive voice-based retrieval system for lifelogs which has been redeveloped and optimised to participate in the fifth Lifelog Search Challenge LSC'22, at ACM ICMR'22. Based on the previous experience in the LSC competition and ranked ...
Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs
LSC '21: Proceedings of the 4th Annual on Lifelog Search Challenge

In this paper, we describe an extended version of Voxento which is an interactive voice-based retrieval system for lifelogs that has been developed to participate in the fourth Lifelog Search Challenge LSC'21, at ACM ICMR'21. Voxento provides a spoken ...
Voxento: A Prototype Voice-controlled Interactive Search Engine for Lifelogs
LSC '20: Proceedings of the Third Annual Workshop on Lifelog Search Challenge

In this paper, we describe an interactive voice-based retrieval system for lifelogs which has been developed to participate in the third Lifelog Search Challenge LSC'20, at ACM ICMR'20. Based on a standard text-based retrieval methodology, the novelty ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LSC '23: Proceedings of the 6th Annual ACM Lifelog Search Challenge

June 2023

74 pages

ISBN:9798400701887

DOI:10.1145/3592573

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

The Insight Centre for Data Analytics
The Ministry of Education in Saudi Arabia

Conference

ICMR '23

Sponsor:

SIGMM

ICMR '23: International Conference on Multimedia Retrieval

June 12 - 15, 2023

Thessaloniki, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
285
Total Downloads

Downloads (Last 12 months)262
Downloads (Last 6 weeks)38

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vopálková ZYaghob JStroh MSchlegel ULokoc J(2024)Searching Temporally Distant Activities in Lifelog Data With PraK Tool V2Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661131(111-116)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661131
Alateeq ARoantree MGurrin C(2024)Voxento-Pro: An Advanced Voice Lifelog Retrieval Interaction for Multimodal LifelogsProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661130(105-110)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661130
Tran LNguyen TGurrin CZhou L(2024)MyEachtraX: Lifelog Question Answering on MobileProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661128(93-98)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661128
Alam NGraham YGurrin C(2024)Memento 4.0: A Prototype Conversational Search System for LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661126(82-87)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661126
Hezel NSchall KSchilling BJung KBarthel K(2024)Libro - Lifelog Search BrowserProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661124(70-75)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661124
Rader MSchoeffmann K(2024)lifeXplore at the Lifelog Search Challenge 2024Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661123(64-69)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661123
Le HNguyen TLe TTran MNguyen TNinh VZhou LGurrin C(2024)LifeSeeker 6.0: Leveraging the linguistic aspect of the lifelog system in LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661121(53-57)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661121
Sauter LGasser RRettig LSchuldt HRossetto L(2024)General Purpose Multimedia Retrieval with vitrivr at LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661120(47-52)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661120
Steffensen INystad SLiahjell MDang-Nguyen DKesavulu MNinh VGurrin CVuong GHo VTran M(2024)T@Retrospect: A Journey Through TimeProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661118(36-40)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661118
Nguyen-Ho TKongmeesub OTran MNie DHealy GGurrin C(2024)EAGLE: Eyegaze-Assisted Guidance and Learning Evaluation for Lifeloging RetrievalProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661115(18-23)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661115
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents