Export Citations
Save this search
Please login to be able to save your searches and receive alerts for new content matching your search criteria.
- research-articleJune 2024
LifeGraph 4 - Lifelog Retrieval using Multimodal Knowledge Graphs and Vision-Language Models
- Luca Rossetto,
- Athina Kyriakou,
- Svenja Lange,
- Florian Ruosch,
- Ruijie Wang,
- Kathrin Wardatzky,
- Abraham Bernstein
LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search ChallengeJune 2024, Pages 88–92https://doi.org/10.1145/3643489.3661127In the scope of the 7th Lifelog Search Challenge (LSC'24), we present the 4th iteration of LifeGraph, a multimodal knowledge-graph approach with data augmentations using Vision-Language Models (VLM). We extend the LifeGraph model presented in former LSC ...
- research-articleJune 2024
General Purpose Multimedia Retrieval with vitrivr at LSC'24
LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search ChallengeJune 2024, Pages 47–52https://doi.org/10.1145/3643489.3661120The collection of lifelog data --- visual and multi-sensory data, including biometric and spatiotemporal metadata --- becomes easier and more supported by commercial products every year. Naturally, lifelog data is multi-modal, with arguably a major audio-...
- research-articleJune 2024
Spatiotemporal Lifelog Analytics in Virtual Reality with vitrivr-VR
LSC '24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search ChallengeJune 2024, Pages 7–11https://doi.org/10.1145/3643489.3661113Modern wearables and smart devices make it easier than ever to collect a detailed, digital record of biometric as well as visual and aural information. Reasons to collect such a lifelog range from health applications to vacation documentation. With the ...
- abstractJune 2024
Introduction to the Seventh Annual Lifelog Search Challenge, LSC'24
- Cathal Gurrin,
- Liting Zhou,
- Graham Healy,
- Werner Bailer,
- Duc-Tien Dang Nguyen,
- Steve Hodges,
- Björn Þór Jónsson,
- Jakub Lokoč,
- Luca Rossetto,
- Minh-Triet Tran,
- Klaus Schöffmann
ICMR '24: Proceedings of the 2024 International Conference on Multimedia RetrievalMay 2024, Pages 1334–1335https://doi.org/10.1145/3652583.3658891For the seventh time since 2018, the Lifelog Search Challenge (LSC) benchmarked interactive lifelog search systems in a live challenge. The LSC goal is to comparatively evaluate system capabilities to access large multimodal lifelogs comprising hundreds ...
- short-paperJune 2024
Reproducibility Companion Paper of "MMSF: A Multimodal Sentiment-Fused Method to Recognize Video Speaking Style"
ICMR '24: Proceedings of the 2024 International Conference on Multimedia RetrievalMay 2024, Pages 1232–1235https://doi.org/10.1145/3652583.3658373To support the replication of "MMSF: A Multimodal Sentiment-Fused Method to Recognize Video Speaking Style", which was presented at ICMR'23, this companion paper provides the details of the artifacts. Speaking style recognition is aimed at recognizing ...
-
- short-paperJune 2024Honorable Mention Short Paper
OpenLifelogCam - A Low-Cost Open-Source Wearable Camera Platform
ICMR '24: Proceedings of the 2024 International Conference on Multimedia RetrievalMay 2024, Pages 1236–1240https://doi.org/10.1145/3652583.3657588The capture and subsequent analysis of egocentric imagery in the form of Lifelogs can be useful in several application areas. However, suitable hardware to record such data is not always available or can be cost-prohibitive. This paper introduced the ...
- proceedingJune 2024
ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval
- Cathal Gurrin,
- Rachada Kongkachandra,
- Klaus Schoeffmann,
- Duc-Tien Dang-Nguyen,
- Luca Rossetto,
- Shin'ichi Satoh,
- Liting Zhou
We are pleased to present the 2024 edition of the ACM International Conference on Multimedia Retrieval, ACM ICMR 2024, that took place from 10-14 June 2024, in Phuket, Thailand.
Effectively and efficiently retrieving information from multimedia ...
- ArticleMay 2024
QAGCN: Answering Multi-relation Questions via Single-Step Implicit Reasoning over Knowledge Graphs
AbstractMulti-relation question answering (QA) is a challenging task, where given questions usually require long reasoning chains in KGs that consist of multiple relations. Recently, methods with explicit multi-step reasoning over KGs have been ...
- ArticleJanuary 2024
A New Retrieval Engine for Vitrivr
AbstractWhile the vitrivr stack has seen many changes in components over the years, its feature extraction and query processing engine traces its history back almost a decade. Some aspects of its architecture and operation are no longer current, limiting ...
- ArticleJanuary 2024
Exploring Multimedia Vector Spaces with vitrivr-VR
AbstractVirtual reality (VR) interfaces are becoming more commonplace as the number of capable and affordable devices increases. However, VR user interfaces for common computing tasks often fail to take full advantage of the affordances provided by this ...
- ArticleJanuary 2024
Augmented Reality Photo Presentation and Content-Based Image Retrieval on Mobile Devices with AR-Explorer
AbstractMobile devices are increasingly being used not only to take photos but also to display and present them to their users in an easily accessible and attractive way. Especially for spatially referenced objects, Augmented Reality (AR) offers new and ...
- research-articleOctober 2023
Spatially Localised Immersive Contemporary and Historic Photo Presentation on Mobile Devices in Augmented Reality
SUMAC '23: Proceedings of the 5th Workshop on analySis, Understanding and proMotion of heritAge ContentsNovember 2023, Pages 13–19https://doi.org/10.1145/3607542.3617358These days, taking a photo is the most common way of capturing a moment. Some of these photos captured in the moment are never to be seen again. Others are almost immediately shared with the world. Yet, the context of the captured moment can only be ...
- short-paperDecember 2023
Novice-Friendly Text-based Video Search with vitrivr
CBMI '23: Proceedings of the 20th International Conference on Content-based Multimedia IndexingSeptember 2023, Pages 163–167https://doi.org/10.1145/3617233.3617262Video retrieval still offers many challenges which can so far only be effectively mediated through interactive, human-in-the-loop retrieval approaches. The vitrivr multimedia retrieval stack offers a broad range of query mechanisms to enable users to ...
- research-articleAugust 2023
Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th VBS
- Jakub Lokoč,
- Stelios Andreadis,
- Werner Bailer,
- Aaron Duane,
- Cathal Gurrin,
- Zhixin Ma,
- Nicola Messina,
- Thao-Nhu Nguyen,
- Ladislav Peška,
- Luca Rossetto,
- Loris Sauter,
- Konstantin Schall,
- Klaus Schoeffmann,
- Omar Shahbaz Khan,
- Florian Spiess,
- Lucia Vadicamo,
- Stefanos Vrochidis
Multimedia Systems (MUME), Volume 29, Issue 6Dec 2023, Pages 3481–3504https://doi.org/10.1007/s00530-023-01143-5AbstractThis paper presents findings of the eleventh Video Browser Showdown competition, where sixteen teams competed in known-item and ad-hoc search tasks. Many of the teams utilized state-of-the-art video retrieval approaches that demonstrated high ...
- research-articleJune 2023
The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid System
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search ChallengeJune 2023, Pages 65–68https://doi.org/10.1145/3592573.3593107Personal lifelog data collections are becoming more common as a memory aid, as well as for analytical tasks, such as health and fitness analysis. Due to the multimodal and personal nature of lifelog data, interactive multimedia retrieval approaches are ...
- research-articleJune 2023
Multi-Mode Clustering for Graph-Based Lifelog Retrieval
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search ChallengeJune 2023, Pages 36–40https://doi.org/10.1145/3592573.3593102As part of the 6th Lifelog Search Challenge, this paper presents an approach to arrange Lifelog data in a multi-modal knowledge graph based on cluster hierarchies. We use multiple sequence clustering approaches to address the multi-modal nature of ...
- abstractJune 2023
Introduction to the Sixth Annual Lifelog Search Challenge, LSC’23
- Cathal Gurrin,
- Björn Þór Jónsson,
- Duc Tien Dang Nguyen,
- Graham Healy,
- Jakub Lokoc,
- Liting Zhou,
- Luca Rossetto,
- Minh-Triet Tran,
- Wolfgang Hürst,
- Werner Bailer,
- Klaus Schoeffmann
ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia RetrievalJune 2023, Pages 678–679https://doi.org/10.1145/3591106.3592304For the sixth time since 2018, the Lifelog Search Challenge (LSC) was organized as a comparative benchmarking exercise for various interactive lifelog search systems. The goal of this international competition is to test system capabilities to access ...
- short-paperJune 2023
A Comparison of Video Browsing Performance between Desktop and Virtual Reality Interfaces
ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia RetrievalJune 2023, Pages 535–539https://doi.org/10.1145/3591106.3592292Interactive retrieval with user-friendly and performant interfaces remains a necessity for video retrieval, even in light of significant gains in retrieval performance through multi-modal encoders. In recent years, novel interaction modalities such as ...
- ArticleMarch 2023
Exploring Effective Interactive Text-Based Video Search in vitrivr
Abstractvitrivr is a general purpose retrieval system that supports a wide range of query modalities. In this paper, we briefly introduce the system and describe the changes and adjustments made for the 2023 iteration of the video browser showdown. These ...