research-article

Interactive Video Retrieval in the Age of Deep Learning

Authors:

Klaus Schoeffmann,

Cathal GurrinAuthors Info & Claims

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

Pages 2 - 4

https://doi.org/10.1145/3323873.3326588

Published: 05 June 2019 Publication History

Abstract

We present a tutorial focusing on video retrieval tasks, where state-of-the-art deep learning approaches still benefit from interactive decisions of users. The tutorial covers general introduction to the interactive video retrieval research area, state-of-the-art video retrieval systems, evaluation campaigns and recently observed results. Moreover, a significant part of the tutorial is dedicated to a practical exercise with three selected state-of-the-art systems in the form of an interactive video retrieval competition. Participants of this tutorial will gain a practical experience and also a general insight of the interactive video retrieval topic, which is a good start to focus their research on unsolved challenges in this area.

References

[1]

George Awad, Asad Butt, Jonathan Fiscus, Martial Michel, David Joy, Wessel Kraaij, Alan F. Smeaton, Georges Quénot, Maria Eskevich, Roeland Ordelman, Gareth J. F. Jones, and Benoit Huet. 2017. TRECVID 2017: Evaluating Ad-hoc and Instance Video Search, Events Detection, Video Captioning and Hyperlinking. In Proceedings of TRECVID 2017. NIST, USA.

[2]

J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248--255.

[3]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, Minh-Triet Tran, Jakub Lokovic, and Wolfgang Hürst. 2019. Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018). ITE Transactions on Media Technology and Applications, Vol. 7 (04/2019 2019), 46--59.

[4]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, Minh-Triet Tran, Jakub Lokovc, and Wolfgang Hürst. 2019. {Invited papers} Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018). ITE Transactions on Media Technology and Applications, Vol. 7, 2 (2019), 46--59.

[5]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Bernd Munzer, Rami Albatal, Frank Hopfgartner, Liting Zhou, and Duc-Tien Dang-Nguyen. 2019. A Test Collection for Interactive Lifelog Retrieval. In MultiMedia Modeling, Ioannis Kompatsiaris, Benoit Huet, Vasileios Mezaris, Cathal Gurrin, Wen-Huang Cheng, and Stefanos Vrochidis (Eds.). Springer International Publishing, Cham, 312--324.

[6]

Jakub Lokoc, Gregor Kovalcik, Bernd Mü nzer, Klaus Schö ffmann, Werner Bailer, Ralph Gasser, Stefanos Vrochidis, Phuong Anh Nguyen, Sitapa Rujikietgumjorn, and Kai Uwe Barthel. 2019. Interactive Search or Sequential Browsing? A Detailed Analysis of the Video Browser Showdown 2018. TOMCCAP, Vol. 15, 1 (2019), 29:1--29:18.

[7]

Jakub Lokovc, Werner Bailer, Klaus Schoeffmann, Bernd Mü nzer, and George Awad. 2018. On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015--2017. IEEE Trans. Multimedia, Vol. 20, 12 (2018), 3361--3376.

Digital Library

[8]

Jakub Lokovc, Gregor Kovalvc'i k, Tomá vs Souvcek, Jaroslav Moravec, and Pvremysl Cech. 2019. VIRET: A Video Retrieval Tool for Interactive Known-item Search. In International Conference on Multimedia Retrieval (ICMR '19), June 10--13, 2019, Ottawa, ON, Canada. 1--5.

[9]

Jakub Lokovc, Tomávs Souvcek, and Gregor Kovalvcik. 2018. Using an Interactive Video Retrieval Tool for LifeLog Data. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge, LSC@ICMR 2018, Yokohama, Japan, June 11, 2018. 15--19.

Digital Library

[10]

Luca Rossetto, Ivan Giangreco, Claudiu Tanase, and Heiko Schuldt. 2016. vitrivr: A flexible retrieval stack supporting multiple query modes for searching in multimedia collections. In Proceedings of the 2016 ACM on Multimedia Conference. ACM, 1183--1186.

Digital Library

[11]

Luca Rossetto, Mahnaz Amiri Parian, Ralph Gasser, Ivan Giangreco, Silvan Heller, and Heiko Schuldt. 2019. Deep Learning-Based Concept Detection in vitrivr. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part II. 616--621.

[12]

Luca Rossetto, Heiko Schuldt, George Awad, and Asad A. Butt. 2019. V3C - A Research Video Collection. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part I. 349--360.

[13]

Klaus Schoeffmann, Werner Bailer, Cathal Gurrin, George Awad, and Jakub Lokovc. 2018. Interactive Video Search: Where is the User in the Age of Deep Learning?. In Proceedings of the 26th ACM International Conference on Multimedia (MM '18). ACM, New York, NY, USA, 2101--2103.

Digital Library

[14]

Klaus Schoeffmann, Bernd Mü nzer, Andreas Leibetseder, Jürgen Primus, and Sabrina Kletz. 2019. Autopiloting Feature Maps: The Deep Interactive Video Exploration (diveXplore) System at VBS2019. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part II. 585--590.

[15]

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2017. Learning Transferable Architectures for Scalable Image Recognition. CoRR, Vol. abs/1707.07012 (2017). arxiv: 1707.07012 http://arxiv.org/abs/1707.07012

Cited By

Mühling MKorfhage NPustu-Iren KBars JKnapp MBellafkir HVogelbacher MSchneider DHörth AEwerth RFreisleben B(2022)VIVA: visual information retrieval in video archivesInternational Journal on Digital Libraries10.1007/s00799-022-00337-y23:4(319-333)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00799-022-00337-y

Index Terms

Interactive Video Retrieval in the Age of Deep Learning
1. Information systems
  1. Information retrieval
  2. Information systems applications
    1. Multimedia information systems
2. Security and privacy
  1. Security services
    1. Digital rights management

Recommendations

Interactive Video Search: Where is the User in the Age of Deep Learning?
MM '18: Proceedings of the 26th ACM international conference on Multimedia

In this tutorial we discuss interactive video search tools and methods, review their need in the age of deep learning, and explore video and multimedia search challenges and their role as evaluation benchmarks in the field of multimedia information ...
Towards Explainable Interactive Multi-modal Video Retrieval with Vitrivr
MultiMedia Modeling
Abstract
This paper presents the most recent iteration of the vitrivr multimedia retrieval system for its participation in the Video Browser Showdown (VBS) 2021. Building on existing functionality for interactive multi-modal retrieval, we overhaul query ...
VideoCLIP 2.0: An Interactive CLIP-Based Video Retrieval System for Novice Users at VBS2024
MultiMedia Modeling
Abstract
In this paper, we present an interactive video retrieval system named VideoCLIP 2.0 developed for the Video Browser Showdown in 2024. Building upon the foundation of the previous year’s system, VideoCLIP, this upgraded version incorporates ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

June 2019

427 pages

ISBN:9781450367653

DOI:10.1145/3323873

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada
,
Alberto Del Bimbo
University of Florence, Italy
,
Zhongfei Zhang
Binghamton University, State University of New York, USA
,
Program Chairs:
Alexander Hauptmann
Carnegie Mellon University, USA
,
K. Selcuk Candan
Arizona State University, USA
,
Marco Bertini
University of Florence, Italy
,
Lexing Xie
Australia National University, Australia
,
Xiao-Yong Wei
Sichuan University, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Science Foundation Ireland
Czech Science Foundation

Conference

ICMR '19

Sponsor:

SIGMM

ICMR '19: International Conference on Multimedia Retrieval

June 10 - 13, 2019

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
192
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mühling MKorfhage NPustu-Iren KBars JKnapp MBellafkir HVogelbacher MSchneider DHörth AEwerth RFreisleben B(2022)VIVA: visual information retrieval in video archivesInternational Journal on Digital Libraries10.1007/s00799-022-00337-y23:4(319-333)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00799-022-00337-y

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents