tutorial

Interactive Video Search: Where is the User in the Age of Deep Learning?

Authors:

Klaus Schoeffmann,

Jakub LokočAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 2101 - 2103

https://doi.org/10.1145/3240508.3241473

Published: 15 October 2018 Publication History

Abstract

In this tutorial we discuss interactive video search tools and methods, review their need in the age of deep learning, and explore video and multimedia search challenges and their role as evaluation benchmarks in the field of multimedia information retrieval. We cover three different campaigns (TRECVID, Video Browser Showdown, and the Lifelog Search Challenge), discuss their goals and rules, and present their achieved findings over the last half-decade. Moreover, we talk about datasets, tasks, evaluation procedures, and examples of interactive video search tools, as well as how they evolved over the years. Participants of this tutorial will be able to gain collective insights from all three challenges and use them for focusing their research efforts on outstanding problems that still remain unsolved in this area.

References

[1]

George Awad, Asad Butt, Jonathan Fiscus, Martial Michel, David Joy, Wessel Kraaij, Alan F. Smeaton, Georges Quénot, Maria Eskevich, Roeland Ordelman, Gareth J. F. Jones, and Benoit Huet. 2017. TRECVID 2017: Evaluating Ad-hoc and Instance Video Search, Events Detection, Video Captioning and Hyperlinking. In Proceedings of TRECVID 2017. NIST, USA.

[2]

Werner Bailer and Herwig Rehatschek. 2009. Comparing Fact Finding Tasks and User Survey for Evaluating a Video Browsing Tool. In Proceedings of ACM Multimedia. Beijing, CN.

Digital Library

[3]

Nancy A Chinchor, James J Thomas, Pak Chung Wong, Michael G Christel, and William Ribarsky. 2010. Multimedia analysis+ visual analytics= multimedia analytics. IEEE computer graphics and applications, Vol. 30, 5 (2010), 52--60.

Digital Library

[4]

Claudiu Cobârzan, Klaus Schoeffmann, Werner Bailer, Wolfgang Hürst, Adam Blavz ek, Jakub Lokovc, Stefanos Vrochidis, Kai Uwe Barthel, and Luca Rossetto. 2017. Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimedia Tools and Applications, Vol. 76, 4 (2017), 5539--5571.

Digital Library

[5]

Duc-Tien Dang-Nguyen, Klaus Schoeffmann, and Wolfgang Hurst. 2018. LSE2018 Panel - Challenges of Lifelog Search and Access. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 1--2.

Digital Library

[6]

Aaron Duane, Cathal Gurrin, and Wolfgang Huerst. 2018. Virtual Reality Lifelog Explorer: Lifelog Search Challenge at ACM ICMR 2018. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 20--23.

Digital Library

[7]

Christoph Kofler, Martha Larson, and Alan Hanjalic. 2016. User intent in multimedia search: a survey of the state of the art and future challenges. ACM Computing Surveys (CSUR), Vol. 49, 2 (2016), 36.

Digital Library

[8]

Martha Larson, Mohammad Soleymani, Guillaume Gravier, Bogdan Ionescu, and Gareth JF Jones. 2017. The benchmarking initiative for multimedia evaluation: MediaEval 2016. IEEE MultiMedia, Vol. 24, 1 (2017), 93--96.

[9]

J. Lokovc, W. Bailer, K. Schoeffmann, B. Muenzer, and G. Awad. 2018a. On influential trends in interactive video retrieval: Video Browser Showdown 2015--2017. IEEE Transactions on Multimedia (2018), 1--1.

[10]

Jakub Lokovc, Gregor Kovalvc'i k, and Tomá vs Souvcek. 2018b. Revisiting SIRET Video Retrieval Tool. In MultiMedia Modeling - 24th International Conference, MMM 2018, Bangkok, Thailand, February 5--7, 2018, Proceedings, Part II. 419--424.

[11]

Jakub Lokovc, Tomá vs Souvcek, and Gregor Kovalvc'i k. 2018c. Using an Interactive Video Retrieval Tool for LifeLog Data. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge, LSC@ICMR 2018, Yokohama, Japan, June 11, 2018. 15--19.

Digital Library

[12]

Bernd Münzer, Andreas Leibetseder, Sabrina Kletz, Manfred Jürgen Primus, and Klaus Schoeffmann. 2018. lifeXplore at the Lifelog Search Challenge 2018. Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 3--8.

Digital Library

[13]

Manfred Jürgen Primus, Bernd Münzer, Andreas Leibetseder, and Klaus Schoeffmann. 2018. The ITEC Collaborative Video Search System at the Video Browser Showdown 2018. In International Conference on Multimedia Modeling. Springer, 438--443.

[14]

Klaus Schoeffmann, Marco A. Hudelist, and Jochen Huber. 2015. Video Interaction Tools: A Survey of Recent Work. ACM Comput. Surv., Vol. 48, 1, Article 14 (Sept. 2015), 34 pages.

Digital Library

[15]

Marcel Worring, Paul Sajda, Simone Santini, David A Shamma, Alan F Smeaton, and Qiang Yang. 2012. Where is the user in multimedia retrieval? IEEE MultiMedia, Vol. 19, 4 (2012), 6--10.

Digital Library

Cited By

Schoeffmann KLokoč JBailer WChua TWang JTian QGurrin CJia JZhang HSun Q(2021)10 years of video browser showdownProceedings of the 2nd ACM International Conference on Multimedia in Asia10.1145/3444685.3450215(1-3)Online publication date: 7-Mar-2021
https://dl.acm.org/doi/10.1145/3444685.3450215
Khan OJónsson BRudinac SZahálka JRagnarsdóttir HÞorleiksdóttir ÞGuðmundsson GAmsaleg LWorring M(2020)Interactive Learning for Multimedia at LargeAdvances in Information Retrieval10.1007/978-3-030-45439-5_33(495-510)Online publication date: 8-Apr-2020
https://doi.org/10.1007/978-3-030-45439-5_33
Khan OJónsson BZahálka JRudinac SWorring MGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)Exquisitor at the Lifelog Search Challenge 2019Proceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329156(7-11)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329156
Show More Cited By

Index Terms

Interactive Video Search: Where is the User in the Age of Deep Learning?
1. Information systems
  1. Information retrieval

Recommendations

Interactive Video Retrieval in the Age of Deep Learning
ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

We present a tutorial focusing on video retrieval tasks, where state-of-the-art deep learning approaches still benefit from interactive decisions of users. The tutorial covers general introduction to the interactive video retrieval research area, state-...
Interactive Video Search
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

With an increasing amount of video data in our daily life, the need for content-based search in videos increases as well. Though a lot of research has been spent on video retrieval tools and methods which allow for automatic search in videos through ...
Exploring Effective Interactive Text-Based Video Search in vitrivr
MultiMedia Modeling
Abstract
vitrivr is a general purpose retrieval system that supports a wide range of query modalities. In this paper, we briefly introduce the system and describe the changes and adjustments made for the 2023 iteration of the video browser showdown. These ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
174
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schoeffmann KLokoč JBailer WChua TWang JTian QGurrin CJia JZhang HSun Q(2021)10 years of video browser showdownProceedings of the 2nd ACM International Conference on Multimedia in Asia10.1145/3444685.3450215(1-3)Online publication date: 7-Mar-2021
https://dl.acm.org/doi/10.1145/3444685.3450215
Khan OJónsson BRudinac SZahálka JRagnarsdóttir HÞorleiksdóttir ÞGuðmundsson GAmsaleg LWorring M(2020)Interactive Learning for Multimedia at LargeAdvances in Information Retrieval10.1007/978-3-030-45439-5_33(495-510)Online publication date: 8-Apr-2020
https://doi.org/10.1007/978-3-030-45439-5_33
Khan OJónsson BZahálka JRudinac SWorring MGurrin CSchoeffmann KJoho HDang-Nguyen DRiegler MPiras L(2019)Exquisitor at the Lifelog Search Challenge 2019Proceedings of the ACM Workshop on Lifelog Search Challenge10.1145/3326460.3329156(7-11)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3326460.3329156
Lokoč JSchoeffmann KBailer WRossetto LGurrin CEl Saddik ADel Bimbo AZhang ZHauptmann ACandan KBertini MXie LWei X(2019)Interactive Video Retrieval in the Age of Deep LearningProceedings of the 2019 on International Conference on Multimedia Retrieval10.1145/3323873.3326588(2-4)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3323873.3326588
Jónsson BKhan OKoelma DRudinac SWorring MZahálka J(2019)Exquisitor at the Video Browser Showdown 2020MultiMedia Modeling10.1007/978-3-030-37734-2_72(796-802)Online publication date: 24-Dec-2019
https://doi.org/10.1007/978-3-030-37734-2_72

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents