research-article

Efficient targeted search using a focus and context video browser

Authors:

Marcel WorringAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 8, Issue 4

Article No.: 51, Pages 1 - 19

https://doi.org/10.1145/2379790.2379793

Published: 30 November 2012 Publication History

Abstract

Currently there are several interactive content-based video retrieval techniques and systems available. However, retrieval performance depends heavily on the means of interaction. We argue that effective CBVR requires efficient, specialized user interfaces. In this article we propose guidelines for such an interface, and we propose an effective CBVR engine: the ForkBrowser, which builds upon the principle of focus and context. This browser is evaluated using a combination of user simulation and real user evaluation. Results indicate that the ideas have merit, and that the browser performs very well when compared to the state-of-the-art in video retrieval.

References

[1]

Adcock, J., Cooper, M., and Chen, F. 2007. Fxpal mediamagic video search system. In Proceedings of the ACM International Conference on Image and Video Retrieval. 644--644.

Digital Library

[2]

Adcock, J., Cooper, M., Girgensohn, A., and Wilcox, L. 2005. Interactive video search using multilevel indexing. In Proceedings of the ACM International Conference on Image and Video Retrieval. Lecture Notes in Computer Science, vol. 3568. Springer. 205--214.

Digital Library

[3]

Chang, C.-C. and Lin, C.-J. 2001. Libsvm: A library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm/.

Digital Library

[4]

Chen, M.-Y., Christel, M., Hauptmann, A., and Wactlar, H. 2005. Putting active learning into multimedia applications: Dynamic Definition and refinement of concept classifiers. In Proceedings of the 13th Annual ACM International Conference on Multimedia. ACM, New York, 902--911.

Digital Library

[5]

Christel, M. G., Huang, C., Moraveji, N., and Papernick, N. 2004. Exploiting multiple modalities for interactive video retrieval. In Proceedings of the IEEE International Conference on Acoustic, Speech and Signal Processing. Vol. 3. 1032--1035.

[6]

Christel, M. G. and Yan, R. 2007. Merging storyboard strategies and automatic retrieval for improving interactive video search. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval. ACM, New York, 486--493.

Digital Library

[7]

Cord, M., Gosselin, P. H., and Philipp-Foliguet, S. 2007. Stochastic exploration and active learning for image retrieval. Image Vis. Comput. 25, 1, 14--23.

[8]

de Rooij, O. and Worring, M. 2010. Browsing video along multiple threads. IEEE Trans. Multimedia 12, 2, 121--130.

Digital Library

[9]

Furnas, G. 1986. Generalized fisheye views. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 16--23.

Digital Library

[10]

Gosselin, P. H. and Cord, M. 2004. A comparison of active classification methods for content-based image retrieval. In Proceedings of the 1st International Workshop on Computer Vision Meets Databases. ACM, New York, 51--58.

Digital Library

[11]

Hauptmann, A. G. and Christel, M. G. 2004. Successful approaches in the trec video retrieval evaluations. In Proceedings of the 12th Annual ACM International Conference on Multimedia. ACM, New York, 668--675.

Digital Library

[12]

Hauptmann, A. G., Lin, W.-H., Yan, R., Yang, J., and Chen, M.-Y. 2006. Extreme video retrieval: Joint maximization of human and computer performance. In Proceedings of the 14th Annual ACM International Conference on Multimedia. ACM Press, New York, 385--394.

Digital Library

[13]

Huijbregts, M., Ordelman, R., and de Jong, F. 2007. Annotation of heterogeneous multimedia content using automatic speech recognition. In Proceedings of the International Conference on Semantics and Digital Media Technologies. Lecture Notes in Computer Science.

Digital Library

[14]

Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-Based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl. 2, 1, 1--19.

Digital Library

[15]

Luan, H.-B., Neo, S.-Y., Goh, H.-K., Zhang, Y.-D., Lin, S.-X., and Chua, T.-S. 2007. Segregated feedback with performance-based adaptive sampling for interactive news video retrieval. In Proceedings of the 15th International Conference on Multimedia. ACM, New York, 293--296.

Digital Library

[16]

Natsev, A. P., Haubold, A., Tešić, J., Xie, L., and Yan, R. 2007. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In Proceedings of the 15th International Conference on Multimedia. ACM, New York, 991--1000.

Digital Library

[17]

Natsev, A. P., Naphade, M. R., and Tešić, J. 2005. Learning the semantics of multimedia queries and concepts from a small number of examples. In Proceedings of the 13th Annual ACM International Conference on Multimedia. ACM, New York, 598--607.

Digital Library

[18]

Petersohn, C. 2004. Fraunhofer HHI at TRECVID 2004: Shot boundary detection system. In Proceedings of the TRECVID Workshop.

[19]

Rautiainen, M., Seppänen, T., and Ojala, T. 2006. On the significance of cluster-temporal browsing for generic video retrieval: A statistical analysis. In Proceedings of the 14th Annual ACM International Conference on Multimedia. ACM, New York, 125--128.

Digital Library

[20]

Robertson, G., Czerwinski, M., Larson, K., Robbins, D. C., Thiel, D., and van Dantzich, M. 1998. Data mountain: Using spatial memory for document management. In Proceedings of the 11th Annual ACM Symposium on User interface Software and Technology. ACM Press, New York, 153--162.

Digital Library

[21]

Smeaton, A. F., Over, P., and Kraaij, W. 2006. Evaluation campaigns and trecvid. In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval. ACM Press, New York, 321--330.

Digital Library

[22]

Snoek, C. G. M., van de Sande, K. E. A., de Rooij, O., Huurnink, B., van Gemert, J. C., Uijlings, J. R. R., He, J., Li, X., Everts, I., Nedović, V., van Liempt, M., van Balen, R., Yan, F., Tahir, M. A., Mikolajczyk, K., Kittler, J., de Rijke, M., Geusebroek, J.-M., Gevers, T., Worring, M., Smeulders, A. W. M., and Koelma, D. C. 2008. The MediaMill TRECVID 2008 semantic video search engine. In Proceedings of the 6th TRECVID Workshop.

[23]

Snoek, C. G. M., Worring, M., Koelma, D. C., and Smeulders, A. W. M. 2007. A learned lexicon-driven paradigm for interactive video retrieval. IEEE Trans. Multimedia 9, 2, 280--292.

Digital Library

[24]

Sundaram, H. and Chang, S.-F. 2001. Condensing computable scenes using visual complexity and film syntax analysis. In Proceedings of the IEEE International Conference on Multimedia and Expo. 70.

[25]

Tong, S. and Chang, E. 2001. Support vector machine active learning for image retrieval. In Proceedings of the 9th ACM International Conference on Multimedia. ACM, New York, 107--118.

Digital Library

[26]

van Gemert, J. C., Snoek, C. G. M., Veenman, C. J., Smeulders, A. W. M., and Geusebroek, J. M. 2010. Comparing compact codebooks for visual categorization. Computer Vision and Image Understanding in press.

Digital Library

[27]

Wang, D., Liu, X., Luo, L., Li, J., and Zhang, B. 2007. Video diver: Generic video indexing with diverse features. In Proceedings of the International Workshop on Multimedia Information Retrieval. ACM, New York, 61--70.

Digital Library

[28]

Ware, C. 2000. Information Visualization: Perception for Design. Morgan Kaufmann Publishers Inc., San Francisco, CA.

Digital Library

[29]

Yan, R., Natsev, A., and Campbell, M. 2007. An efficient manual image annotation approach based on tagging and browsing. In Workshop on Multimedia Information Retrieval on the Many Faces of Multimedia Semantics. ACM, New York, 13--20.

Digital Library

[30]

Yee, K.-P., Swearingen, K., Li, K., and Hearst, M. 2003. Faceted metadata for image search and browsing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 401--408.

Digital Library

[31]

Zavesky, E., Chang, S.-F., and Yang, C.-C. 2008. Visual islands: Intuitive browsing of visual search results. In Proceedings of the International Conference on Content-Based Image and Video Retrieval. ACM, New York, 617--626.

Digital Library

Cited By

Khan OJónsson BZahálka JRudinac SWorring MCheng WKankanhalli MWang MChu WLiu JWorring M(2021)Impact of Interaction Strategies on User Relevance FeedbackProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463663(590-598)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463663
Huang JHuck Yang CLiu FTian MLiu YWu TLin IWang KMorikawa HChang HTegner JWorring M(2021)DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00249(2441-2451)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00249
Spolaôr NLee HTakaki WEnsina LParmezan AOliva JCoy CWu F(2021)A video indexing and retrieval computational prototype based on transcribed speechMultimedia Tools and Applications10.1007/s11042-021-11401-1Online publication date: 30-Aug-2021
https://doi.org/10.1007/s11042-021-11401-1
Show More Cited By

Recommendations

Browsing Video Along Multiple Threads

This paper describes a novel method for browsing a large video collection. It links various forms of related video fragments together as threads. These threads are based on query results, the timeline as well as visual and semantic similarity. We design ...
Query on demand video browsing
MM '07: Proceedings of the 15th ACM international conference on Multimedia

This paper describes a novel method for browsing a large collection of news video by linking various forms of related video fragments together as threads. Each thread contains a sequence of shots with high feature-based similarity. Two interfaces are ...
Balancing thread based navigation for targeted video search
CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

Various query methods for video search exist. Because of the semantic gap each method has its limitations. We argue that for effective retrieval query methods need to be combined at retrieval time. However, switching query methods often involves a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 8, Issue 4

November 2012

139 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2379790

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2012

Accepted: 01 May 2011

Revised: 01 December 2010

Received: 01 June 2010

Published in TOMM Volume 8, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
190
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khan OJónsson BZahálka JRudinac SWorring MCheng WKankanhalli MWang MChu WLiu JWorring M(2021)Impact of Interaction Strategies on User Relevance FeedbackProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463663(590-598)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463663
Huang JHuck Yang CLiu FTian MLiu YWu TLin IWang KMorikawa HChang HTegner JWorring M(2021)DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00249(2441-2451)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00249
Spolaôr NLee HTakaki WEnsina LParmezan AOliva JCoy CWu F(2021)A video indexing and retrieval computational prototype based on transcribed speechMultimedia Tools and Applications10.1007/s11042-021-11401-1Online publication date: 30-Aug-2021
https://doi.org/10.1007/s11042-021-11401-1
Antaris SRafailidis D(2015)Similarity Search over the Cloud Based on Image Descriptors' Dimensions Value CardinalitiesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/271631511:4(1-23)Online publication date: 2-Jun-2015
https://dl.acm.org/doi/10.1145/2716315
Kloosterman AMapes AGeradts Zvan Eijk EKoper Cvan den Berg JVerheij Svan der Steen Mvan Asten A(2015)The interface between forensic science and technology: how technology could cause a paradigm shift in the role of forensic institutes in the criminal justice systemPhilosophical Transactions of the Royal Society B: Biological Sciences10.1098/rstb.2014.0264370:1674(20140264)Online publication date: 22-Jun-2015
https://doi.org/10.1098/rstb.2014.0264
Zahalka JWorring M(2014)Towards interactive, intelligent, and integrated multimedia analytics2014 IEEE Conference on Visual Analytics Science and Technology (VAST)10.1109/VAST.2014.7042476(3-12)Online publication date: Oct-2014
https://doi.org/10.1109/VAST.2014.7042476

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents