short-paper

Interactive inquiry for object of interest in video playback by motion-augmented graph cut

Authors:

Po-Nung Tseng,

Yen-Liang Lin,

Winston H. HsuAuthors Info & Claims

MM '10: Proceedings of the 18th ACM international conference on Multimedia

Pages 811 - 814

https://doi.org/10.1145/1873951.1874085

Published: 25 October 2010 Publication History

Get Access

Abstract

The touch-based displays (devices) have entailed rich interactions between the videos and users. The objects appearing in videos usually interest users in wanting to know relative knowledge about them. In this paper, we proposed a video playback system for users to interactively query objects of interest in videos. Since the text information accompanied with videos might not be strongly related to the object of interest, we adopt visual appearances as features to retrieve similar objects from large image collections. The tags associated with the retrieved images are used to reveal related information of the object of interest for further exploiting related knowledge. Solely relying on single viewpoint of the object to query may suffer from different poses, occlusions and is not robust. So we present a novel video object segmentation approach to improve retrieval precision. The approach is based on a 3D graph cut framework. To ensure prompt response and effectiveness, we augment the algorithm with compressed-domain motion vectors; compared with the prior method, the processing speed of our approach is significantly faster. The experiments on community-contributed videos demonstrate the effectiveness of our approach based on multi-frame object region query and the improvement of retrieval precision.

References

[1]

H. Miyamori and K. Tanaka. Webified video: media conversion from TV program to web content and their integrated viewing method. In WWW, 2005.

Digital Library

Google Scholar

[2]

T. Yeh, J. J. Lee, and T. Darrell. Photo-based question answering. In ACM Multimedia, 2008.

Digital Library

Google Scholar

[3]

Y. Boykov and M.-P. Jolly. Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In ICCV, 2001.

Crossref

Google Scholar

[4]

Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. PAMI, 26(9), Sep. 2004.

Digital Library

Google Scholar

[5]

C. Rother et al. GrabCut: Interactive foreground extraction using iterated graph cuts. In SIGGRAPH, 2004.

Digital Library

Google Scholar

[6]

Y. Li, J. Sun and H.-Y. Shum. Video object cut and paste. In SIGGRAPH, 2005.

Digital Library

Google Scholar

[7]

J. Wang, P. Bhat, A. Colburn, M. Agrawala and M. Cohen. Interactive video cutout. In SIGGRAPH, 2005.

Digital Library

Google Scholar

[8]

W. Zeng, J. Du, W. Gao and Q. Huang. Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model. JRTIP, 11(4), Aug. 2005.

Digital Library

Google Scholar

[9]

Z. Liu, Y. Lu and Z. Zhang. Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain. JVCIR, 18(3), June 2007.

Digital Library

Google Scholar

[10]

K.-H. Lin et al. Boosting object retrieval by estimating pseudo-objects. In ICIP, 2009.

Digital Library

Google Scholar

[11]

J. Philbin et al. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.

Crossref

Google Scholar

[12]

Y.-H. Yang et al. ContextSeer: Context search and recommendation at query time for shared consumer photos. In ACM Multimedia, 2008.

Digital Library

Google Scholar

[13]

S. Khan et al. Object based segmentation of video using color, motion, and spatial information. In CVPR, 2001.

Crossref

Google Scholar

Cited By

View all

Yang WYang JSong H(2019)The Improved SIFT Algorithm Based on Rectangular Operator and Its Parallel ImplementationJournal of Information Technology Research10.4018/JITR.201901010112:1(1-17)Online publication date: Jan-2019
https://doi.org/10.4018/JITR.2019010101
Lu YLi YSong BZhang WChen HPeng L(2016)Parallelizing image feature extraction algorithms on multi-core platformsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2016.03.00192:C(1-14)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1016/j.jpdc.2016.03.001
Chen PYang DZhang WLi YZang BChen H(2012)Adaptive Pipeline Parallelism for Image Feature Extraction AlgorithmsProceedings of the 2012 41st International Conference on Parallel Processing10.1109/ICPP.2012.14(299-308)Online publication date: 10-Sep-2012
https://dl.acm.org/doi/10.1109/ICPP.2012.14

Index Terms

Interactive inquiry for object of interest in video playback by motion-augmented graph cut
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Interest point and salient region detections

Recommendations

Object-based queries using color points of interest
CBAIVL '01: Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01)

Image retrieval systems based on the query-by-exampleparadigm usually imply a global description of the image.An approach involving points of interest has been developedand comes to light with object/sub-image retrieval taskswhich require more local ...
Region-based image retrieval using an object ontology and relevance feedback

An image retrieval methodology suited for search in large collections of heterogeneous images is presented. The proposed approach employs a fully unsupervised segmentation algorithm to divide images into regions and endow the indexing and retrieval ...
Object of interest-based visual navigation, retrieval, and semantic content identification system
Special issue on color for image indexing and retrieval

This study presents a content-based image retrieval system IMALBUM based on local region of interest called object of interest (OOI). Each segmented or user-selected OOI is indexed with new local adapted descriptors associated to color, texture, and ...

Comments

Information & Contributors

Information

Published In

MM '10: Proceedings of the 18th ACM international conference on Multimedia

October 2010

1836 pages

ISBN:9781605589336

DOI:10.1145/1873951

General Chairs:
Alberto del Bimbo
University of Florence, Italy
,
Shih-Fu Chang
Columbia University, USA
,
Program Chair:
Arnold Smeulders
University of Amsterdam, NL

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

MM '10

Sponsor:

SIGMM

MM '10: ACM Multimedia Conference

October 25 - 29, 2010

Firenze, Italy

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
163
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yang WYang JSong H(2019)The Improved SIFT Algorithm Based on Rectangular Operator and Its Parallel ImplementationJournal of Information Technology Research10.4018/JITR.201901010112:1(1-17)Online publication date: Jan-2019
https://doi.org/10.4018/JITR.2019010101
Lu YLi YSong BZhang WChen HPeng L(2016)Parallelizing image feature extraction algorithms on multi-core platformsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2016.03.00192:C(1-14)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1016/j.jpdc.2016.03.001
Chen PYang DZhang WLi YZang BChen H(2012)Adaptive Pipeline Parallelism for Image Feature Extraction AlgorithmsProceedings of the 2012 41st International Conference on Parallel Processing10.1109/ICPP.2012.14(299-308)Online publication date: 10-Sep-2012
https://dl.acm.org/doi/10.1109/ICPP.2012.14

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Object-based queries using color points of interest

Region-based image retrieval using an object ontology and relevance feedback

Object of interest-based visual navigation, retrieval, and semantic content identification system