poster

Leveraging probabilistic season and location context models for scene understanding

Authors:

Jiebo LuoAuthors Info & Claims

CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

Pages 169 - 178

https://doi.org/10.1145/1386352.1386379

Published: 07 July 2008 Publication History

Abstract

Recent research has shown the power of context-aware scene understanding in bridging the semantic gap between high-level semantic concepts and low-level image features. In this paper, we present a new method to exploit nonvisual context information from the season and location proximity in which pictures were taken to facilitate region (object) annotation in consumer photos. Our method does not require precise time and location from the capture device or user input. Instead, it learns from rough location (e.g., states in the US) and time (e.g., seasons) information, which can be obtained through picture metadata automatically or through minimal user input (e.g., grouping). In addition, the visual context within the image is obtained by analyzing the spatial relationships between different regions (objects) in the scene. Both visual and nonvisual context information are fused using a probabilistic graphical model to improve the accuracy of object region recognition. Our method has been evaluated on a database that consists of over 10,000 regions in more than 1000 images collected from both the Web and consumers. Experimental results show that incorporating the season and location context significantly improves the performance of region recognition.

References

[1]

Berretti, S., Bimbo, A. D., and Vicario, E. 2002. Spatial arrangement of color in retrieval by visual similarity. Pattern Recog. 35 (2002) 1661--1674.

[2]

Boutell, M., Luo, J., and Brown, C. M. 2004. Learning spatial configuration models using modified Dirichlet priors. In Proceedings of the Workshop on Statistical Relational Learning (in Conjunction with ICML) (2004).

[3]

Boutell, M., Choudhury, A., Luo, J., and Brown, C. M. 2006. Using semantic features for scene classification: How good do they need to be? In Proceedings of the IEEE International Conference on Multimedia and Exposition. (Jul. 2006).

[4]

Boutell, M., Luo, J., and Brown, C. 2007. Scene parsing using region-based generative models. IEEE Trans. Multimedia.

Digital Library

[5]

Chen, Y. and Wang, J. Z. 2004. Image categorization by learning and reasoning with regions. J. Machine Learn. Res. 5 (2004) 913--939.

Digital Library

[6]

Christoudias, C., Georgescu, B., and Meer, P. 2002. Synergism in low-level vision. In Proceedings of the 16th International Conference on Pattern Recognition (Aug. 2002).

Digital Library

[7]

Comanicu, D. and Meer, P. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine Intell. 24 (May 2002) 603--619.

Digital Library

[8]

Datta, R., Joshi, D., Li, J., and Wang, J. A. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys.

Digital Library

[9]

Flickr, http://www.flickr.com

[10]

Haering, N. and da Vitoria Lobo, N. 1999. Features and classification methods to locate deciduous trees in images. Comput. Vis. Image Understand. 75(1/2) (Jul.-Aug. 1999) 133--149.

Digital Library

[11]

Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence (1995).

Digital Library

[12]

Kschischang, F. R., Frey, B. H., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm. IEEE Trans. Info. Theory. 47(2) (Feb. 2001) 498--519.

Digital Library

[13]

Kumar, S. and Hebert, M. 2005. A hierarchical field framework for unified context-based classification. In Proceedings of the International Conference on Computer Vision (2005).

Digital Library

[14]

Meer, P. and Georgescu. B. Edge detection with embedded confidence. IEEE Trans. Pattern Anal. Machine Intell. 23 (Dec. 2001) 1351--1365.

Digital Library

[15]

Murphy, K., Torralba, A., and Freeman, W. T. 2003. Using the forest to see the trees: A graphical model relating features, objects, and scenes. In Proceedings of the Neural Information Processing Systems (2003).

[16]

Naphade, M. R. and Huang, T. S. 2001. A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans. Multimedia 3(1) (Mar. 2001) 141--151.

Digital Library

[17]

Naphade, M. and Huang, T. S. 2002. A factor graph framework for semantic video indexing. IEEE Trans. Circ. Syst. Video Technol. 12(1) (Jan. 2002).

Digital Library

[18]

Picasa, http://picasa.google.com/

[19]

Rui, Y., Huang, T. S., and Chang, S. F. 1999. Image retrieval: Current techniques, promising directions and open issues. J. Visual Commun. Image Represent. (1999).

[20]

Saber, E., Tekalp, A. M., Eschbach, R., and Knox, K. 1996. Automatic image annotation using adaptive color classification. CVGIP: Graphic. Models Imag. Proc. 58 (1996) 115--126.

Digital Library

[21]

Singhal, A., Luo, J., and Zhu, W. Probabilistic spatial context models for scene content understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Madison, WI, Jun. 2003).

Digital Library

[22]

Smeulders, A. W. M. et al., Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Machine Intell. 22 (2000) 1349--1380.

Digital Library

[23]

Smith, J. R. and Li, C.-S. 1998. Decoding image semantics using composite region templates. In Proceedings of the IEEE International Workshop on Content-based Access of Image and Video Database (1998).

Digital Library

[24]

Vailaya, A. and Jain, A. 2000. "Detecting sky and vegetation in outdoor images," In Proceedings of the SPIE. 3972 (2000).

[25]

Wang, W., Song, Y., and Zhang, A. 2002. Semantics retrieval by content and context of image regions. In Proceedings of the 15th International Conference on Vision Interface. (2002).

[26]

Yuan, J., Li, J., and Zhang, B. 2007. Exploiting spatial context constraints for automatic image region annotation. In Proceedings of the ACM International Conf. On Multimedia. (2007).

Digital Library

Cited By

Arbinger CBullin MHenrich A(2022)Exploiting Geodata to Improve Image Recognition with Deep LearningCompanion Proceedings of the Web Conference 202210.1145/3487553.3524645(648-655)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3487553.3524645
Armitage JKacupaj ETahmasebzadeh GSwati Maleshkova MEwerth RLehmann Jd'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)MLMProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412783(2967-2974)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412783
Chu GPotetz BWang WHoward ASong YBrucher FLeung TAdam H(2019)Geo-Aware Networks for Fine-Grained Recognition2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2019.00033(247-254)Online publication date: Oct-2019
https://doi.org/10.1109/ICCVW.2019.00033
Show More Cited By

Index Terms

Leveraging probabilistic season and location context models for scene understanding
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Search engine architectures and scalability
      1. Search engine indexing

Recommendations

Understanding spatial contexts of the real world under explicit or tacit roles of location
OzCHI '13: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration

To realize context-aware services in a ubiquitous computing environment, it is essential to extract the spatial structure of non-linguistic context information that involves a person at a certain location in the real world. With the techniques of ...
Visual vocabulary optimization with spatial context for image annotation and classification
MMM'12: Proceedings of the 18th international conference on Advances in Multimedia Modeling

In this paper, we propose a new approach of visual vocabulary optimization with spatial context, which contains important spatial information that has not been fully exploited. The novelty of our method mainly lies in two aspects: when spatial ...
Combining Feature Context and Spatial Context for Image Pattern Discovery
ICDM '11: Proceedings of the 2011 IEEE 11th International Conference on Data Mining

Once an image is decomposed into a number of visual primitives, e.g., local interest points or salient image regions, it is of great interests to discover meaningful visual patterns from them. Conventional clustering (e.g., k-means) of visual primitives, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIVR '08: Proceedings of the 2008 international conference on Content-based image and video retrieval

July 2008

674 pages

ISBN:9781605580708

DOI:10.1145/1386352

General Chairs:
Jiebo Luo
Kodak Research Laboratories
,
Ling Guan
Ryerson University
,
Program Chairs:
Alan Hanjalic
Delft University of Technology
,
Mohan Kankanhalli
National University of Singapore
,
Ivan Lee
University of South Australia

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIVR08

Sponsor:

CIVR08: CIVR'08 - International Conference on Content-based Image and Video Retrieval

July 7 - 9, 2008

Niagara Falls, Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
386
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Arbinger CBullin MHenrich A(2022)Exploiting Geodata to Improve Image Recognition with Deep LearningCompanion Proceedings of the Web Conference 202210.1145/3487553.3524645(648-655)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3487553.3524645
Armitage JKacupaj ETahmasebzadeh GSwati Maleshkova MEwerth RLehmann Jd'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)MLMProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412783(2967-2974)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412783
Chu GPotetz BWang WHoward ASong YBrucher FLeung TAdam H(2019)Geo-Aware Networks for Fine-Grained Recognition2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2019.00033(247-254)Online publication date: Oct-2019
https://doi.org/10.1109/ICCVW.2019.00033
Jaffali SJamoussi SKhelifi NHamadou A(2019)Survey on Social Networks Data AnalysisInnovations for Community Services10.1007/978-3-030-37484-6_6(100-119)Online publication date: 15-Dec-2019
https://doi.org/10.1007/978-3-030-37484-6_6
Guo WWu RChen YZhu X(2018)Deep Learning Scene Recognition Method Based on Localization EnhancementSensors10.3390/s1810337618:10(3376)Online publication date: 10-Oct-2018
https://doi.org/10.3390/s18103376
Fan Y(2016)Improved scene classification using region semantics and spatial context information2016 19th International Conference on Computer and Information Technology (ICCIT)10.1109/ICCITECHN.2016.7860239(443-450)Online publication date: Dec-2016
https://doi.org/10.1109/ICCITECHN.2016.7860239
Spyrou EMylonas P(2016)A survey on Flickr multimedia research challengesEngineering Applications of Artificial Intelligence10.1016/j.engappai.2016.01.00651:C(71-91)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1016/j.engappai.2016.01.006
Li LJha RThomee BShamma DCao LWang Y(2016)Where the Photos Were Taken: Location Prediction by Learning from Flickr PhotosLarge-Scale Visual Geo-Localization10.1007/978-3-319-25781-5_3(41-58)Online publication date: 6-Jul-2016
https://doi.org/10.1007/978-3-319-25781-5_3
Yanai K(2015)[Invited Paper] A Review of Web Image MiningITE Transactions on Media Technology and Applications10.3169/mta.3.1563:3(156-169)Online publication date: 2015
https://doi.org/10.3169/mta.3.156
Tang KPaluri MFei-Fei LFergus RBourdev L(2015)Improving Image Classification with Location Context2015 IEEE International Conference on Computer Vision (ICCV)10.1109/ICCV.2015.121(1008-1016)Online publication date: Dec-2015
https://doi.org/10.1109/ICCV.2015.121
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents