research-article

Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach

Authors:

James Z. WangAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 25, Issue 9

Pages 1075 - 1088

https://doi.org/10.1109/TPAMI.2003.1227984

Published: 01 September 2003 Publication History

Abstract

Automatic linguistic indexing of pictures is an important but highly challenging problem for researchers in computer vision and content-based image retrieval. In this paper, we introduce a statistical modeling approach to this problem. Categorized images are used to train a dictionary of hundreds of statistical models each representing a concept. Images of any given concept are regarded as instances of a stochastic process that characterizes the concept. To measure the extent of association between an image and the textual description of a concept, the likelihood of the occurrence of the image based on the characterizing stochastic process is computed. A high likelihood indicates a strong association. In our experimental implementation, we focus on a particular group of stochastic processes, that is, the two-dimensional multiresolution hidden Markov models (2D MHMMs). We implemented and tested our ALIP (Automatic Linguistic Indexing of Pictures) system on a photographic image database of 600 different concepts, each with about 40 training images. The system is evaluated quantitatively using more than 4,600 images outside the training database and compared with a random annotation scheme. Experiments have demonstrated the good accuracy of the system and its high potential in linguistic indexing of photographic images.

References

[1]

K. Barnard and D. Forsyth, “Learning The Semantics of Words and Pictures,” Proc. Int'l Conf. Computer Vision, vol 2, pp. 408-415, 2001.

[2]

A. Berman and L.G. Shapiro, “Efficient Image Retrieval with Multiple Distance Masures,” Proc. SPIE, vol. 3022, pp. 12-21, Feb. 1997.

[3]

R. Chellappa and A.K. Jain, Markov Random Fields: Theory and Applications. Academic Press, 1993.

[4]

Y. Chen and J.Z. Wang, “A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, Sept. 2002.

Digital Library

[5]

I. Daubechies, Ten Lectures on Wavelets. Capital City Press, 1992.

Digital Library

[6]

R.L. Dobrushin, “The Description of a Random Field by Means of Conditional Probabilities and Conditions of Its Regularity,” Theory Probability Application, vol. 13, pp. 197-224, 1968.

[7]

P. Duygulu K. Barnard and D. Forsyth, “Clustering Art,” Computer Vision and Pattern Recognition, vol. 2, pp. 434-439, 2001.

[8]

P. Duygulu K. Barnard N. de Freitas and D. Forsyth, “Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary,” Proc. European Conf. Computer Vision, vol. 4, pp. 97-112, 2002.

Digital Library

[9]

D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice Hall, 2002.

Digital Library

[10]

S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, Nov. 1984.

Digital Library

[11]

Q. Iqbal and J.K. Aggarwal, “Retrieval by Classification of Images Containing Large Manmade Objects Using Perceptual Grouping,” Pattern Recognition J., vol. 35, no. 7, pp. 1463-1479, 2002.

[12]

R. Kindermann and L. Snell, Markov Random Fields and Their Applications. Am. Math. Soc. 1980.

[13]

J. Li R.M. Gray and R.A. Olshen, “Multiresolution Image Classification by Hierarchical Modeling with Two Dimensional Hidden Markov Models,” IEEE Trans. Information Theory, vol. 46, no. 5, pp. 1826-41, Aug. 2000.

Digital Library

[14]

J. Li and R.M. Gray, Image Segmentation and Compression Using Hidden Markov Models. Kluwer Academic, 2000.

Digital Library

[15]

J. Li A. Najmi and R.M. Gray, “Image Classification by a Two Dimensional Hidden Markov Model,” IEEE Trans. Signal Processing, vol. 48, no. 2, pp. 517-33, Feb. 2000.

Digital Library

[16]

T.P. Minka and R.W. Picard, “Interactive Learning Using a Society of Models,” Pattern Recognition, vol. 30, no. 3, p. 565, 1997.

[17]

S. Ravela and R. Manmatha, “Image Retrieval by Appearance,” Proc. SIGIR, pp. 278-285, July 1997.

Digital Library

[18]

G. Sheikholeslami S. Chatterjee and A. Zhang, “WaveCluster: A Multi-Resolution Clustering Approach for Very Large Spatial Databases,” Proc. Very Large Date Bases Conf., pp. 428-439, Aug. 1998.

Digital Library

[19]

J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.

Digital Library

[20]

A.W.M. Smeulders M. Worring S. Santini A. Gupta and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22,no. 12, pp. 1349-1380, Dec. 2000.

Digital Library

[21]

M. Unser, “Texture Classification and Segmentation Using Wavelet Frames,” IEEE Trans. Image Processing, vol. 4, no. 11, pp. 1549-1560, Nov. 1995.

Digital Library

[22]

J.Z. Wang, Integrated Region-Based Image Retrieval. Dordrecht: Kluwer Academic, 2001.

Digital Library

[23]

J.Z. Wang J. Li and G. Wiederhold, “SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 947-963, Sept. 2001.

Digital Library

[24]

J.Z. Wang and M.A. Fischler, “Visual Similarity, Judgmental Certainty and Stereo Correspondence,” Proc. DARPA Image Understanding Workshop, G. Lukes, ed., vol. 2, pp. 1237-1248, Nov. 1998.

[25]

J.Z. Wang G. Wiederhold O. Firschein and X.W. Sha, “Content-Based Image Indexing and Searching Using Daubechies' Wavelets,” Int'l J. Digital Libraries (IJODL), vol. 1, no. 4, pp. 311-328, 1998.

[26]

Q. Zhang S.A. Goldman W. Yu and J.E. Fritts, “Content-Based Image Retrieval Using Multiple-Instance Learning,” Proc. Int'l Conf. Machine Learning, 2002.

Digital Library

[27]

S. Zhu and A.L. Yuille, “Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18,no. 9, pp. 884-900, Sept. 1996.

Digital Library

Cited By

Feng QLi PLu ZLi CWang ZLiu ZDuan CHuang FWeng JYu P(2024)EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud ComputingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337066834:8(7467-7483)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3370668
Li HZhu GZhang LJiang YDang YHou HShen PZhao XShah SBennamoun M(2024)Scene Graph GenerationNeurocomputing10.1016/j.neucom.2023.127052566:COnline publication date: 21-Jan-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.127052
Krishna AChaitra YBharadwaj AAbbas KAbraham APrasad A(2024)Real-time Machine Vision System for the Visually ImpairedSN Computer Science10.1007/s42979-024-02741-45:4Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1007/s42979-024-02741-4
Show More Cited By

Index Terms

Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach

Recommendations

Learning-based linguistic indexing of pictures with 2--d MHMMs
MULTIMEDIA '02: Proceedings of the tenth ACM international conference on Multimedia

Automatic linguistic indexing of pictures is an important but highly challenging problem for researchers in computer vision and content-based image retrieval. In this paper, we introduce a statistical modeling approach to this problem. Categorized ...
On Hierarchical Content-Based Image Retrieval by Dynamic Indexing and Guided Search

This paper presents a new approach to content-based image retrieval by using dynamic indexing and guided search in a hierarchical structure, and extending data mining and data warehousing techniques. The proposed algorithms include a wavelet-based scheme ...
Content-based image retrieval by clustering
MIR '03: Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval

In a typical content-based image retrieval (CBIR) system, query results are a set of images sorted by feature similarities with respect to the query. However, images with high feature similarities to the query may be very different from the query in ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 25, Issue 9

September 2003

146 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © Copyright © 2003 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 September 2003

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

300
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Feng QLi PLu ZLi CWang ZLiu ZDuan CHuang FWeng JYu P(2024)EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud ComputingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337066834:8(7467-7483)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3370668
Li HZhu GZhang LJiang YDang YHou HShen PZhao XShah SBennamoun M(2024)Scene Graph GenerationNeurocomputing10.1016/j.neucom.2023.127052566:COnline publication date: 21-Jan-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.127052
Krishna AChaitra YBharadwaj AAbbas KAbraham APrasad A(2024)Real-time Machine Vision System for the Visually ImpairedSN Computer Science10.1007/s42979-024-02741-45:4Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1007/s42979-024-02741-4
Ali L(2023)Generic Graphical User Interface for CBIR FrameworkApplied Computer Systems10.2478/acss-2023-002028:2(203-209)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.2478/acss-2023-0020
Varish NSingh PTugiti PManikanta MYedlapalli BPappusetty AThakkar HSharma G(2023)Color Image Retrieval Method Using Low Dimensional Salient Visual Feature Descriptors for IoT ApplicationsComputational Intelligence and Neuroscience10.1155/2023/62575732023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/6257573
Feng QLi PLu ZZhou ZWu YWeng JHuang F(2023)DHANApplied Soft Computing10.1016/j.asoc.2022.109935133:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.asoc.2022.109935
Salih FAbdulla A(2023)Two-layer content-based image retrieval technique for improving effectivenessMultimedia Tools and Applications10.1007/s11042-023-14678-682:20(31423-31444)Online publication date: 4-Mar-2023
https://dl.acm.org/doi/10.1007/s11042-023-14678-6
Liao Jbaoran li Wang JQi QLi TWang J(2022)Rapid Relevance Feedback Strategy Based on Distributed CBIR SystemInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.201804010114:2(1-26)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.4018/IJSWIS.2018040101
Shaheen FRaibagkar R(2022)Efficient Content-Based Image Retrieval System with Two-Tier Hybrid FrameworksApplied Computer Systems10.2478/acss-2022-001827:2(166-182)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.2478/acss-2022-0018
Xia ZJi QGu QYuan CXiao F(2022)A Format-compatible Searchable Encryption Scheme for JPEG Images Using Bag-of-wordsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/349270518:3(1-18)Online publication date: 15-Mar-2022
https://dl.acm.org/doi/10.1145/3492705
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents