Article

The rate adapting poisson model for information retrieval and object recognition

Authors:

Peter V. Gehler,

Max WellingAuthors Info & Claims

ICML '06: Proceedings of the 23rd international conference on Machine learning

Pages 337 - 344

https://doi.org/10.1145/1143844.1143887

Published: 25 June 2006 Publication History

Abstract

Probabilistic modelling of text data in the bag-of-words representation has been dominated by directed graphical models such as pLSI, LDA, NMF, and discrete PCA. Recently, state of the art performance on visual object recognition has also been reported using variants of these models. We introduce an alternative undirected graphical model suitable for modelling count data. This "Rate Adapting Poisson" (RAP) model is shown to generate superior dimensionally reduced representations for subsequent retrieval or classification. Models are trained using contrastive divergence while inference of latent topical representations is efficiently achieved through a simple matrix multiplication.

References

[1]

Airoldi, E., Cohen, W., & Fienberg, S. (2005). Bayesian methods for frequent terms in text. Proc. of the CSNA & INTERFACE Annual Meetings.]]

[2]

Blei, D. M., & Jordan, M. I. (2004). Variational inference for dirichlet process mixtures. Bayesian Analysis, 1, 121--144.]]

[3]

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993--1022.]]

Digital Library

[4]

Buntine, W. (Ed.). (2002). Variational extensions to em and multinomial pca, vol. 2430 of Lecture Notes in Computer Science. Helsinki, Finland: Springer.]]

Digital Library

[5]

Buntine, W., & Jakulin, A. (2004). Applying discrete pca in data analysis. Proceedings of the 20th conference on Uncertainty in artificial intelligence (pp. 59--66). Banff, Canada.]]

Digital Library

[6]

Carreira-Perpinan, M., & Hinton, G. (2005). On contrastive divergence learning. Tenth International Workshop on Artificial Intelligence and Statistics. Barbados.]]

[7]

Casella, G., & Robert, C. (1996). Rao-blackwellisation of sampling schemes. Biometrika, 83(1), 81--94.]]

[8]

Deerwester, S., Dumais, S., Landauer, T., Furnas, G., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41, 391--407.]]

[9]

Fergus, R., Fei-Fei, L., Perona, P., & Zisserman, A. (2005). Learning object categories from google's image search. Proceedings of the International Conference on Computer Vision.]]

Digital Library

[10]

Girolami, M., & Kaban, A. (2003). On an equivalence between PLSI and LDA. Proceedings of SIGIR 2003.]]

Digital Library

[11]

Griffiths, T., & Steyvers, M. (2002). A probabilistic approach to semantic representation. Proceedings of the 24th Annual Conference of the Cognitive Science Society.]]

[12]

Hinton, G. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771--1800.]]

Digital Library

[13]

Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief networks. Neural Computation. to appear.]]

Digital Library

[14]

Hofmann, T. (1999). Probabilistic latent semantic analysis. Proc. of Uncertainty in Artificial Intelligence, UAI '99. Stockholm.]]

[15]

Kadir, T., & Brady, M. (2001). Saliency, scale and image description. Int. J. Comput. Vision, 45, 83--105.]]

Digital Library

[16]

Lee, D., & Seung, H. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788--791.]]

[17]

Li, F., & Perona, P. (2005). A bayesian hierarchical model for learning natural scene categories. Proceedings of the Conference on Computer Vision and Pattern Recognition.]]

Digital Library

[18]

McCallum, A. (1996). Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. http://www.cs.cmu.edu/~mccallum/bow.]]

[19]

Minka, T., & Lafferty, J. (2002). Expectation-propogation for the generative aspect model. Proc. of the 18th Annual Conference on Uncertainty in Artificial Intelligence (pp. 352--359).]]

[20]

Olshausen, A., & Field, D. (1997). Sparse coding with over-complete basis set: A strategy employed by v1? Vision Research, 37, 3311--3325.]]

[21]

Roweis, S. (1997). Em algorithms for pca and spca. Neural Information Processing Systems (pp. 626--632).]]

Digital Library

[22]

Welling, M., & Hinton, G. (2001). A new learning algorithm for mean field Boltzmann machines. Proc. of the Int'l Conf. on Artificial Neural Networks. Madrid, Spain.]]

Digital Library

[23]

Welling, M., Hinton, G., & Osindero, S. (2002). Learning sparse topographic representations with products of student-t distributions. Neural Information Processing Systems.]]

[24]

Welling, M., Rosen-Zvi, M., & Hinton, G. (2004). Exponential family harmoniums with an application to information retrieval. Neural Information Processing Systems.]]

[25]

Xing, E., Yan, R., & Hauptman, A. (2005). Mining associated text and images with dual-wing harmoniums. Proc. of the Conf. on Uncertainty in Artificial Intelligence.]]

Cited By

Papadopoulos TTripoliti EGoletsis YBechlioulis ANaka KFotiadis D(2024)Bayesian Optimization in Restricted Boltzmann Machines for Heart Failure severity estimation2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC53108.2024.10782831(1-4)Online publication date: 15-Jul-2024
https://doi.org/10.1109/EMBC53108.2024.10782831
Aggarwal CAggarwal C(2023)Restricted Boltzmann MachinesNeural Networks and Deep Learning10.1007/978-3-031-29642-0_7(231-264)Online publication date: 30-Mar-2023
https://doi.org/10.1007/978-3-031-29642-0_7
Dabelow LUeda M(2022)Three learning stages and accuracy–efficiency tradeoff of restricted Boltzmann machinesNature Communications10.1038/s41467-022-33126-x13:1Online publication date: 17-Sep-2022
https://doi.org/10.1038/s41467-022-33126-x
Show More Cited By

Index Terms

The rate adapting poisson model for information retrieval and object recognition

Recommendations

A study of Poisson query generation model for information retrieval
SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

Many variants of language models have been proposed for information retrieval. Most existing models are based on multinomial distribution and would score documents based on query likelihood computed based on a query generation probabilistic model. In ...
Affine invariant topic model for generic object recognition
ISNN'10: Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part II

This paper presents a novel topic model named Affine Invariant Topic Model(AITM) for generic object recognition Abandoning the “bag of words” assumption in traditional topic models, AITM incorporates spatial structure into traditional LDA AITM extends ...
Discriminative information preservation for face recognition

It is usually difficult to find the optimal low dimensional subspace for face recognition. Patch alignment framework (PAF) is an important systematic framework that can be applied to understand the common thought and essential differences of a numerous ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '06: Proceedings of the 23rd international conference on Machine learning

June 2006

1154 pages

ISBN:1595933832

DOI:10.1145/1143844

Program Chairs:
William Cohen,
Andrew Moore

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
426
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)3

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Papadopoulos TTripoliti EGoletsis YBechlioulis ANaka KFotiadis D(2024)Bayesian Optimization in Restricted Boltzmann Machines for Heart Failure severity estimation2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC53108.2024.10782831(1-4)Online publication date: 15-Jul-2024
https://doi.org/10.1109/EMBC53108.2024.10782831
Aggarwal CAggarwal C(2023)Restricted Boltzmann MachinesNeural Networks and Deep Learning10.1007/978-3-031-29642-0_7(231-264)Online publication date: 30-Mar-2023
https://doi.org/10.1007/978-3-031-29642-0_7
Dabelow LUeda M(2022)Three learning stages and accuracy–efficiency tradeoff of restricted Boltzmann machinesNature Communications10.1038/s41467-022-33126-x13:1Online publication date: 17-Sep-2022
https://doi.org/10.1038/s41467-022-33126-x
Paranthaman MPalanivel Rajan S(2022)Augmented Intelligence: Deep Learning Models for HealthcareAugmented Intelligence in Healthcare: A Pragmatic and Integrated Analysis10.1007/978-981-19-1076-0_10(167-187)Online publication date: 20-Apr-2022
https://doi.org/10.1007/978-981-19-1076-0_10
Bulso NRoudi Y(2021)Restricted Boltzmann Machines as Models of Interacting VariablesNeural Computation10.1162/neco_a_01420(1-36)Online publication date: 19-Jul-2021
https://doi.org/10.1162/neco_a_01420
Dewi CChen RHendry Hung H(2021)Experiment Improvement of Restricted Boltzmann Machine Methods for Image ClassificationVietnam Journal of Computer Science10.1142/S219688882150018408:03(417-432)Online publication date: 19-Jan-2021
https://doi.org/10.1142/S2196888821500184
Tang JLi LLiu YLin K(2021)Automatic Identification of Bottleneck Tasks for Business Process Management using Fusion-based Text ClusteringIFAC-PapersOnLine10.1016/j.ifacol.2021.08.14254:1(1200-1205)Online publication date: 2021
https://doi.org/10.1016/j.ifacol.2021.08.142
Zhao HWu HLi JZhang HWang X(2020)Dtree2vec: A High-Accuracy and Dynamic Scheme for Real-Time Book Recommendation by Serialized Chapters and Local Fine-Grained PartitioningIEEE Access10.1109/ACCESS.2020.29682208(23197-23208)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2968220
Koo JKlabjan D(2020)Improved Classification Based on Deep Belief NetworksArtificial Neural Networks and Machine Learning – ICANN 202010.1007/978-3-030-61609-0_43(541-552)Online publication date: 14-Oct-2020
https://doi.org/10.1007/978-3-030-61609-0_43
Skabar A(2019)Restricted Boltzmann Machines: an Eigencentrality-based Approach2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8852267(1-8)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8852267
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents