Abstract
We are interested in distributions which are derived as a maximum entropy distribution from a given set of constraints. More specifically, we are interested in the case where the constraints are the expectation of individual and pairs of attributes. For such a given maximum entropy distribution (with some technical restrictions) we develop an efficient learning algorithm for read-once DNF. We extend our results to monotone read-k DNF following the techniques of (Hancock & Mansour, 1991).
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Angluin, D. & Kharitonov, M. (1991). When won't membership queries help? In Proceedings of the 23rd Annual ACM Symposium Theory Computing (pp. 444–454). New York: ACM.
Aslam, J. A. & Decatur, S. E. (1994). Improved nose-tolerant learning and generalized statistical queries. Tech. rep. TR-17-94, Harvard University.
Benedek, G. M. & Itai, A. (1991). Learnability with respect to fixed distributions. Theoretical Computer Science, 86(2), 377–390.
Cover, T. M. & Thomas, J. A. (1991). Elements of Information Theory. New York, NY: John Wiley and Sons Inc.
Hancock, T. & Mansour, Y. (1991). Learning monotone k? dnf formulas on product distributions. In Proceedings of the 2nd Annual Workshop on Computational Learning Theory (pp. 179–183). San Mateo, CA: Morgan Kaufmann.
Haussler, D., Kearns, M., Littlestone, N., & Warmuth, M. K. (1991). Equivalence of models for polynominal learnability. Information and Computation, 95(2), 129–161.
Kearns, M. (1993). Efficient noise-tolerant learning from statistical queries. In Proceedings of the 25th Annual ACM Symposium on Theory of Computing (pp. 392–401).
Kearns, M., Li, M., & Valiant, L. G. (1994). Learning boolean formulas. Journal of the ACM, 41(6), 1298–1328.
Kearns, M. & Valiant, L. G. (1989). Cryptographic limitations on learning boolean formulae and finite automata. In Proceedings of the 21st Annual ACM Symposium on Theory of Computing (pp. 433–444).
Kilian, J.& Naor,M. (1995). On the complexity of statistical reasoning. In Proceedings of the 3rd Israel Symposium on Theory of Computing and Systems (pp. 209–217).
Koller, D. & Megiddo, N. (1993). Constructing small sample spaces satisfying given constraints. In Proceedings of the 25th Annual ACM Symposium on Theory of Computing (pp. 268–277).
Natarajan, B. K. (1987). On learning boolean functions. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing (pp. 296–304). New York.
Papoulis, A. (1991). Probability, Random Variables, and Stochastic Processes (3rd edn.), ch. 15. New York, NY: McGraw-Hill.
Pitt, L. & Valiant, L. G. (1988). Computational limitations on learning from examples. Journal of the ACM, 35(4), 965–984.
Smith, R. C. & Grandy,W. (1985). Maximum-Entropy and Bayesian Methods in Inverse Problems. Dordrecht: D. Reidel Publishing Company.
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27(11), 1134–1142.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Mansour, Y., Schain, M. Learning with Maximum-Entropy Distributions. Machine Learning 45, 123–145 (2001). https://doi.org/10.1023/A:1010950718922
Issue Date:
DOI: https://doi.org/10.1023/A:1010950718922