Abstract
The problem of learning universally quantified function free first order Horn expressions is studied. Several models of learning from equivalence and membership queries are considered, including the model where interpretations are examples (Learning from Interpretations), the model where clauses are examples (Learning from Entailment), models where extensional or intentional background knowledge is given to the learner (as done in Inductive Logic Programming), and the model where the reasoning performance of the learner rather than identification is of interest (Learning to Reason). We present learning algorithms for all these tasks for the class of universally quantified function free Horn expressions. The algorithms are polynomial in the number of predicate symbols in the language and the number of clauses in the target Horn expression but exponential in the arity of predicates and the number of universally quantified variables. We also provide lower bounds for these tasks by way of characterising the VC-dimension of this class of expressions. The exponential dependence on the number of variables is the main gap between the lower and upper bounds.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Aizenstein, H., & Pitt, L. (1995). On the learnability of disjunctive normal form formulas. Machine Learning, 19, 183–208.
Angluin, D. (1998). Queries and concept learning. Machine Learning, 2(4), 319–342.
Angluin, D., Frazier, M., & Pitt, L. (1992). Learning conjunctions of Horn clauses. Machine Learning, 9, 147–164.
Arimura, H. (1997). Learning acyclic first-order Horn sentences from entailment. Proceedings of the International Conference on Algorithmic Learning Theory, Sendai, Japan, Springer-Verlag, LNAI 1316.
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M.K. (1989). Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4), 929–865.
Chang, C., & Keisler, J. (1990). Model Theory. Amsterdam, Holland: Elsevier.
Cohen,W. (1995a). PAC-learning recursive logic programs: Efficient algorithms. Journal of Artificial Intelligence Research, 2, 501–539.
Cohen, W. (1995b). PAC-learning recursive logic programs: Negative results. Journal of Artificial Intelligence Research, 2, 541–573.
De Raedt, L. (1997). Logical settings for concept learning. Artificial Intelligence, 95(1), 187–201. See also relevant Errata (forthcoming).
De Raedt, L., & Bruynooghe, M. (1992). An overview of the interactive concept learner and theory revisor CLINT. In S. Muggleton (Ed.), Inductive Logic Programming. Academic Press.
De Raedt, L., & Dzeroski, S. (1994). First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70, 375–392.
Del Val, A. (1996). Approximate knowledge compilation: the first order case. Proceedings of the National Conference on Artificial Intelligence, Portland, Oregon (pp. 498–503). AAAI Press.
Dzeroski, S., Muggleton, S., & Russell, S. (1992). PAC-learnability of determinate logic programs. Proceedings of the Conference on Computational Learning Theory, Pittsburgh, PA (pp. 128–135). ACM Press.
Frazier, M., & Pitt, L. (1993). Learning from entailment: An application to propositional Horn sentences. In Proceedings of the International Conference on Machine Learning, Amherst, MA (pp. 120–127). Morgan Kaufmann.
Frazier, M., & Pitt, L. (1996). CLASSIC learning. Machine Learning, 25, 151–193.
Geibel, P., & Wysotzki, F. (1997). A logical framework for graph theoretical decision tree learning. International Workshop on Inductive Logic Programming, Prague, Czech Republic (pp. 173–180). Springer. LNAI 1297.
Haussler, D. (1989). Learning conjunctive concepts in structural domains. Machine Learning, 4(1), 7–40.
Horn, A. (1951). On sentences which are true on direct unions of algebras. Journal of Symbolic Logic, 16(1), 14–21.
Horvath, T., Sloan, B., & Turán, G. (1997). Learning logic programs by using the product homomorphism method. Proceedings of the Conference on Computational Learning Theory, Nashville, Tennessee (pp. 10–20). ACM Press.
Horvath, T., & Turán, G. (1996). Learning logic programs with structured background knowledge. In L. De Raedt, (Ed.), Advances in Inductive Logic Programming. IOS Press.
Khardon, R. (1999a). Learning range-restricted Horn expressions. Proceedings of the Fourth European Conference on Computational Learning Theory, Nordkirchen, Germany (pp. 111–125). Springer-Verlag. LNAI 1572.
Khardon, R. (1999b). Learning to take actions. Machine Learning, 35(1), 57–90.
Khardon, R., & Roth, D. (1996). Reasoning with models. Artificial Intelligence, 87(1–2), 187–213.
Khardon, R., & Roth, D. (1997). Learning to reason. Journal of the ACM, 44(5), 697–725.
Lloyd, J. (1987). Foundations of Logic Programming, Springer Verlag. 2nd Edition.
Maass, W., & Turán, G. (1992). Lower bound methods and separation results for on-line learning models. Machine Learning, 9, 107–145.
McKinsey, J.C.C. (1943). The decision problem for some classes of sentences without quantifiers. Journal of Symbolic Logic, 8(3), 61–76.
Minker, J. (Ed.) (1988). Foundations of Deductive Databases and Logic Programming. Morgan Kaufmann.
Muggleton, S., & Buntine, W. (1992). Machine invention of first order predicates by inverting resolution. In S. Muggleton (Ed.), Inductive Logic Programming. Academic Press.
Muggleton, S., & De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 20, 629–679.
Muggleton, S., & Feng, C. (1992). Efficient induction of logic programs. In S. Muggleton (Ed.), Inductive Logic Programming. Academic Press.
Nienhuys-Cheng, S., & DeWolf, R. (1997). Foundations of Inductive Logic Programming. Springer-Verlag. LNAI 1228.
Page, D. (1993). Anti-Unification in Constraint Logics: Foundations and Applications to Learnability in First-Order Logic, to Speed-Up Learning, and to Deduction. Ph.D. thesis, Department of Computer Science, University of Illinois at Urbana-Champaign. Available as a Technical Report UIUCDCS-R–93–1820.
Papadimitriou, C.H., & Yannakakis, M. (1997). On the complexity of database queries. Proceedings of the symposium on Principles of Database Systems, Tucson, Arizona (pp. 12–19). ACM Press.
Plotkin, G.D. (1970). A note on inductive generalization. In B. Meltzer, & D. Michie (Eds.), Machine Intelligence 5. American Elsevier.
Rao, K., & Sattar, A. (1998). Learning from entailment of logic programs with local variables. Proceedings of the International Conference on Algorithmic Learning Theory, Otzenhausen, Germany. Springer-Verlag. LNAI 1501.
Reddy, C., & Tadepalli, P. (1997). Learning Horn definitions with equivalence and membership queries. International Workshop on Inductive Logic Programming, Prague, Czech Republic (pp. 243–255). Springer. LNAI 1297.
Reddy, C., & Tadepalli, P. (1998). Learning first order acyclic Horn programs from entailment. International Conference on Inductive Logic Programming, Madison, WI (pp. 23–37). Springer. LNAI 1446.
Reddy, C., Tadepalli, P., & Roncagliolo, S. (1996). Theory guided empirical speedup learning of goal decomposition rules. In International Conference on Machine Learning, Bari, Italy (pp. 409–416). Morgan Kaufmann.
Sagiv, Y. (1988). Optimizing datalog programs. In J. Minker (Eds.), Foundations of Deductive Databases and Logic Programming. Morgan Kaufmann.
Sammut, C., & Banerji, R. (1986). Learning concepts by asking questions. In R. Michalski, J. Carbonell, & T. Mitchell, (Eds.), Machine Learning: An AI Approach, (Vol. II). Morgan Kaufman.
Selman, B., & Kautz, H. (1996). Knowledge compilation and theory approximation. Journal of the ACM, 43(2), 193–224.
Shapiro, E. (1991). Inductive inference of theories from facts. In J. Lassez, & G. Plotkin (Eds.), Computational Logic. MIT Press.
Shapiro, E.Y. (1983). Algorithmic Program Debugging. Cambridge, MA: MIT Press.
Valiant, L.G. (1984). A theory of the learnable. Communications of the ACM, 27(11), 1134–1142.
Valiant, L.G. (1985). Learning disjunctions of conjunctions. In Proceedings of the International Joint Conference of Artificial Intelligence, Los Angeles, CA (pp. 560–566). Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Khardon, R. Learning Function-Free Horn Expressions. Machine Learning 37, 241–275 (1999). https://doi.org/10.1023/A:1007610422992
Issue Date:
DOI: https://doi.org/10.1023/A:1007610422992