Abstract
We study several complexity parameters for first order formulas and their suitability for first order learning models. We show that the standard notion of size is not captured by sets of parameters that are used in the literature and thus they cannot give a complete characterization in terms of learnability with polynomial resources. We then identify an alternative notion of size and a simple set of parameters that are useful for first order Horn Expressions. These parameters are the number of clauses in the expression, the maximum number of distinct terms in a clause, and the maximum number of literals in a clause. Matching lower bounds derived using the Vapnik Chervonenkis dimension complete the picture showing that these parameters are indeed crucial.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Angluin, D. (1988). Queries and Concept Learning. Machine Learning, 2(4), 319–342.
Arias, M., & Khardon, R. (2002). Learning Closed Horn Expressions. Information and Computation, 178, 214–240.
Arias, M., Khardon, R., & Servedio, R. A. (2003). Polynomial Certificates for Propositional Classes. In: Proceedings of the Conference on Computational Learning Theory (pp. 537–551). Springer-Verlag.
Arimura, H. (1997). Learning Acyclic First-order Horn Sentences from Entailment. In: Proceedings of the International Conference on Algorithmic Learning Theory. Springer-Verlag.
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989). Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4), 929–965.
Cohen, W. (1995). PAC-learning recursive logic programs: Efficient algorithms. Journal of Artificial Intelligence Research, 2.
De Raedt, L. (1997). Logical Settings for Concept Learning. Artificial Intelligence, 95(1), 187–201.
De Raedt, L., & Džeroski, S. (1994). First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70.
Džeroski, S., Muggleton, S., & Russell, S. (1992). PAC-learnability of Determinate Logic Programs. In: D. Haussler (Ed.) Proceedings of the Conference on Computational Learning Theory. (pp. 128–135). Pittsburgh, PA: ACM Press.
Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989). A General Lower Bound on the Number of Examples Needed for Learning. Information and Computation, 82(3).
Frazier, M., & Pitt, L. (1993). Learning from entailment: An application to propositional horn sentences’. In: Proceedings of the International Conference on Machine Learning. (pp. 120–127). Amherst, MA: Morgan Kaufmann.
Gottlob, G. (1987). Subsumption and implication. Information Processing Letters, 24(2), 109–111.
Grohe, M., & Turán, G. (2002). Learnability and Definability in Trees and Similar Structures. In: Proceedings of the 19th Annual Symposium on Theoretical Aspects of Computer Science (STACS) (pp. 645–658). Springer-Verlag. LNCS 2285.
Hegedűs, T. (1995) On Generalized Teaching Dimensions and the query complexity of learning. In: Proceedings of the Conference on Computational Learning Theory. (pp. 108–117). New York, NY, USA: ACM Press.
Hellerstein, L., Pillaipakkamnatt, K., Raghavan, V., & Wilkins, D. (1996). How Many Queries are Needed to Learn? Journal of the ACM, 43(5), 840–862.
Horváth, T., & Turán, G. (2001). Learning logic programs with structured background knowledge. Artificial Intelligence, 128(1–2), 31–97.
Kearns, M., & Vazirani, U. (1994). An Introduction to computational learning theory. Cambridge: MIT Press.
Khardon, R. (1999a). Learning Function Free Horn Expressions. Machine Learning, 37.
Khardon, R. (1999b). Learning Range Restricted Horn Expressions. In: Proceedings of the Fourth European Conference on Computational Learning Theory (pp. 111–125), Springer-Verlag.
Kietz, J. U., & Džeroski, S. (1994). Inductive logic programming and learnability. SIGART Bulletin, 5(1), 22–32.
Krishna Rao, M., & Sattar, A. (1998). Learning from entailment of logic programs with local variables. In: Proceedings of the International Conference on Algorithmic Learning Theory. Springer-Verlag.
Lloyd, J. W. (1987). Foundations of logic programming; (2nd extended ed.). Springer-Verlag.
Maass, W., & Turán, G. (1992). Lower bound methods and separation results for online learning models. Machine Learning, 9, 107–145.
Maass, W., & Turán, G. (1995). On learnability and predicate logic (extended abstract). In: Proceedings of the 4th Bar-Ilan Symposium on Foundations of AI (BISFAI).
Muggleton, S., & De Raedt, L. (1994). Inductive Logic Programming: Theory and methods. The Journal of Logic Programming, 19 & 20, 629–680.
Muggleton, S., & Feng, C. (1992). Efficient Induction of Logic Programs. In: S. Muggleton (Ed.) Inductive Logic Programming. (pp. 281–298) Academic Press.
Reddy, C., & Tadepalli, P. (1997). Learning Horn Definitions with Equivalence and Membership Queries. In: International Workshop on Inductive Logic Programming. (pp. 243–255). Prague, Czech Republic: Springer-Verlag.
Reddy, C., & Tadepalli, P. (1998). Learning First Order Acyclic Horn Programs from Entailment. In: International Conference on Inductive Logic Programming. (pp. 23–37), Madison, WI: Springer-Verlag.
Vapnik, V., & Chervonenkis, A. (1971). On the Uniform Convergence of Relative Frequencies of Events to their probabilities. Theory of Probability and its Applications, 16(2), 264–280.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been partly supported by NSF Grant IIS-0099446. A preliminary version of this paper appeared in the proceeding of the conference on Inductive Logic Programming 2003.
Most of this work was done while M.A. was at Tufts University.
Editors: Tamás Horváth and Akihiro Yamamoto
Rights and permissions
About this article
Cite this article
Arias, M., Khardon, R. Complexity parameters for first order classes. Mach Learn 64, 121–144 (2006). https://doi.org/10.1007/s10994-006-8261-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-006-8261-3