article

Learning DFA from Simple Examples

Authors:

Vasant G. HonavarAuthors Info & Claims

Machine Learning, Volume 44, Issue 1-2

Pages 9 - 35

https://doi.org/10.1023/A:1010822518073

Published: 01 July 2001 Publication History

Abstract

Efficient learning of DFA is a challenging research problem in grammatical inference. It is known that both exact and approximate (in the PAC sense) identifiability of DFA is hard. Pitt has posed the following open research problem: “Are DFA PAC-identifiable if examples are drawn from the uniform distribution, or some other known simple distribution?” (Pitt, in Lecture Notes in Artificial Intelligence, 397, pp. 18–44, Springer-Verlag, 1989). We demonstrate that the class of DFA whose canonical representations have logarithmic Kolmogorov complexity is efficiently PAC learnable under the Solomonoff Levin universal distribution (m). We prove that the class of DFA is efficiently learnable under the PACS (PAC learning with simple examples) model (Denis, D'Halluin & Gilleron, STACS'96—Proceedings of the 13th Annual Symposium on the Theoretical Aspects of Computer Science, pp. 231–242, 1996) wherein positive and negative examples are sampled according to the universal distribution conditional on a description of the target concept. Further, we show that any concept that is learnable under Gold's model of learning from characteristic samples, Goldman and Mathias' polynomial teachability model, and the model of learning from example based queries is also learnable under the PACS model.

References

[1]

Angluin, D. (1981). A note on the number of queries needed to identify regular languages. Information and Control, 51, 76-87.

[2]

Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and Computation, 75, 87-106.

Digital Library

[3]

Angluin, D. (1988). Queries and concept learning. Machine Learning, 2:4, 319-342.

[4]

Castro, J., & Guijarro, D. (1998). Query, pacs and simple-pac learning. Technical Report LSI-98-2-R, Universitat Polytéctica de Catalunya, Spain.

[5]

Chomsky, N. (1956). Three models for the description of language. PGIT, 2:3, 113-124.

[6]

Denis, F., D'Halluin, C., & Gilleron, R. (1996). Pac learning with simple examples. STACS'96--Proceedings of the 13th Annual Symposium on the Theoretical Aspects of Computer Science (pp. 231-242).

Digital Library

[7]

Denis, F., & Gilleron, R. (1997). Pac learning under helpful distributions. In Proceedings of the Eighth International Workshop on Algorithmic Learning Theory (ALT'97), Lecture Notes in Artificial Intelligence 1316 (pp. 132- 145), Sendai, Japan.

Digital Library

[8]

Dupont, P. (1996). Incremental regular inference. In L. Miclet, & C. Higuera, (Eds.), Proceedings of the Third ICGI-96, Lecture Notes in Artificial Intelligence 1147 (pp. 222-237), Montpellier, France, Springer.

[9]

Dupont, P. (1996). Utilisation et apprentissage de modèles de language pour la reconnaissance de la parole continue. PhD thesis, Ecole Normale Supérieure des Télécommunications, Paris, France.

[10]

Dupont, P., Miclet, L., & Vidal, E. (1994). What is the search space of the regular inference? In Proceedings of the Second International Colloquium on Grammatical Inference (ICGI'94) (pp. 25-37). Alicante, Spain.

Digital Library

[11]

Gold, E. (1978). Complexity of automaton identification from given data. Information and Control, 37:3, 302-320.

[12]

Goldman, S., & Mathias, H. (1993). Teaching a smarter learner. In Proceedings of the Workshop on Computational Learning Theory (COLT'93) (pp. 67-76). ACM Press.

Digital Library

[13]

Goldman, S., & Mathias, H (1996). Teaching a smarter learner. Journal of Computer and System Sciences, 52, 255-267.

Digital Library

[14]

Colin de la Higuera (1996). Characteristic sets for polynomial grammatical inference. In L. Miclet, & C. Higuera, (Eds.), Proceedings of the Third ICGI-96, Lecture Notes in Artificial Intelligence 1147 (pp. 59-71). Montpellier, France, Springer.

[15]

Hopcroft, J., & Ullman, J. (1979). Introduction to automata theory, languages, and computation. Reading, MA: Addison Wesley.

Digital Library

[16]

Jackson, J., & Tomkins, A. (1992). A computational model of teaching. In Proceedings of the Workshop on Computational Learning Theory (COLT'92) (pp. 319-326). ACM Press.

Digital Library

[17]

Kearns, M., & Valiant, L. G. (1989). Cryptographic limitations on learning boolean formulae and finite automata. In Proceedings of the 21st Annual ACM Symposium on Theory of Computing (pp. 433-444). New York: ACM.

Digital Library

[18]

Lang, K. (1992). Random DFAs can be approximately learned from sparse uniform sample. In Proceedings of the 5th ACM workshop on Computational Learning Theory (pp. 45-52).

Digital Library

[19]

Li, M., & Vitányi, P. (1991). Learning simple concepts under simple distributions. SIAM Journal of Computing, 20:5, 911-935.

Digital Library

[20]

Li, M., & Vitányi, P. (1997). An introduction to Kolmogorov complexity and its applications, (2nd ed.) New York: Springer Verlag.

Digital Library

[21]

Oncina, J., & Garcia, P. (1992). Inferring regular languages in polynomial update time. In N. Pérez et al. (eds.), Pattern recognition and image analysis (pp. 49-61). Singapore: World Scientific.

[22]

Pao, T., & Carr, J. (1978). A solution of the syntactic induction-inference problem for regular languages. Computer Languages, 3, 53-64.

Digital Library

[23]

Parekh, R., & Honavar, V. (1993). Efficient learning of regular languages using teacher supplied positive examples and learner generated queries. In Proceedings of the Fifth UNB Conference on AI (pp. 195-203). Fredricton, Canada.

[24]

Parekh, R., & Honavar, V. (1997). Learning DFA from simple examples. In Proceedings of the Eighth International Workshop on Algorithmic Learning Theory (ALT'97), Lecture Notes in Artificial Intelligence 1316 (pp. 116- 131). Sendai, Japan, Springer. Also presented at the Workshop on Grammar Inference, Automata Induction, and Language Acquisition (ICML'97), Nashville, TN, July 12, 1997.

Digital Library

[25]

Parekh, R & Honavar, V. (1999). Simple DFA are polynomially probably exactly learnable from simple examples. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML'99) (pp. 298-306). Bled, Slovenia.

Digital Library

[26]

Pitt, L. (1989). Inductive inference, DFAs and computational complexity. In Analogical and Inductive Inference, Lecture Notes in Artificial Intelligence, 397 (pp. 18-44). Springer-Verlag.

Digital Library

[27]

Pitt, L., & Warmuth, M. K. (1988). Reductions among prediction problems: on the difficulty of predicting automata. In Proceedings of the 3rd IEEE Conference on Structure in Complexity Theory (pp. 60-69).

[28]

Pitt, L., & Warmuth, M. K. (1989). The minimum consistency DFA problem cannot be approximated within any polynomial. In Proceedings of the 21st ACM Symposium on the Theory of Computing (pp. 421-432). ACM.

Digital Library

[29]

Rivest, R. L. & Schapire, R. E. (1993). Inference of finite automata using homing sequences. Information and Computation, 103:2, 299-347.

Digital Library

[30]

Trakhtenbrot, B., & Barzdin, Ya. (1973). Finite Automata: Behavior and Synthesis. Amsterdam, North Holland.

[31]

Valiant, L. (1984). A theory of the learnable. Communications of the ACM, 27, 1134-1142.

Digital Library

Cited By

Chen QBanerjee ADemiralp ÇDurrett GDillig I(2023)Data Extraction via Semantic Regular Expression SynthesisProceedings of the ACM on Programming Languages10.1145/36228637:OOPSLA2(1848-1877)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622863
Lingg Jde Oliveira Oliveira MWolf P(2022)Learning from Positive and Negative Examples: Dichotomies and Parameterized AlgorithmsCombinatorial Algorithms10.1007/978-3-031-06678-8_29(398-411)Online publication date: 7-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-06678-8_29
Bhagwan RMehta SRadhakrishna AGarg SGrundy J(2021)Learning patterns in configurationProceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE51524.2021.9678525(817-828)Online publication date: 15-Nov-2021
https://dl.acm.org/doi/10.1109/ASE51524.2021.9678525
Show More Cited By

Index Terms

Learning DFA from Simple Examples
1. Computing methodologies
  1. Machine learning
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Inductive inference

Recommendations

The complexity of properly learning simple concept classes

We consider the complexity of properly learning concept classes, i.e. when the learner must output a hypothesis of the same form as the unknown concept. We present the following new upper and lower bounds on well-known concept classes:*We show that ...
Efficient learning algorithms yield circuit lower bounds

We describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin's model of exact learning from membership and ...
Learning Regular Languages from Simple Positive Examples

Learning from positive data constitutes an important topic in Grammatical Inference since it is believed that the acquisition of grammar by children only needs syntactically correct (i.e. positive) instances. However, classical learning models provide ...

Comments

Information & Contributors

Information

Published In

cover image Machine Language

Machine Language Volume 44, Issue 1-2

July-August 2001

186 pages

ISSN:0885-6125

Issue’s Table of Contents

Copyright © Copyright © 2001 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 July 2001

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen QBanerjee ADemiralp ÇDurrett GDillig I(2023)Data Extraction via Semantic Regular Expression SynthesisProceedings of the ACM on Programming Languages10.1145/36228637:OOPSLA2(1848-1877)Online publication date: 16-Oct-2023
Lingg Jde Oliveira Oliveira MWolf P(2022)Learning from Positive and Negative Examples: Dichotomies and Parameterized AlgorithmsCombinatorial Algorithms10.1007/978-3-031-06678-8_29(398-411)Online publication date: 7-Jun-2022
Bhagwan RMehta SRadhakrishna AGarg SGrundy J(2021)Learning patterns in configurationProceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE51524.2021.9678525(817-828)Online publication date: 15-Nov-2021
Chen QWang XYe XDurrett GDillig IDonaldson ATorlak E(2020)Multi-modal synthesis of regular expressionsProceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3385412.3385988(487-502)Online publication date: 11-Jun-2020
Zhang C(2020)Minimal consistent DFA from sample stringsActa Informatica10.1007/s00236-020-00365-857:3-5(657-670)Online publication date: 1-Oct-2020
Kowalski JKisielewicz A(2018)Regular Language Inference for Learning Rules of Simplified Boardgames2018 IEEE Conference on Computational Intelligence and Games (CIG)10.1109/CIG.2018.8490435(1-8)Online publication date: 14-Aug-2018
Pina LGrumberg DAndronidis ACadar CSilva DFord B(2017)A DSL approach to reconcile equivalent divergent program executionsProceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference10.5555/3154690.3154730(417-429)Online publication date: 12-Jul-2017
Lee MSo SOh H(2016)Synthesizing regular expressions from examples for introductory automata assignmentsACM SIGPLAN Notices10.1145/3093335.299324452:3(70-80)Online publication date: 20-Oct-2016
Lee MSo SOh HFischer BSchaefer I(2016)Synthesizing regular expressions from examples for introductory automata assignmentsProceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/2993236.2993244(70-80)Online publication date: 20-Oct-2016
Fischer BSchaefer I(2016)Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesundefinedOnline publication date: 20-Oct-2016
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents