Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Exact Learning from Membership Queries: Some Techniques, Results and New Directions

  • Conference paper
Algorithmic Learning Theory (ALT 2013)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8139))

Included in the following conference series:

Abstract

Given a black box that contains a function f:D → R from some class of functions C. The black box can receive an element d (query) of the domain D and in time T returns the value f(d) ∈ R. Our goal is to exactly find (exactly learn) f with minimum number of queries and optimal time complexity. Or at least decide whether f ≡ g for some function g ∈ C.

This problem has different names in different areas: Interpolation, Exactly Learning, Inferring, Identifying, Active Learning, Guessing Game, Testing, Functional Verification, Hitting Set and Black Box PIT from Substitution or Membership Queries.

In this survey we give some of the results known from the literature, different techniques used mainly for the problem of exact learning and new directions that we think are worth investigating.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Aigner, M.: Combinatorial Search. Wiley Teubner Series on Applicable Theory in Computer Science. Teubner, Stuttgart (1988)

    MATH  Google Scholar 

  2. Angluin, D.: Queries and Concept Learning. Machine Learning 2(4), 319–342 (1987)

    Google Scholar 

  3. Angluin, D.: Learning Regaular Sets from Queries and Counterexamples. Information and Computation 75, 87–106 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  4. Abboud, E., Agha, N., Bshouty, N.H., Radwan, N., Saleh, F.: Learning Threshold Functions with Small Weights Using Membership Queries. In: COLT 1999, pp. 318–322 (1999)

    Google Scholar 

  5. Abasi, H., Bshouty, N.H.: On Exact Learning DNF from Membership Queries (in preperation)

    Google Scholar 

  6. Alekhnovich, M., Braverman, M., Feldman, V., Klivans, A.R., Pitassi, T.: The Complexity of Properly Learning Simple Concept classes. J. Comput. Syst. Sci. 74(1), 16–34 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Angluin, D., Frazier, M., Pitt, L.: Learning Conjunctions of Horn Clauses. Machine Learning 9, 147–164 (1992)

    MATH  Google Scholar 

  8. Aizenstein, H., Hellerstein, L., Pitt, L.: Read-Thrice DNF Is Hard to Learn With Membership and Equivalence Queries. In: FOCS 1992, pp. 523–532 (1992)

    Google Scholar 

  9. Angluin, D., Krikis, M., Sloan, R.H., Turán, G.: Malicious Omissions and Errors in Answers to Membership Queries. Machine Learning 28(2-3), 211–255 (1997)

    Article  MATH  Google Scholar 

  10. Alon, N., Moshkovitz, D., Safra, S.: Algorithmic construction of sets for k-restrictions. ACM Transactions on Algorithms 2(2), 153–177 (2006)

    Article  MathSciNet  Google Scholar 

  11. Anderson, M., van Melkebeek, D., Volkovich, I.: Derandomizing Polynomial Identity Testing for Multilinear Constant-Read Formulae. In: CCC 2011, pp. 273–282 (2011)

    Google Scholar 

  12. Aizenstein, H., Pitt, L.: Exact Learning of Read-Twice DNF Formulas. In: FOCS 1991, pp. 170–179 (1991)

    Google Scholar 

  13. Angluin, D., Slonim, D.K.: Randomly Fallible Teachers: Learning Monotone DNF with an Incomplete Membership Oracle. Machine Learning 14(1), 7–26 (1994)

    MATH  Google Scholar 

  14. Agrawal, M., Saptharishi, R.: Classifying polynomials and identity testing. Current Trends in Science (2009), http://www.cse.iitk.ac.in/users/manindra/survey/Identity.pdf.3

  15. Agrawal, M., Saha, C., Saxena, N.: Quasi-polynomial Hitting-set for Set-depth-Δ Formulas. In: STOC 2013, pp. 321–330 (2013)

    Google Scholar 

  16. Agrawal, M., Vinay, V.: Arithmetic Circuits: A Chasm at Depth Four. In: FOCS 2008, pp. 67–75 (2008)

    Google Scholar 

  17. Bogdanov, A.: Pseudorandom Generators for Low Degree Polynomials. In: STOC 2005, pp. 21–30 (2005)

    Google Scholar 

  18. Bshouty, N.H.: Exact Learning Boolean Function via the Monotone Theory. Inf. Comput. 123(1), 146–153 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  19. Bshouty, N.H.: Simple Learning Algorithms Using Divide and Conquer. Computational Complexity 6(2), 174–194 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  20. Bshouty, N.H.: Exact Learning of Formulas in Parallel. Machine Learning 26(1), 25–41 (1997)

    Article  MATH  Google Scholar 

  21. Bshouty, N.H.: On the Coin Weighing Problem with the Presence of Noise. In: APPROX-RANDOM 2012, pp. 471–482 (2012)

    Google Scholar 

  22. Bshouty, N.H.: Testers and their Applications. Electronic Collouium on Computational Complexity (ECCC) 19, 11 (2012)

    Google Scholar 

  23. Bshouty, N.H.: Multilinear Complexity is Equivalent to Optimal Tester Size. Electronic Collouium on Computational Complexity (ECCC) 20, 11 (2013)

    Google Scholar 

  24. Bshouty, N.H.: Dense Testers and Their Applications (in preperation)

    Google Scholar 

  25. Bshouty, N.H.: Non-adaptive Deterministic Learning XOR of Terms and Decision Tree from Membership Queries (in preperation)

    Google Scholar 

  26. Bshouty, D., Bshouty, N.H.: On Interpolating Arithmetic Read-Once Formulas with Exponentiation. J. Comput. Syst. Sci. 56(1), 112–124 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  27. Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning Functions Represented as Multiplicity Automata. J. ACM 47(3), 506–530 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  28. Bisht, L., Bshouty, N.H., Khoury, L.: Learning with Errors in Answering to Memebership Queries. J. Comput. Syst. Sci. 74(1), 2–15 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  29. Bshouty, N.H., Cleve, R.: Interpolating Arithmetic Read-Once Formulas in Parallel. SIAM J. Comput. 27(2), 401–413 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  30. Bshouty, N.H., Eiron, N.: Learning Monotone DNF from a Teacher that Almost Does Not Answer Membership Queries. JMLR 3, 49–57 (2002)

    MathSciNet  Google Scholar 

  31. Bshouty, N.H., Eiron, N., Kushilevitz, E.: PAC Learning with Nasty Noise. Theor. Comput. Sci. 288(2), 255–275 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  32. Biglieri, E., Gyorfi, L.: Multiple Access Channels: Theory and Practice. IOS Press (2007)

    Google Scholar 

  33. Bshouty, N.H., Goldman, S.A., Hancock, T.R., Matar, S.: Asking Questions to Minimize Errors. J. Comput. Syst. Sci. 52(2), 268–286 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  34. Bshouty, N.H., Hellerstein, L.: Attribute-Efficient Learning in Query and Mistakebound Models. In: COLT 1996, pp. 235–243 (1996)

    Google Scholar 

  35. Bshouty, N.H., Hancock, T.R., Hellerstein, L.: Learning Boolean Read-Once Formulas over Generalized Bases. J. Comput. Syst. Sci. 50(3), 521–542 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  36. Bshouty, N.H., Hancock, T.R., Hellerstein, L.: Learning Arithmetic Read-Once Formulas. SIAM J. Comput. 24(4), 706–735 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  37. Bshouty, N.H., Hancock, T.R., Hellerstein, L., Karpinski, M.: An Algorithm to Learn Read-Once Threshold Formulas, and Transformations Between Learning Models. Computational Complexity 4, 37–61 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  38. Bläser, M., Hardt, M., Lipton, R.J., Vishnoi, N.K.: Deterministically Testing Sparse Polynomial Identities of Unbounded Degree. Inf. Process. Lett. 109(3), 187–192 (2009)

    Article  MATH  Google Scholar 

  39. Bläser, M., Hardt, M., Steurer, D.: Asymptotically Optimal Hitting Sets Against Polynomials. In: ICALP (1), pp. 345–356 (2008)

    Google Scholar 

  40. Bshouty, N.H., Mansour, Y.: Simple Learning Algorithms for Decision Trees and Multivariate Polynomials. SIAM J. Comput. 31(6), 1909–1925 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  41. Bshouty, N.H., Mossel, E., O’Donnell, R., Servedio, R.A.: Learning DNF from Random Walks. In: FOCS 2003, pp. 189–198 (2003)

    Google Scholar 

  42. Blum, A., Rudich, S.: Fast Learning of k-Term DNF Formulas with Queries. J. Comput. Syst. Sci. 51(3), 367–373 (1995)

    Article  MathSciNet  Google Scholar 

  43. Ben-Or, M., Tiwari, P.: A Deterministic Algorithm for Sparse Multivariate Polynomial Interpolation. In: STOC 1988, pp. 301–309 (1988)

    Google Scholar 

  44. Clausen, M., Dress, A.W.M., Grabmeier, J., Karpinski, M.: On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields. Theor. Comput. Sci. 84(2), 151–164 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  45. Chattopadhyay, A., Gavaldà, R., Hansen, K.A., Thérien, D.: Learning Read-Constant Polynomials of Constant Degree Modulo Composites. In: Kulikov, A., Vereshchagin, N. (eds.) CSR 2011. LNCS, vol. 6651, pp. 29–42. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  46. Cheng, J., Kamoi, K., Watanabe, Y.: User Identification by Signature Code for Noisy Multiple-Access Adder Channel. In: IEEE International Symposium on In- formation Theory, pp. 1974–1977 (2006)

    Google Scholar 

  47. Domingo, C.: Exact Learning of Subclasses of CDNF Formulars with Membership Queries. In: Jiang, T., Lee, D.T. (eds.) COCOON 1997. LNCS, vol. 1276, pp. 516–520. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  48. Damaschke, P.: Adaptive Versus Nonadaptive Attribute-Efficient Learning. Machine Learning 41(2), 197–215 (2000)

    Article  MATH  Google Scholar 

  49. Damaschke, P.: On Parallel Attribute-Efficient Learning. J. Comput. Syst. Sci. 67(1), 46–62 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  50. Du, D., Hwang, F.K.: Combinatorial Group Testing and Its Applications. World Scientific Pub. Co. Inc. (2000)

    Google Scholar 

  51. Du, D., Hwang, F.K.: Pooling Design and Nonadaptive Group Testing: Important Tools for DNA Sequencing. World Scientific Publishing Company (2006)

    Google Scholar 

  52. Domingo, C., Mishra, N., Pitt, L.: Efficient Read-Restricted Monotone CNF/DNF Dualization by Learning with Membership Queries. Machine Learning 37(1), 89–110 (1999)

    Article  MATH  Google Scholar 

  53. Feldman, V.: Attribute-Efficient and Non-adaptive Learning of Parities and DNF Expressions. JMLR 8, 1431–1460 (2007)

    MATH  Google Scholar 

  54. Frazier, M., Goldman, S.A., Mishra, N., Pitt, L.: Learning from a Consistently Ignorant Teacher. J. Comput. Syst. Sci. 52(3), 471–492 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  55. Fortnow, L., Klivans, A.R.: Efficient Learning Algorithms Yield Circuit Lower Bounds. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 350–363. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  56. Grigoriev, D., Karpinski, M.: Algorithms for Sparse Rational Interpolation. In: ISSAC 1991, pp. 7–13 (1991)

    Google Scholar 

  57. Grigoriev, D., Karpinski, M., Singer, M.F.: Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields. SIAM J. Comput. 19(6), 1059–1063 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  58. Goldman, S.A., Kearns, M.J., Schapire, R.E.: Exact Identification of Read-Once Formulas Using Fixed Points of Amplification Functions. SIAM J. Comput. 22(4), 705–726 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  59. Grigoriev, D., Karpinski, M., Singer, M.F.: Interpolation of Sparse Rational Functions Without Knowing Bounds on Exponents. In: FOCS 1990, pp. 840–846 (1990)

    Google Scholar 

  60. Grigoriev, D., Karpinski, M., Singer, M.F.: Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields. SIAM J. Comput. 19(6), 1059–1063 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  61. Goldreich, O., Levin, L.A.: A Hard-core Predicate for all One-way Functions. In: STOC 1989, pp. 25–32 (1989)

    Google Scholar 

  62. Gasarch, W.I., Smith, C.H.: Learning via Queries. J. ACM 39(3), 649–674 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  63. Gavaldà, R., Thérien, D.: An algebraic perspective on boolean function learning. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS, vol. 5809, pp. 201–215. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  64. Hellerstein, L., Karpinski, M.: Learning Read-Once Formulas Using Membership Queries. In: COLT 1989, pp. 146–161 (1989)

    Google Scholar 

  65. Jackson, J.C.: An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution. J. Comput. Syst. Sci. 55(3), 414–440 (1997)

    Article  MATH  Google Scholar 

  66. Jacksona, J., Shamir, E., Shwartzmanb, C.: Learning with Queries Corrupted by Classification Noise. Discrete Applied Mathematics 92(2-3), 157–175 (1999)

    Article  MathSciNet  Google Scholar 

  67. Kushilevitz, E.: A Simple Algorithm for Learning O (logn)-Term DNF. Inf. Process. Lett. 61(6), 289–292 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  68. Kearns, M.J.: Efficient Noise-Tolerant Learning from Statistical Queries. J. ACM 45(6), 983–1006 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  69. Kutyniok, G.: Compressed Sensing: Theory and Applications. CoRR abs/1203.3815 (2012)

    Google Scholar 

  70. Kabanets, V., Impagliazzo, R.: Derandomizing Polynomial Identity Tests means Proving Circuit Lower Bounds. In: STOC 2003, pp. 355–364 (2003)

    Google Scholar 

  71. Kaltofen, E., Lakshman, Y.N.: Improved Sparse Multivariate Polynomial Interpolation Algorithms. In: Gianni, P. (ed.) ISSAC 1988. LNCS, vol. 358, pp. 467–474. Springer, Heidelberg (1989)

    Chapter  Google Scholar 

  72. Kearns, M.J., Li, M.: Learning in the Presence of Malicious Errors. SIAM J. Comput. 22(4), 807–837 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  73. Kushilevitz, E., Mansour, Y.: Learning Decision Trees Using the Fourier Spectrum. SIAM J. Comput. 22(6), 1331–1348 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  74. Karnin, Z.S., Mukhopadhyay, P., Shpilka, A., Volkovich, I.: Deterministic Identity Testing of Depth-4 Multilinear Circuits with Bounded top Fan-in. In: STOC 2010, pp. 649–658 (2010)

    Google Scholar 

  75. Kautz, W.H., Singleton, R.C.: Nonrandom binary superimposed codes. IEEE Trans. Inform. Theory 10(4), 363–377 (1964)

    Article  MATH  Google Scholar 

  76. Kleitman, D.J., Spencer, J.: Families of k-independent sets. Discrete Mathematics 6(3), 255–262 (1972)

    Article  MathSciNet  Google Scholar 

  77. Klivans, A., Spielman, D.A.: Randomness Efficient Identity Testing of Multivariate Polynomials. In: STOC 2001, pp. 216–223 (2001)

    Google Scholar 

  78. Kayal, N., Saraf, S.: Blackbox Polynomial Identity Testing for Depth 3 Circuits. Electronic Colloquium on Computational Complexity (ECCC) 16, 32 (2009)

    Google Scholar 

  79. Karnin, Z.S., Shpilka, A.: Black Box Polynomial Identity Testing of Generalized Depth-3 Arithmetic Circuits with Bounded Top Fan-In. In: CCC 2008, pp. 280–291 (2008)

    Google Scholar 

  80. Naor, M., Schulman, L.J., Srinivasan, A.: Splitters and Near-optimal Derandomization. In: FOCS 1995, pp. 182–191 (1995)

    Google Scholar 

  81. Introduction to Coding Theory. Cambridge University Press (2007)

    Google Scholar 

  82. Raz, R., Shpilka, A.: Deterministic Polynomial Identity Testing in Non-commutative Models. Computational Complexity 14(1), 1–19 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  83. Saxena, N.: Progress on Polynomial Identity Testing. Bulletin of the EATCS 99, 49–79 (2009)

    MATH  Google Scholar 

  84. Settles, B.: Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison (2010)

    Google Scholar 

  85. Sakakibara, Y.: On Learning from Queries and Counterexamples in the Presence of Noise. Inf. Process. Lett. 37(5), 279–284 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  86. Schwartz, J.T.: Fast Probabilistic Algorithms for Verification of Polynomial Identities. Journal of the ACM 27(4), 701–717 (1980)

    Article  MATH  Google Scholar 

  87. Schapire, R.E., Sellie, L.M.: Learning Sparse Multivariate Polynomials over a Field with Queries and Counterexamples. In: COLT, pp. 17–26 (1996)

    Google Scholar 

  88. Saxena, N., Seshadhr, C.: Blackbox Identity Testing for Bounded top Fanin Depth-3 Circuits: the Field doesn’t matter. In: STOC 2011, pp. 431–440 (2011)

    Google Scholar 

  89. Shpilka, A., Yehudayoff, A.: Arithmetic Circuits: A Survey of Recent Results and Open Questions. Foundations and Trends in Theoretical Computer Science 5(3-4), 207–388 (2010)

    Article  MathSciNet  Google Scholar 

  90. Shpilka, A., Volkovich, I.: Improved Polynomial Identity Testing for Read-once Formulas. In: Dinur, I., Jansen, K., Naor, J., Rolim, J. (eds.) APPROX 2009. LNCS, vol. 5687, pp. 700–713. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  91. Shpilka, A., Volkovich, I.: Read-Once Polynomial Identity Testing. Electronic Colloquium on Computational Complexity (ECCC) 17, 11 (2010)

    Google Scholar 

  92. Saraf, S., Volkovich, I.: Black-box Identity Testing of Depth-4 Multilinear Circuits. In: STOC 2011, pp. 421–430 (2011)

    Google Scholar 

  93. Stinson, D.R., Wei, R., Zhu, L.: Some New Bounds for Cover-free Families. Journal of Combinatorial Theory, Series A 90(1), 224–234 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  94. Shevchenko, V.N., Zolotykh, N.Y.: Lower Bounds for the Complexity of Learning Half-Spaces with Membership Queries. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 61–71. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  95. Uehara, R., Tsuchida, K., Wegener, I.: Optimal Attribute-Efficient Learning of Disjunction, Parity and Threshold Functions. In: Ben-David, S. (ed.) EuroCOLT 1997. LNCS, vol. 1208, pp. 171–184. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  96. Valiant, L.G.: Completeness Classes in Algebra. In: Proc. of 11th ACM STOC, pp. 249–261 (1979)

    Google Scholar 

  97. Valiant, L.G.: Learning Disjunction of Conjunctions. In: IJCAI 1985, pp. 560–566 (1985)

    Google Scholar 

  98. Werther, K.: The Complexity of Sparse Polynomial Interpolation over Finite Fields. Appl. Algebra Eng. Commun. Comput. 5, 91–103 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  99. Wikipedia, http://en.wikipedia.org/wiki/Guessing_game

  100. Zippel, R.: Probabilistic Algorithms for Sparse Polynomials. In: Ng, K.W. (ed.) EUROSAM 1979 and ISSAC 1979. LNCS, vol. 72, pp. 216–226. Springer, Heidelberg (1979)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bshouty, N.H. (2013). Exact Learning from Membership Queries: Some Techniques, Results and New Directions. In: Jain, S., Munos, R., Stephan, F., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2013. Lecture Notes in Computer Science(), vol 8139. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40935-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-40935-6_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-40934-9

  • Online ISBN: 978-3-642-40935-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics