research-article

Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds

Authors:

Ryan C. Harkins,

John M. HitchcockAuthors Info & Claims

ACM Transactions on Computation Theory (TOCT), Volume 5, Issue 4

Article No.: 18, Pages 1 - 11

https://doi.org/10.1145/2539126.2539130

Published: 01 November 2013 Publication History

Abstract

This article extends and improves the work of Fortnow and Klivans [2009], who showed that if a circuit class C has an efficient learning algorithm in Angluin’s model of exact learning via equivalence and membership queries [Angluin 1988], then we have the lower bound EXP^NP not C. We use entirely different techniques involving betting games [Buhrman et al. 2001] to remove the NP oracle and improve the lower bound to EXP not C. This shows that it is even more difficult to design a learning algorithm for C than the results of Fortnow and Klivans [2009] indicated. We also investigate the connection between betting games and natural proofs, and as a corollary the existence of strong pseudorandom generators.

Our results also yield further evidence that the class of Boolean circuits has no efficient exact learning algorithm. This is because our separation is strong in that it yields a natural proof [Razborov and Rudich 1997] against the class. From this we conclude that an exact learning algorithm for Boolean circuits would imply that strong pseudorandom generators do not exist, which contradicts widely believed conjectures from cryptography. As a corollary we obtain that if strong pseudorandom generators exist, then there is no exact learning algorithm for Boolean circuits.

References

[1]

Aizenstein, H., Hegeds, T., Hellerstein, L., and Pitt, L. 1998. Complexity theoretic hardness results for query learning. Comput. Complex. 7, 1, 19--53.

Digital Library

[2]

Angluin, D. 1988. Queries and concept learning. Mach. Learn. 2, 4, 319--342.

[3]

Buhrman, H. and Homer, S. 1992. Superpolynomial circuits, almost sparse oracles and the exponential hierarchy. In Proceedings of the 12th Conference on Foundations of Software Technology and Theoretical Computer Science. Springer, 116--127.

Digital Library

[4]

Buhrman, H., van Melkebeek, D., Regan, K. W., Sivakumar, D., and Strauss, M. 2001. A generalization of resource-bounded measure, with application to the bpp vs. exp problem. SIAM J. Comput. 30, 2, 576--601.

Digital Library

[5]

Fortnow, L. and Klivans, A. R. 2009. Efficient learning algorithms yield circuit lower bounds. J. Comput. Syst. Sci. 75, 1, 27--36.

Digital Library

[6]

Hitchcock, J. M. 2007. Online learning and resource-bounded dimension: Winnow yields new lower bounds for hard sets. SIAM J. Comput. 36, 6, 1696--1708.

Digital Library

[7]

Karp, R. M. and Lipton, R. J. 1980. Some connections between nonuniform and uniform complexity classes. In Proceedings of the 12th Annual ACM Symposium on Theory of Computing. 302--309.

Digital Library

[8]

Kearns, M. and Valiant, L. 1994. Cryptographic limitations on learning boolean formulae and finite automata. J. ACM 41, 1, 67--95.

Digital Library

[9]

Kharitonov, M. 1995. Cryptographic lower bounds for learnability of boolean functions on the uniform distribution. J. Comput. Syst. Sci. 50, 3, 600--610.

Digital Library

[10]

Lindner, W., Schuler, R., and Watanabe, O. 2000. Resource-bounded measure and learnability. Theory Comput. Syst. 33, 2, 151--170.

[11]

Littlestone, N. 1988. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn. 2, 4, 285--318.

Digital Library

[12]

Lutz, J. H. 1992. Almost everywhere high nonuniform complexity. J. Comput. Syst. Sci. 44, 2, 220--258.

Digital Library

[13]

Lutz, J. H. 1997. The quantitative structure of exponential time. In Complexity Theory Retrospective II, L. A. Hemaspaandra and A. L. Selman Eds., Springer, 225--254.

Digital Library

[14]

Merkle, W., Miller, J. S., Nies, A., Reimann, J., and Stephan, F. 2006. Kolmogorov-loveland randomness and stochasticity. Ann. Pure Appl. Logic 138, 1--3, 183--210.

[15]

Muchnik, A. A., Semenov, A. L., and Uspensky, V. A. 1998. Mathematical metaphysics of randomness. Theor. Comput. Sci. 207, 2, 263--317.

Digital Library

[16]

Razborov, A. A. and Rudich, S. 1997. Natural proofs. J. Comput. System Sci. 55, 1, 24--35.

Digital Library

[17]

Regan, K. W., Sivakumar, D., and Cai, J. 1995. Pseudorandom generators, measure theory, and natural proofs. In Proceedings of the 36th Symposium on Foundations of Computer Science. IEEE Computer Society, Los Alamitos, CA, 26--35.

Digital Library

[18]

Toda, S. 1991. On the computational power of pp and _p. SIAM J. Comput. 20, 5, 865--877.

Digital Library

[19]

Valiant, L. G. 1984. A Theory of the Learnable. Comm. ACM 27, 11, 1134--1142.

Digital Library

Cited By

Chen LRothblum RTell RYogev E(2023)On Exponential-time Hypotheses, Derandomization, and Circuit Lower BoundsJournal of the ACM10.1145/359358170:4(1-62)Online publication date: 20-Apr-2023
https://dl.acm.org/doi/10.1145/3593581
Arunachalam SGrilo AGur TOliveira ISundaram A(2022)Quantum learning algorithms imply circuit lower bounds2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00062(562-573)Online publication date: Feb-2022
https://doi.org/10.1109/FOCS52979.2021.00062
Hitchcock JSekoni AShafei H(2021)Polynomial-Time Random Oracles and Separating Complexity ClassesACM Transactions on Computation Theory10.1145/343438913:1(11-16)Online publication date: 21-Jan-2021
https://dl.acm.org/doi/10.1145/3434389
Show More Cited By

Index Terms

Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds
1. Theory of computation
  1. Computational complexity and cryptography
    1. Complexity classes

Recommendations

Nondeterministic circuit lower bounds from mildly derandomizing Arthur-Merlin games

In several settings, derandomization is known to follow from circuit lower bounds that themselves are equivalent to the existence of pseudorandom generators. This leaves open the question whether derandomization implies the circuit lower bounds that are ...
Efficient learning algorithms yield circuit lower bounds

We describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin's model of exact learning from membership and ...
Exact learning algorithms, betting games, and circuit lower bounds
ICALP'11: Proceedings of the 38th international colloquim conference on Automata, languages and programming - Volume Part I

This paper extends and improves work of Fortnow and Klivans [5], who showed that if a circuit class C has an efficient learning algorithm in Angluin's model of exact learning via equivalence and membership queries [2], then we have the lower bound EXP^NP ...

Reviews

Reviewer: Marius Zimand

It is natural to expect that the more complex a concept is, the harder it is to learn. This paper improves this type of correlation for concepts that are represented by a class of circuits . It shows that if is learnable by an efficient algorithm that makes membership and equivalence queries, then there exists a set in EXP that cannot be computed by a circuit of type . Previously, this was known for the larger complexity class EXP . We recall that a membership query for learning an unknown circuit C is of the type "What is the value of C on x __?__" An equivalence query is of the type "Is C =d__?__" (to which the teacher says YES or gives an input x on which C and d differ). The proof given in this paper uses efficient martingales of a certain type. The authors show that a learning algorithm for C can be transformed into a martingale that succeeds on C . On the other hand, it is known that there exist sets in EXP on which no martingale (of the type considered in this paper) succeeds. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computation Theory

ACM Transactions on Computation Theory Volume 5, Issue 4

November 2013

103 pages

ISSN:1942-3454

EISSN:1942-3462

DOI:10.1145/2539126

Editor:
Eric Allender
Rutgers University

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013

Accepted: 01 August 2013

Revised: 01 August 2013

Received: 01 February 2013

Published in TOCT Volume 5, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
158
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen LRothblum RTell RYogev E(2023)On Exponential-time Hypotheses, Derandomization, and Circuit Lower BoundsJournal of the ACM10.1145/359358170:4(1-62)Online publication date: 20-Apr-2023
https://dl.acm.org/doi/10.1145/3593581
Arunachalam SGrilo AGur TOliveira ISundaram A(2022)Quantum learning algorithms imply circuit lower bounds2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00062(562-573)Online publication date: Feb-2022
https://doi.org/10.1109/FOCS52979.2021.00062
Hitchcock JSekoni AShafei H(2021)Polynomial-Time Random Oracles and Separating Complexity ClassesACM Transactions on Computation Theory10.1145/343438913:1(11-16)Online publication date: 21-Jan-2021
https://dl.acm.org/doi/10.1145/3434389
Chen LRothblum RTell RYogev E(2020)On Exponential-Time Hypotheses, Derandomization, and Circuit Lower Bounds: Extended Abstract2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS46700.2020.00010(13-23)Online publication date: Nov-2020
https://doi.org/10.1109/FOCS46700.2020.00010
Kawachi A(2018)Circuit lower bounds from learning-theoretic approachesTheoretical Computer Science10.1016/j.tcs.2018.04.038733(83-98)Online publication date: Jul-2018
https://doi.org/10.1016/j.tcs.2018.04.038
Oliveira ISanthanam R(2017)Conspiracies between learning algorithms, circuit lower bounds, and pseudorandomnessProceedings of the 32nd Computational Complexity Conference10.5555/3135595.3135613(1-49)Online publication date: 9-Jul-2017
https://dl.acm.org/doi/10.5555/3135595.3135613

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents