research-article

Optimal bounds for sign-representing the intersection of two halfspaces by polynomials

Author:

Alexander A. SherstovAuthors Info & Claims

STOC '10: Proceedings of the forty-second ACM symposium on Theory of computing

Pages 523 - 532

https://doi.org/10.1145/1806689.1806762

Published: 05 June 2010 Publication History

Abstract

The threshold degree of a function f:{0,1}ⁿ->{-1,+1} is the least degree of a real polynomial p with f=sgn p. We prove that the intersection of two halfspaces on {0,1}ⁿ has threshold degree Omega(n), which matches the trivial upper bound and completely answers a question due to Klivans (2002). The best previous lower bound was Omega(sqrt n). Our result shows that the intersection of two halfspaces on {0,1}ⁿ only admits a trivial 2^Θ(n)-time learning algorithm based on sign-representation by polynomials, unlike the advances achieved in PAC learning DNF formulas and read-once Boolean formulas. The proof introduces a new technique of independent interest, based on Fourier analysis and matrix theory.

References

[1]

M. Alekhnovich, M. Braverman, V. Feldman, A. R. Klivans, and T. Pitassi. The complexity of properly learning simple concept classes. J. Comput. Syst. Sci., 74(1):16--34, 2008.

Digital Library

[2]

A. Ambainis, A. M. Childs, B. Reichardt, R. Spalek, and S. Zhang. Any AND-OR formula of size N can be evaluated in time N^1-o(1) on a quantum computer. In Proc. of the 48th Symposium on Foundations of Computer Science (FOCS), pages 363--372, 2007.

Digital Library

[3]

R. I. Arriaga and S. Vempala. An algorithmic theory of learning: Robust concepts and random projection. Mach. Learn., 63(2):161--182, 2006.

Digital Library

[4]

J. Aspnes, R. Beigel, M. L. Furst, and S. Rudich. The expressive power of voting polynomials. Combinatorica, 14(2):135--148, 1994.

[5]

R. Beigel. Perceptrons, PP, and the polynomial hierarchy. Computational Complexity, 4:339--349, 1994.

Digital Library

[6]

R. Beigel, N. Reingold, and D. A. Spielman. PP is closed under intersection. J. Comput. Syst. Sci., 50(2):191--202, 1995.

Digital Library

[7]

A. Blum and R. Kannan. Learning an intersection of a constant number of halfspaces over a uniform distribution. J. Comput. Syst. Sci., 54(2):371--380, 1997.

Digital Library

[8]

A. L. Blum and R. L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5:117--127, 1992.

Digital Library

[9]

H. Buhrman, N. K. Vereshchagin, and R. de Wolf. On computation and communication with small bias. In Proc. of the 22nd Conf. on Computational Complexity (CCC), pages 24--32, 2007.

Digital Library

[10]

E. Farhi, J. Goldstone, and S. Gutmann. A quantum algorithm for the Hamiltonian NAND tree. Theory of Computing, 4(1):169--190, 2008.

[11]

S. A. Gershgorin. Ueber die Abgrenzung der Eigenwerte einer Matrix. Izv. Akad. Nauk. U.S.S.R. Otd. Fiz.-Mat. Nauk, 7:749--754, 1931.

[12]

J. C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. J. Comput. Syst. Sci., 55(3):414--440, 1997.

Digital Library

[13]

S. Jukna. Extremal Combinatorics with Applications in Computer Science. Springer--Verlag, Berlin, 2001.

Digital Library

[14]

S. Khot and R. Saket. On hardness of learning intersection of two halfspaces. In Proc. of the 40th Symposium on Theory of Computing (STOC), pages 345--354, 2008.

Digital Library

[15]

A. R. Klivans. A Complexity-Theoretic Approach to Learning. PhD thesis, MIT, 2002.

Digital Library

[16]

A. R. Klivans, P. M. Long, and A. K. Tang. Baum's algorithm learns intersections of halfspaces with respect to log-concave distributions. In Proc. of the 13th Intl. Workshop on Randomization and Computation, pages 588--600, 2009.

Digital Library

[17]

A. R. Klivans, R. O'Donnell, and R. A. Servedio. Learning intersections and thresholds of halfspaces. J. Comput. Syst. Sci., 68(4):808--840, 2004.

Digital Library

[18]

A. R. Klivans and R. A. Servedio. Learning DNF in time 2^{O(n^1/3)}. J. Comput. Syst. Sci., 68(2):303--318, 2004.

Digital Library

[19]

A. R. Klivans and R. A. Servedio. Learning intersections of halfspaces with a margin. J. Comput. Syst. Sci., 74(1):35--48, 2008.

Digital Library

[20]

A. R. Klivans and A. A. Sherstov. Cryptographic hardness for learning intersections of halfspaces. J. Comput. Syst. Sci., 75(1):2--12, 2009.

Digital Library

[21]

M. Krause and P. Pudlák. On the computational power of depth-2 circuits with threshold and modulo gates. Theor. Comput. Sci., 174(1--2):137--156, 1997.

Digital Library

[22]

M. Krause and P. Pudlák. Computing Boolean functions by polynomials and threshold circuits. Comput. Complex., 7(4):346--370, 1998.

Digital Library

[23]

S. Kwek and L. Pitt. PAC learning intersections of halfspaces with membership queries. Algorithmica, 22(1/2):53--75, 1998.

[24]

T. Lee. A note on the sign degree of formulas, 2009. Available at http://arxiv.org/abs/0909.4607.

[25]

M. L. Minsky and S. A. Papert. Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge, 1969.

Digital Library

[26]

S. Muroga. Threshold Logic and Its Applications. John Wiley & Sons, New York, 1971.

[27]

J. Myhill and W. H. Kautz. On the size of weights required for linear-input switching functions. IRE Trans. on Electronic Computers, 10(2):288--290, 1961.

[28]

D. J. Newman. Rational approximation to |x|. Michigan Math. J., 11(1):11--14, 1964.

[29]

R. O'Donnell and R. A. Servedio. New degree bounds for polynomial threshold functions. In Proc. of the 35th Symposium on Theory of Computing (STOC), pages 325--334, 2003.

Digital Library

[30]

R. Paturi and M. E. Saks. Approximating threshold circuits by rational functions. Inf. Comput., 112(2):257--272, 1994.

Digital Library

[31]

A. A. Razborov and A. A. Sherstov. The sign-rank of AC0. SIAM J. Comput., 39(5):1833--1855, 2010. Preliminary version in 49th FOCS, 2008.

Digital Library

[32]

T. J. Rivlin. An Introduction to the Approximation of Functions. Dover Publications, New York, 1981.

[33]

M. E. Saks. Slicing the hypercube. Surveys in Combinatorics, pages 211--255, 1993.

Digital Library

[34]

A. A. Sherstov. The intersection of two halfspaces has high threshold degree. In Proc. of the 50th Symposium on Foundations of Computer Science (FOCS), 2009.

Digital Library

[35]

A. A. Sherstov. The pattern matrix method. SIAM J. Comput., 2010. To appear. Preliminary version in 40th STOC, 2008.

[36]

A. A. Sherstov. Separating AC0 from depth-2 majority circuits. SIAM J. Comput., 38(6):2113--2129, 2009. Preliminary version in 39th STOC, 2007.

Digital Library

[37]

A. A. Sherstov. The unbounded-error communication complexity of symmetric functions. In Proc. of the 49th Symposium on Foundations of Computer Science (FOCS), pages 384--393, 2008.

Digital Library

[38]

K.-Y. Siu and J. Bruck. On the power of threshold circuits with small weights. SIAM J. Discrete Math., 4(3):423--435, 1991.

Digital Library

[39]

K.-Y. Siu, V. P. Roychowdhury, and T. Kailath. Rational approximation techniques for analysis of neural networks. IEEE Transactions on Information Theory, 40(2):455--466, 1994.

Digital Library

[40]

L. G. Valiant. A theory of the learnable. Commun. ACM, 27(11):1134--1142, 1984.

Digital Library

[41]

S. Vempala. A random sampling based algorithm for learning the intersection of halfspaces. In Proc. of the 38th Symposium on Foundations of Computer Science (FOCS), pages 508--513, 1997.

Digital Library

[42]

E. I. Zolotarev. Application of elliptic functions to questions of functions deviating least and most from zero. Izvestiya Imp. Akad. Nauk, 30(5), 1877.

Cited By

Aaronson SKothari RKretschmer WThaler JAceto L(2020)Quantum lower bounds for approximate counting via laurent polynomialsProceedings of the 35th Computational Complexity Conference10.4230/LIPIcs.CCC.2020.7(1-47)Online publication date: 28-Jul-2020
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2020.7
Bun MThaler J(2015)Hardness Amplification and the Approximate Degree of Constant-Depth CircuitsAutomata, Languages, and Programming10.1007/978-3-662-47672-7_22(268-280)Online publication date: 20-Jun-2015
https://doi.org/10.1007/978-3-662-47672-7_22
Sherstov AShmoys D(2014)Breaking the minsky-papert barrier for constant-depth circuitsProceedings of the forty-sixth annual ACM symposium on Theory of computing10.1145/2591796.2591871(223-232)Online publication date: 31-May-2014
https://dl.acm.org/doi/10.1145/2591796.2591871
Show More Cited By

Index Terms

Optimal bounds for sign-representing the intersection of two halfspaces by polynomials
1. Theory of computation
  1. Computational complexity and cryptography
    1. Complexity classes

Recommendations

The Intersection of Two Halfspaces Has High Threshold Degree
FOCS '09: Proceedings of the 2009 50th Annual IEEE Symposium on Foundations of Computer Science

The threshold degree of a Boolean function f:{0,1}^n->{-1,+1} is the least degree of a real polynomial p such f(x)=sgn p(x). We construct two halfspaces on {0,1}^n whose intersection has threshold degree Theta(sqrt n), an exponential improvement on ...
Unconditional lower bounds for learning intersections of halfspaces

We prove new lower bounds for learning intersections of halfspaces, one of the most important concept classes in computational learning theory. Our main result is that any statistical-query algorithm for learning the intersection of $\sqrt{n}$ halfspaces in n ...
The Intersection of Two Halfspaces Has High Threshold Degree
^† Special Section on the Fiftieth Annual IEEE Symposium on Foundations of Computer Science (FOCS 2009)

The threshold degree of a Boolean function $f\!\!\,: \{0,1\}^n \rightarrow \{-1, +1\}$ is the least degree of a real polynomial $p$ such that $f(x)\equiv{{\rm sgn}\,p(x)}.$ We construct two halfspaces on $\{0,1\}^n$ whose intersection has threshold degree $\...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC '10: Proceedings of the forty-second ACM symposium on Theory of computing

June 2010

812 pages

ISBN:9781450300506

DOI:10.1145/1806689

General Chair:
Michael Mitzenmacher
Harvard
,
Program Chair:
Leonard J. Schulman
Caltech

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

STOC'10

Sponsor:

SIGACT

STOC'10: Symposium on Theory of Computing

June 5 - 8, 2010

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
136
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aaronson SKothari RKretschmer WThaler JAceto L(2020)Quantum lower bounds for approximate counting via laurent polynomialsProceedings of the 35th Computational Complexity Conference10.4230/LIPIcs.CCC.2020.7(1-47)Online publication date: 28-Jul-2020
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2020.7
Bun MThaler J(2015)Hardness Amplification and the Approximate Degree of Constant-Depth CircuitsAutomata, Languages, and Programming10.1007/978-3-662-47672-7_22(268-280)Online publication date: 20-Jun-2015
https://doi.org/10.1007/978-3-662-47672-7_22
Sherstov AShmoys D(2014)Breaking the minsky-papert barrier for constant-depth circuitsProceedings of the forty-sixth annual ACM symposium on Theory of computing10.1145/2591796.2591871(223-232)Online publication date: 31-May-2014
https://dl.acm.org/doi/10.1145/2591796.2591871
Sherstov AKarloff HPitassi T(2012)Making polynomials robust to noiseProceedings of the forty-fourth annual ACM symposium on Theory of computing10.1145/2213977.2214044(747-758)Online publication date: 19-May-2012
https://dl.acm.org/doi/10.1145/2213977.2214044
Vempala S(2010)A random-sampling-based algorithm for learning intersections of halfspacesJournal of the ACM10.1145/1857914.185791657:6(1-14)Online publication date: 5-Nov-2010
https://dl.acm.org/doi/10.1145/1857914.1857916
Amano K(2010)New Upper Bounds on the Average PTF Density of Boolean FunctionsAlgorithms and Computation10.1007/978-3-642-17517-6_28(304-315)Online publication date: 2010
https://doi.org/10.1007/978-3-642-17517-6_28

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents