article

Size--Depth Tradeoffs for Threshold Circuits

Authors:

Russell Impagliazzo,

Ramamohan Paturi,

Michael E. SaksAuthors Info & Claims

SIAM Journal on Computing, Volume 26, Issue 3

Pages 693 - 707

https://doi.org/10.1137/S0097539792282965

Published: 01 June 1997 Publication History

Publisher Site

Abstract

The following size--depth tradeoff for threshold circuits is obtained: any threshold circuit of depth $d$ that computes the parity function on $n$ variables must have at least $n^{1 + c\theta^{-d }}$ edges, where $c>0$ and $\theta \leq 3$ are constants independent of $n$ and $d$. Previously known constructions show that up to the choice of $c$ and $\theta$ this bound is best possible. In particular, the lower bound implies an affirmative answer to the conjecture of Paturi and Saks that a bounded-depth threshold circuit that computes parity requires a superlinear number of edges. This is the first superlinear lower bound for an explicit function that holds for any fixed depth and the first that applies to threshold circuits with unrestricted weights.

The tradeoff is obtained as a consequence of a general restriction theorem for threshold circuits with a small number of edges: For any threshold circuit with $n$ inputs, depth $d$, and at most $kn$ edges, there exists a partial assignment to the inputs that fixes the output of the circuit to a constant while leaving $\lfloor n/(c_1k)^{c_2\theta^{d}} \rfloor$ variables unfixed, where $c_1,c_2 > 0$ and $ \theta \leq 3$ are constants independent of $n$, $k$, and $d$.

A tradeoff between the number of gates and depth is also proved: any threshold circuit of depth $d$ that computes the parity of $n$ variables has at least $(n/2)^{1/2(d-1)}$ gates. This tradeoff, which is essentially the best possible, was proved previously (with a better constant in the exponent) for the case of threshold circuits with polynomially bounded weights in [K. Siu, V. Roychowdury, and T. Kailath, IEEE Trans. Inform. Theory , 40 (1994), pp. 455--466]; the result in the present paper holds for unrestricted weights.

Cited By

View all

Khalife SCheng HBasu A(2024)Neural networks with linear threshold activations: structure and algorithmsMathematical Programming: Series A and B10.1007/s10107-023-02016-5206:1-2(333-356)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s10107-023-02016-5
Kumar V(2023)Tight Correlation Bounds for Circuits Between AC0 and TC0Proceedings of the conference on Proceedings of the 38th Computational Complexity Conference10.4230/LIPIcs.CCC.2023.18(1-40)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2023.18
Hatami PHoza WTal ATell RSaha BServedio R(2023)Depth-𝑑 Threshold Circuits vs. Depth-(𝑑+1) AND-OR TreesProceedings of the 55th Annual ACM Symposium on Theory of Computing10.1145/3564246.3585216(895-904)Online publication date: 2-Jun-2023
https://dl.acm.org/doi/10.1145/3564246.3585216
Show More Cited By

Index Terms

Size--Depth Tradeoffs for Threshold Circuits
1. Theory of computation
  1. Computational complexity and cryptography
    1. Complexity classes

Recommendations

Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits
STOC '16: Proceedings of the forty-eighth annual ACM symposium on Theory of Computing

In order to formally understand the power of neural computing, we first need to crack the frontier of threshold circuits with two and three layers, a regime that has been surprisingly intractable to analyze. We prove the first super-linear gate lower ...
Size-Treewidth Tradeoffs for Circuits Computing the Element Distinctness Function

In this work we study the relationship between size and treewidth of circuits computing variants of the element distinctness function. First, we show that for each n, any circuit of treewidth t computing the element distinctness function źn : {0, 1}n ź {...
Tight Correlation Bounds for Circuits Between AC0 and TC0
CCC '23: Proceedings of the conference on Proceedings of the 38th Computational Complexity Conference

We initiate the study of generalized AC⁰ circuits comprised of arbitrary unbounded fan-in gates which only need to be constant over inputs of Hamming weight ≥ k (up to negations of the input bits), which we denote GC⁰(k). The gate set of this class ...

Comments

Information & Contributors

Information

Published In

SIAM Journal on Computing Volume 26, Issue 3

June 1997

268 pages

ISSN:0097-5397

Editor:
Z. Galil
Columbia Univ., New York, NY and Tel-Aviv Univ., Tel-Aviv, Israel

Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 June 1997

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Khalife SCheng HBasu A(2024)Neural networks with linear threshold activations: structure and algorithmsMathematical Programming: Series A and B10.1007/s10107-023-02016-5206:1-2(333-356)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s10107-023-02016-5
Kumar V(2023)Tight Correlation Bounds for Circuits Between AC0 and TC0Proceedings of the conference on Proceedings of the 38th Computational Complexity Conference10.4230/LIPIcs.CCC.2023.18(1-40)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2023.18
Hatami PHoza WTal ATell RSaha BServedio R(2023)Depth-𝑑 Threshold Circuits vs. Depth-(𝑑+1) AND-OR TreesProceedings of the 55th Annual ACM Symposium on Theory of Computing10.1145/3564246.3585216(895-904)Online publication date: 2-Jun-2023
https://dl.acm.org/doi/10.1145/3564246.3585216
Chen LHirahara SOliveira IPich JRajgopal NSanthanam R(2022)Beyond Natural Proofs: Hardness Magnification and LocalityJournal of the ACM10.1145/353839169:4(1-49)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3538391
Bajpai SKrishan VKush DLimaye NSrinivasan S(2022)A #SAT Algorithm for Small Constant-Depth Circuits with PTF gatesAlgorithmica10.1007/s00453-021-00915-784:4(1132-1162)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s00453-021-00915-7
Zhou CZhuang QMattina MWhatmough P(2021)Strong data processing inequality in neural networks with noisy neurons and its implications2021 IEEE International Symposium on Information Theory (ISIT)10.1109/ISIT45174.2021.9517787(1170-1175)Online publication date: 12-Jul-2021
https://dl.acm.org/doi/10.1109/ISIT45174.2021.9517787
Gupta NSaha CThankey BAceto L(2020)A super-quadratic lower bound for depth four arithmetic circuitsProceedings of the 35th Computational Complexity Conference10.4230/LIPIcs.CCC.2020.23(1-31)Online publication date: 28-Jul-2020
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2020.23
Chen LJin CWilliams RMakarychev KMakarychev YTulsiani MKamath GChuzhoy J(2020)Sharp threshold results for computational complexityProceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing10.1145/3357713.3384283(1335-1348)Online publication date: 22-Jun-2020
https://dl.acm.org/doi/10.1145/3357713.3384283
Oliveira IPich JSanthanam RAceto L(2019)Hardness magnification near state-of-the-art lower boundsProceedings of the 34th Computational Complexity Conference10.4230/LIPIcs.CCC.2019.27(1-29)Online publication date: 17-Jul-2019
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2019.27
Chen LWilliams RAceto L(2019)Stronger connections between circuit analysis and circuit lower bounds, via PCPs of proximityProceedings of the 34th Computational Complexity Conference10.4230/LIPIcs.CCC.2019.19(1-43)Online publication date: 17-Jul-2019
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2019.19
Show More Cited By

Abstract

Cited By

Index Terms

Recommendations

Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits

Size-Treewidth Tradeoffs for Circuits Computing the Element Distinctness Function

Tight Correlation Bounds for Circuits Between AC0 and TC0

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations