Article

Hardness of approximate two-level logic minimization and PAC learning with membership queries

Author:

Vitaly FeldmanAuthors Info & Claims

STOC '06: Proceedings of the thirty-eighth annual ACM symposium on Theory of Computing

Pages 363 - 372

https://doi.org/10.1145/1132516.1132569

Published: 21 May 2006 Publication History

Abstract

Producing a small DNF expression consistent with given data is a classical problem in computer science that occurs in a number of forms and has numerous applications. We consider two standard variants of this problem. The first one is two-level logic minimization or finding a minimal DNF formula consistent with a given complete truth table (TT-MinDNF. This problem was formulated by Quine in 1952 and has been since one of the key problems in logic design. It was proved NP-complete by Masek in 1979. The best known polynomial approximation algorithm is based on a reduction to the SET-COVER problem and produces a DNF formula of size O(d ∙ OPT), where d is the number of variables. We prove that TT-MinDNF is NP-hard to approximate within d^γ for some constant γ > 0, establishing the first inapproximability result for the problem.The other DNF minimization problem we consider is PAC learning of DNF expressions when the learning algorithm must output a DNF expression as its hypothesis (referred to as proper learning). We prove that DNF expressions are NP-hard to PAC learn properly even when the learner has access to membership queries, thereby answering a long-standing open question due to Valiant [40]. Finally, we show that inapproximability of TT-MinDNF implies hardness results for restricted proper learning of DNF expressions with membership queries even when learning with respect to the uniform distribution only.

References

[1]

R. Agrawal, J. Gehrke, D. Gunopulos, and P. Raghavan. Automatic subspace clustering of high dimensional data. Data Mining and Knowledge Discovery, 11(1):5--33, July 2005.]]

Digital Library

[2]

M. Alekhnovich, M. Braverman, V. Feldman, A. Klivans, and T. Pitassi. Learnability and automizability. In Proceeding of FOCS '04, pages 621--630, 2004.]]

Digital Library

[3]

E. Allender, L. Hellerstein, P. McCabe, T. Pitassi, and M.Saks. Minimizing DNF Formulas and AC0 Circuits Given a Truth Table. Electronic Colloquium on Computational Complexity (ECCC), (126), Nov, 2005.]]

[4]

E. Allender, L. Hellerstein, T. Pitassi, and M.Saks. On the complexity of finding minimal representations of boolean functions. 2004. Unpublished.]]

[5]

D. Angluin and M. Kharitonov. When won't membership queries help? Journal of Computer and System Sciences, 50(2):336--355, 1995.]]

Digital Library

[6]

M. Bellare, S. Goldwasser, C. Lund, and A. Russell. Efficient probabilistically checkable proofs and applications to approximations. In STOC '93, pages 294--304, 1993.]]

Digital Library

[7]

A. Blum and S. Rudich. Fast learning of k-term DNF formulas with queries. Journal of Computer and System Sciences, 51(3):367--373, 1995.]]

Digital Library

[8]

A. L. Blum and R. L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5(1):117--127, 1992.]]

Digital Library

[9]

O. Coudert and J. Madre. METAPRIME, an Interactive Fault-Tree Analyser. IEEE Transactions on Reliability, 43(1):121--127, 1994.]]

[10]

O. Coudert and T. Sasao. Two-level logic minimization. Kluwer Academic Publishers, 2001.]]

[11]

S. Czort. The complexity of minimizing disjunctive normal form formulas. Master's thesis, University of Aarhus, 1999.]]

[12]

I. Dinur, V. Guruswami, S. Khot, and O. Regev. A new multilayered pcp and the hardness of hypergraph vertex cover. SIAM Journal of Computing, 34(5):1129--1146, 2005.]]

Digital Library

[13]

P. Erdös, P. Frankl, and Z. Furedi. Families of finite sets in which no set is covered by the union of r others. Israel Journal of Mathematics, 51:79--89, 1985.]]

[14]

U. Feige. A threshold of ln n for approximating set cover. Journal of the ACM, 45(4):634--652, 1998.]]

Digital Library

[15]

U. Feige and J. Kilian. Zero knowledge and the chromatic number. In Proceedings of Conference on Computational Complexity (CCC-96), pages 278--289, May 24--27 1996.]]

Digital Library

[16]

J.F. Gimpel. A method for producing a boolean function having an arbitrary prescribed prime implicant table. IEEE Transactions on Computers, 14:485--488, 1965.]]

[17]

E. A. Gold. Complexity of automaton identification from given data. Information and Control, 37:302--320, 1978.]]

[18]

Johan Håstad and Subhash Khot. Query Efficient PCPs with Perfect Completeness. Theory of Computing, 1(7):119--148, 2005.]]

[19]

Lisa Hellerstein and Vijay Raghavan. Exact learning of DNF formulas using DNF hypotheses. In ACM, editor, Proceedings of STOC '02, pages 465--473, 2002.]]

Digital Library

[20]

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55:414--440, 1997.]]

Digital Library

[21]

W. Kautz and R. Singleton. Nonrandom binary superimposed codes. IEEE Trans. Inform. Theory, 10:363--377, 1964.]]

Digital Library

[22]

M. Kearns, M. Li, L. Pitt, and L. Valiant. On the learnability of boolean formulae. In Proceedings of the Nineteenth Annual Symposium on Theory of Computing, pages 285--295, 1987.]]

Digital Library

[23]

A. Klivans and R. Servedio. Learning DNF in time 2Õ(n1/3). In Proceedings of the Thirty-Third Annual Symposium on Theory of Computing, pages 258--265, 2001.]]

Digital Library

[24]

H. Liu. Routing table compaction in ternary cam. IEEE Micro, 22(1):58--64, 2002.]]

Digital Library

[25]

C. Lund and M. Yannakakis. On the hardness of approximating minimization problems. Journal of the ACM, 41(5):960--981, 1994.]]

Digital Library

[26]

W. Masek. Some NP-complete set covering problems. unpublished, 1979.]]

[27]

E. L. Jr. McCluskey. Minimization of Boolean Functions. Bell Sys. Tech. Jour., 35:1417--1444, 1956.]]

[28]

N. Nisan and A. Wigderson. Hardness versus randomness. Journal of Computer and System Sciences, 49:149--167, 1994.]]

Digital Library

[29]

R. Nock, P. Jappy, and J. Sallantin. Generalized graph colorability and compressibility of boolean formulae. In Proceedings of the 9th International Symposium on Algorithms and Computation (ISAAC), 1998.]]

Digital Library

[30]

W. J. Paul. Boolesche minimalpolynome und überdeckungsprobleme. Acta Informatica, 4:321--336, 1974.]]

Digital Library

[31]

L. Pitt and L. Valiant. Computational limitations on learning from examples. Journal of the ACM, 35(4):965--984, 1988.]]

Digital Library

[32]

W.V. Quine. The problem of simplifying truth functions. Americal Mathematical Monthly, 59:521--531, 1952.]]

[33]

W.V. Quine. A way to simplify truth functions. Americal Mathematical Monthly, 62:627--631, 1956.]]

[34]

R. Raz and S. Safra. A sub-constant error-probability low-degree test, and a sub-constant error-probability PCP characterization of NP. In Proceedings of STOC '97, pages 475--484, 1997.]]

Digital Library

[35]

A. Ta-Shma. A Note on PCP vs. MIP. Information Processing Letters, 58(3):135--140, 1996.]]

Digital Library

[36]

L. Trevisan. Non-approximability results for optimization problems on bounded degree instances. In Proceedings of STOC '01, pages 453--461, 2001.]]

Digital Library

[37]

C. Umans. The minimum equivalent DNF problem and shortest implicants. J. Comput. Syst. Sci., 63(4):597--611, 2001.]]

Digital Library

[38]

C. Umans, T. Villa, and A. L. Sangiovanni-Vincentelli. Complexity of two-level logic minimization. Technical Report UCB/ERL M04/45, UC Berkeley, October 2004.]]

[39]

S. Vadhan. Lecture notes on pseudorandomness. Available at http://www.courses.fas.harvard.edu/~cs225/, 2004.]]

[40]

L. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134--1142, 1984.]]

Digital Library

[41]

L. Valiant. Learning disjunctions of conjunctions. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pages 560--566, 1985.]]

Digital Library

Cited By

Koch CStrassle CTan L(2023)Properly learning decision trees with queries is NP-hard2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00146(2383-2407)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00146
Ilango R(2022)Constant Depth Formula and Partial Function Versions of MCSP Are HardSIAM Journal on Computing10.1137/20M138356253:6(FOCS20-317-FOCS20-367)Online publication date: 31-Aug-2022
https://doi.org/10.1137/20M1383562
Ilango R(2022)The Minimum Formula Size Problem is (ETH) Hard2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00050(427-432)Online publication date: Feb-2022
https://doi.org/10.1109/FOCS52979.2021.00050
Show More Cited By

Index Terms

Hardness of approximate two-level logic minimization and PAC learning with membership queries
1. Theory of computation
  1. Randomness, geometry and discrete structures

Recommendations

Hardness of approximate two-level logic minimization and PAC learning with membership queries

Producing a small DNF expression consistent with given data is a classical problem in computer science that occurs in a number of forms and has numerous applications. We consider two standard variants of this problem. The first one is two-level logic ...
The Minimum Equivalent DNF Problem and Shortest Implicants

We prove that the Minimum Equivalent DNF problem is P2-complete, resolving a conjecture due to Stockmeyer. We also consider the complexity and approximability of a related optimization problem in the second level of the polynomial hierarchy, that of ...
Hardness of Learning Halfspaces with Noise

Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noise-free case, when a halfspace consistent with all the training examples exists, the problem can be solved in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC '06: Proceedings of the thirty-eighth annual ACM symposium on Theory of Computing

May 2006

786 pages

ISBN:1595931341

DOI:10.1145/1132516

Program Chair:
Jon Kleinberg
Cornell University, Ithaca, NY

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

STOC06

Sponsor:

STOC06: Symposium on Theory of Computing

May 21 - 23, 2006

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25

Sponsor:
sigact

57th Annual ACM Symposium on Theory of Computing (STOC 2025)

June 23 - 27, 2025

Prague , Czech Republic

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
319
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Koch CStrassle CTan L(2023)Properly learning decision trees with queries is NP-hard2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00146(2383-2407)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00146
Ilango R(2022)Constant Depth Formula and Partial Function Versions of MCSP Are HardSIAM Journal on Computing10.1137/20M138356253:6(FOCS20-317-FOCS20-367)Online publication date: 31-Aug-2022
https://doi.org/10.1137/20M1383562
Ilango R(2022)The Minimum Formula Size Problem is (ETH) Hard2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00050(427-432)Online publication date: Feb-2022
https://doi.org/10.1109/FOCS52979.2021.00050
Ilango R(2020)Constant Depth Formula and Partial Function Versions of MCSP are Hard2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS46700.2020.00047(424-433)Online publication date: Nov-2020
https://doi.org/10.1109/FOCS46700.2020.00047
Arpe JManthey B(2019)Approximability of Minimum AND-CircuitsAlgorithmica10.5555/3118778.311918853:3(337-357)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.5555/3118778.3119188
Wang TRudin CDoshi-Velez FLiu YKlampfl EMacNeille P(2017)A Bayesian framework for learning rule sets for interpretable classificationThe Journal of Machine Learning Research10.5555/3122009.317681418:1(2357-2393)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.5555/3122009.3176814
Wang TRudin CVelez-Doshi FLiu YKlampfl EMacNeille P(2016)Bayesian Rule Sets for Interpretable Classification2016 IEEE 16th International Conference on Data Mining (ICDM)10.1109/ICDM.2016.0171(1269-1274)Online publication date: Dec-2016
https://doi.org/10.1109/ICDM.2016.0171
Hellerstein L(2015)Certificate Complexity and Exact LearningEncyclopedia of Algorithms10.1007/978-3-642-27848-8_66-2(1-5)Online publication date: 25-Feb-2015
https://doi.org/10.1007/978-3-642-27848-8_66-2
Sellie LMitzenmacher M(2009)Exact learning of random DNF over the uniform distributionProceedings of the forty-first annual ACM symposium on Theory of computing10.1145/1536414.1536424(45-54)Online publication date: 31-May-2009
https://dl.acm.org/doi/10.1145/1536414.1536424
Applebaum BBarak BXiao D(2008)On Basing Lower-Bounds for Learning on Worst-Case AssumptionsProceedings of the 2008 49th Annual IEEE Symposium on Foundations of Computer Science10.1109/FOCS.2008.35(211-220)Online publication date: 25-Oct-2008
https://dl.acm.org/doi/10.1109/FOCS.2008.35
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten