research-article

Public Access

Approximate Counting, the Lovász Local Lemma, and Inference in Graphical Models

Author:

Ankur MoitraAuthors Info & Claims

Journal of the ACM (JACM), Volume 66, Issue 2

Article No.: 10, Pages 1 - 25

https://doi.org/10.1145/3268930

Published: 05 April 2019 Publication History

All formats PDF

Abstract

In this article, we introduce a new approach to approximate counting in bounded degree systems with higher-order constraints. Our main result is an algorithm to approximately count the number of solutions to a CNF formula Φ when the width is logarithmic in the maximum degree. This closes an exponential gap between the known upper and lower bounds.

Moreover, our algorithm extends straightforwardly to approximate sampling, which shows that under Lovász Local Lemma-like conditions it is not only possible to find a satisfying assignment, it is also possible to generate one approximately uniformly at random from the set of all satisfying assignments. Our approach is a significant departure from earlier techniques in approximate counting, and is based on a framework to bootstrap an oracle for computing marginal probabilities on individual variables. Finally, we give an application of our results to show that it is algorithmically possible to sample from the posterior distribution in an interesting class of graphical models.

References

[1]

D. Achlioptas and F. Iliopoulos. 2016. Random walks that find perfect objects and the Lovász Local Lemma. J. ACM 63, 3 (2016), 22.

Digital Library

[2]

N. Alon. 1991. A parallel algorithmic version of the local lemma. Rand. Struct. Alg. 2, 4 (1991), 367--378.

Digital Library

[3]

J. Beck, 1991. An algorithmic approach to the Lovász Local Lemma. Rand. Struct. Alg. 2, 4 (1991), 343--365.

Digital Library

[4]

I. Bezákova, A. Galanis, L. A. Goldberg, H. Guo, and D. Stefankovi&cbreve;. 2016. Approximation via correlation decay when strong spatial mixing fails. In Proceedings of the International Colloquium on Automata, Languages, and Programming (ICALP’16), 1--13.

[5]

M. Bordewich, M. Dyer, and M. Karpinski. 2006. Stopping times, metrics, and approximate counting. In Proceedings of the International Colloquium on Automata, Languages, and Programming (ICALP’06), 108--119.

Digital Library

[6]

G. Bresler, E. Mossel, and A. Sly. 2013. Reconstruction of Markov random fields from samples: Some observations and algorithms. SIAM J. Comput. 42, 2 (2013), 563--578.

[7]

K. Chandrasekaran, N. Goyal, and B. Haeupler. 2013. Deterministic algorithms for the Lovász Local Lemma. SIAM J. Comput. 42, 6 (2013), 2132--2155.

[8]

P. Dagum and M. Luby. 1993. Approximating probabilistic inference in Bayesian belief networks is N P-hard. Artific. Intel. 60 (1993), 141--153.

Digital Library

[9]

P. Dagum and M. Luby. 1997. An optimal approximation algorithm for Bayesian inference. Artific. Intel. 93, 1-2 (1997), 1--27.

Digital Library

[10]

P. Erdös and L. Lovász. 1975. Problems and results on 3-chromatic hypergraphs and some related questions. In Infinite and Finite Sets. North Holland, 609--627.

[11]

A. Galanis, D. Stefankovi&cbreve;, and E. Vigoda. 2016. Inapproximability of the partition function for the antiferromagnetic Ising and hard-core models. Combin., Prob. Comput. 25, 4 (2016), 500--559.

[12]

D. Gamarnik and D. Katz. 2012. Correlation decay and deterministic FPTAS for counting colorings of a graph. J . Disc. Alg. 12 (2012), 29--47.

Digital Library

[13]

H. Guo, M. Jerrum, and J. Liu. 2017. Uniform sampling through the Lovász Local Lemma. In Proceedings of the ACM Symposium on Theory of Computing (STOC’17), 342--355.

Digital Library

[14]

H. Guo, C. Liao, P. Lu, and C. Zhang. 2018. Counting hypergraph colourings in the local lemma regime. In Proceedings of the ACM Symposium on Theory of Computing (STOC’18), 926--939.

Digital Library

[15]

B. Haeupler, B. Saha, and A. Srinivasan. 2011. New constructive aspects of the Lovász Local Lemma. In J. ACM 58, 6 (2011), 28.

Digital Library

[16]

D. Harris and A. Srinivasan. 2017. A constructive algorithm for the Lovász Local Lemma on permutations. In Theor . Comput. 13, 1 (2017), 1--41.

[17]

N. Harvey, P. Srivastava, and J. Vondrák. 2018. Computing the independence polynomial in Shearer’s region for the LLL. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA’18), 1557--1576.

Digital Library

[18]

N. Harvey and J. Vondrák. 2015. An algorithmic proof of the Lovász Local Lemma via resampling oracles. In Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS’15), 1327--1346.

Digital Library

[19]

J. Hermon, A. Sly, and Y. Zhang. 2016. Rapid mixing of hypergraph independent set. ArXiv:1610.07999.

[20]

M. Jerrum, L. Valiant, and V. Vazirani. 1986. Random generation of combinatorial structures from a uniform distribution. Theoret. Comput. Sci. 43 (1986), 169--188.

[21]

D. Knuth. 1969. The Art of Computer Programming, Vol I. Addison Wesley, London, p. 396 (exercise 11).

Digital Library

[22]

V. Kolmogorov. 2016. Commutativity in the algorithmic Lovász Local Lemma. In Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS’16), to appear.

[23]

J. Liu and P. Lu. 2015. FPTAS for counting monotone CNF. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA’15), 1531--1548.

Digital Library

[24]

R. Moser and G. Tardos. 2010. A constructive proof of the Lovász Local Lemma. J. ACM 57, 2 (2010), 1--15.

Digital Library

[25]

C. Nair and P. Tetali. 2007. The correlation decay (CD) tree and strong spatial mixing in multi-spin systems. ArXiv:0701494.

[26]

A. Sinclair and M. Jerrum. 1989. Approximately counting, uniform generation and rapidly mixing Markov chains. Inform. Comput. 82 (1989), 93--133.

Digital Library

[27]

A. Sinclair, P. Srivastava, and M. Thurley. 2014. Approximation algorithms for two-state anti-ferromagnetic spin systems. J. Stat. Phys. 155, 4 (2014), 666--686.

[28]

A. Sly. 2010. Computational transition at the uniqueness threshold. In Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS’10), 287--296.

Digital Library

[29]

A. Sly and N. Sun. 2014. Counting in two-spin models on d-regular graphs. Ann. Prob. 42, 6 (2014), 2383--2416.

[30]

D. Weitz. 2006. Counting independent sets up to the tree threshold. In Proceedings of the ACM Symposium on Theory of Computing (STOC’06), 140--149.

Digital Library

Cited By

Chen ZGalanis AGoldberg LGuo HHerrera-Poyatos AMani NMoitra A(2024)Fast Sampling of Satisfying Assignments from Random \(\boldsymbol{k}\)-SAT with Applications to ConnectivitySIAM Journal on Discrete Mathematics10.1137/23M159572238:4(2750-2811)Online publication date: 30-Oct-2024
https://doi.org/10.1137/23M1595722
Feng WGuo HWang CWang JYin Y(2023)Towards derandomising Markov chain Monte Carlo2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00120(1963-1990)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00120
Galvin DMcKinley GPerkins WSarantis MTetali P(2023)On the zeroes of hypergraph independence polynomialsCombinatorics, Probability and Computing10.1017/S0963548323000330(1-20)Online publication date: 21-Sep-2023
https://doi.org/10.1017/S0963548323000330
Show More Cited By

Index Terms

Approximate Counting, the Lovász Local Lemma, and Inference in Graphical Models
1. Mathematics of computing
  1. Discrete mathematics
    1. Combinatorics
      1. Combinatorial algorithms
  2. Probability and statistics
    1. Probabilistic algorithms
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Random walks and Markov chains

Recommendations

A constructive proof of the general lovász local lemma

The Lovász Local Lemma discovered by Erdős and Lovász in 1975 is a powerful tool to non-constructively prove the existence of combinatorial objects meeting a prescribed collection of criteria. In 1991, József Beck was the first to demonstrate that a ...
Approximate counting, the Lovasz local lemma, and inference in graphical models
STOC 2017: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

In this paper we introduce a new approach for approximately counting in bounded degree systems with higher-order constraints. Our main result is an algorithm to approximately count the number of solutions to a CNF formula Ф when the width is logarithmic ...
Directed Lovász local lemma and Shearer’s lemma
Abstract
Moser and Tardos (J. ACM (JACM) 57(2), 11 2010) gave an algorithmic proof of the lopsided Lovász local lemma (LLL) in the variable framework, where each of the undesirable events is assumed to depend on a subset of a collection of independent ...

Comments

Information & Contributors

Information

Published In

cover image Journal of the ACM

Journal of the ACM Volume 66, Issue 2

April 2019

260 pages

ISSN:0004-5411

EISSN:1557-735X

DOI:10.1145/3318168

Editor:
Éva Tardos
Cornell University

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 April 2019

Accepted: 01 November 2018

Revised: 01 October 2018

Received: 01 November 2017

Published in JACM Volume 66, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
890
Total Downloads

Downloads (Last 12 months)163
Downloads (Last 6 weeks)40

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen ZGalanis AGoldberg LGuo HHerrera-Poyatos AMani NMoitra A(2024)Fast Sampling of Satisfying Assignments from Random \(\boldsymbol{k}\)-SAT with Applications to ConnectivitySIAM Journal on Discrete Mathematics10.1137/23M159572238:4(2750-2811)Online publication date: 30-Oct-2024
https://doi.org/10.1137/23M1595722
Feng WGuo HWang CWang JYin Y(2023)Towards derandomising Markov chain Monte Carlo2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00120(1963-1990)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00120
Galvin DMcKinley GPerkins WSarantis MTetali P(2023)On the zeroes of hypergraph independence polynomialsCombinatorics, Probability and Computing10.1017/S0963548323000330(1-20)Online publication date: 21-Sep-2023
https://doi.org/10.1017/S0963548323000330
Galanis AGoldberg LŠtefankovič D(2023)Implementations and the independent set polynomial below the Shearer thresholdTheoretical Computer Science10.1016/j.tcs.2022.10.025939(194-215)Online publication date: Jan-2023
https://doi.org/10.1016/j.tcs.2022.10.025
Coja-Oghlan AMüller NRavelomanana J(2022)Belief propagation on the random k-SAT modelThe Annals of Applied Probability10.1214/21-AAP177232:5Online publication date: 1-Oct-2022
https://doi.org/10.1214/21-AAP1772
Galanis AGuo HWang J(2022)Inapproximability of Counting Hypergraph ColouringsACM Transactions on Computation Theory10.1145/355855414:3-4(1-33)Online publication date: 2-Sep-2022
https://dl.acm.org/doi/10.1145/3558554
Liu HYin YLeonardi SGupta A(2022)Simple parallel algorithms for single-site dynamicsProceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3519935.3519999(1431-1444)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519935.3519999
Huang BSellke M(2022)Tight Lipschitz Hardness for optimizing Mean Field Spin Glasses2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS54457.2022.00037(312-322)Online publication date: Oct-2022
https://doi.org/10.1109/FOCS54457.2022.00037
He KWang CYin Y(2022)Sampling Lovász local lemma for general constraint satisfaction solutions in near-linear time2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS54457.2022.00021(147-158)Online publication date: Oct-2022
https://doi.org/10.1109/FOCS54457.2022.00021
Jain VPham HVuong T(2022)Towards the sampling Lovász Local Lemma2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS52979.2021.00025(173-183)Online publication date: Feb-2022
https://doi.org/10.1109/FOCS52979.2021.00025
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents