Solving statistical mechanics on sparse graphs with feedback-set variational autoregressive networks

Feng Pan, Pengfei Zhou, Hai-Jun Zhou, and Pan Zhang

Phys. Rev. E 103, 012103 – Published 5 January 2021

Abstract

We propose a method for solving statistical mechanics problems defined on sparse graphs. It extracts a small feedback vertex set (FVS) from the sparse graph, converting the sparse system to a much smaller system with many-body and dense interactions with an effective energy on every configuration of the FVS, then learns a variational distribution parametrized using neural networks to approximate the original Boltzmann distribution. The method is able to estimate free energy, compute observables, and generate unbiased samples via direct sampling without autocorrelation. Extensive experiments show that our approach is more accurate than existing approaches for sparse spin glasses. On random graphs and real-world networks, our approach significantly outperforms the standard methods for sparse systems, such as the belief-propagation algorithm; on structured sparse systems, such as two-dimensional lattices our approach is significantly faster and more accurate than recently proposed variational autoregressive networks using convolution neural networks.

Received 22 May 2020
Revised 3 September 2020
Accepted 16 November 2020

DOI:https://doi.org/10.1103/PhysRevE.103.012103

Physics Subject Headings (PhySH)

Classical statistical mechanics Complex systems

Artificial neural networks Random graphs Spin glasses

Ising model Variational approach

Statistical Physics & Thermodynamics

Authors & Affiliations

Feng Pan ^1,2, Pengfei Zhou ^1,2, Hai-Jun Zhou ^1,2,*, and Pan Zhang^1,3,4,†

¹CAS Key Laboratory for Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China
²School of Physical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China
³School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study, UCAS, Hangzhou 310024, China
⁴International Centre for Theoretical Physics Asia-Pacific, Beijing/Hangzhou, China

^*zhouhj@itp.ac.cn
^†panzhang@itp.ac.cn

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand

Issue

Vol. 103, Iss. 1 — January 2021

Reuse & Permissions

Access Options

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
Correlations of spin glass models obtained by our method (FVS), BP, and dDC compared with data given by MCMC running for a long time ( $5 \times 10^{5} n$ steps) on various of graphs. (a) The Viana-Bray spin glass model [20] on a random regular graph, $n = 300$ spins, degree 4, $β = 0.8$ , couplings $J_{i j} \in {+ 1, - 1}$ with $P (J_{i j} = 1) = P (J_{i j} = - 1) = \frac{1}{2}$ ; (b) the Erdös-Rényi random graphs, $n = 1100$ spins, average degree 3, $β = 0.8$ , and couplings are Gaussian random variables with zero mean and unit variance; (c) the model is the same as that of (b) but on the real-world karate club network [21] with $n = 34$ variables, average degree 2.29, and $β = 0.54$ ; (d) the same as (c) but on the real-world political blogs network [22] with $n = 1490$ variables, average degree 11.21, and $β = 0.1$ .
Reuse & Permissions
Figure 2
Free energy (per spin) of the Ising model on the $16 \times 16$ lattice with open boundary condition obtained by our method (FVS), original two versions proposed in Ref. [8] (Dense, Conv) and belief propagation (Bethe) with their relative errors to the exact solution [23]. The vertical dashed line in the inset represents the phase transition point of an infinite system ( $β = 0.4406 868$ ).
Reuse & Permissions
Figure 3
(a) Evolution of the variational free energy through training. Solid lines represent mean of $E (s) + \frac{1}{β} ln q_{θ} (s)$ , and shaded areas represent standard derivation. In the inset, a longer timescale of 5000 epochs is illustrated; (b) Time used for one epoch (training step) in seconds. Each point is the average value over 100 instances. Here FVS is our approach whereas Dense represents densely connected VAN in Ref. [8].
Reuse & Permissions
Figure 4
Inference in censored block model on a small-world network with $n = 1000$ and $p = 0.2$ versus the noise parameter $α$ of censoring. (a) Free energy (per node, the smaller the better) of BP and our method (FVS). (b) Fraction overlap (11) comparison, the FVS points are averaged over ten instances whereas the BP results are averaged over 300 instances due to the large error bars.
Reuse & Permissions
Figure 5
Workflow of a variational autoregressive network.
Reuse & Permissions
Figure 6
Relative errors on free energy given by our method with different FVSes (left panel) and different orderings (right panel) compared to the BP which are shown as the right most ones in the figures. Labels in the left panel represent sizes of the FVSes whereas labels in the right panel represent ordering 1–10. The physical model here is the $16 \times 16$ 2D ferromagnetic Ising model with the open boundary condition and the inverse temperature $β = 1.0$ .
Reuse & Permissions

Physical Review E

covering statistical, nonlinear, biological, and soft matter physics