Comment on “Controlled Bond Expansion for Density Matrix Renormalization Group Ground State Search at Single-Site Costs” (Extended Version)

In a recent Letter[1], Gleis, Li, and von Delft present an algorithm for expanding the bond dimension of a Matrix Product State wave function, giving accuracy similar to 2-site DMRG, but computationally more efficient, closer to the performance of 1-site DMRG. The Controlled Bond Expansion (CBE) algorithm uses the Hamiltonian projected onto two sites (which we refer to as the environment site and the active site), $H^{2\text{s}}\psi^{2\text{s}}$, and then further projected onto the two-site tangent space, to extract a set of $k$ vectors that are used to expand the basis between the two sites. CBE achieves this with a complicated sequence of five singular value decompositions (SVDs), in order to project onto the 2-site tangent space and reduce the bond dimension of the tensor network such that the contraction can be done in time $O(dwD^{3})$, where $w$ is the MPO bond dimension, $d$ is the local Hilbert space size, and $D$ is the MPS bond dimension. This is in addition to the usual SVD for the DMRG truncation procedure. In this Comment, we show that (1) the projection onto the 2-site tangent space is unnecessary, and is generally not helpful; (2) the sequence of 5 SVDs can be replaced by a single $QR$ decomposition (optionally with one SVD as well), making use of the randomized SVD (RSVD)[3] with high accuracy and significantly improved efficiency, scaling as $O(dwkD^{2})$ i.e., the most expensive operations are only quadratic in the bond dimension $D$ and linear in the number of expansion vectors $k$; (3) several statements in Ref. [1] about the variational properties of the CBE algorithm are incorrect, and the variational properties are essentially identical to existing algorithms including 2-site DMRG and single-site subspace expansion (3S)[4]; (4) a similar RSVD approach can be applied to the 3S algorithm, which leads to many advantages over CBE, especially in systems with long range interactions. We also make some comments on the benchmarking MPS algorithms, and the overall computational efficiency with respect to the accuracy of the calculation.

To briefly review the CBE approach, consider one step of DMRG on a right-to-left sweep. We can express the wavefunction as,

$$\ket{\Psi}=\leavevmode\hbox to77.02pt{\vbox to25.59pt{\pgfpicture\makeatletter\hbox{\hskip 6.49997pt\lower-16.83331pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{}}{{}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.75pt}{-3.5pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$\ldots$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{{}}{}{{{}}{}{}{{{{}{}{}{}}}{{}{}{}{}}}{}{}}{{}\pgfsys@moveto{21.33957pt}{5.0pt}\pgfsys@lineto{21.33957pt}{5.0pt}\pgfsys@curveto{24.10103pt}{5.0pt}{26.33957pt}{2.76146pt}{26.33957pt}{0.0pt}\pgfsys@curveto{26.33957pt}{-2.76146pt}{24.10103pt}{-5.0pt}{21.33957pt}{-5.0pt}\pgfsys@lineto{16.33957pt}{-5.0pt}\pgfsys@lineto{16.33957pt}{5.0pt}\pgfsys@closepath\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{17.65082pt}{-16.83293pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$A$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{}{{}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{47.67914pt}{0.0pt}\pgfsys@curveto{47.67914pt}{2.76146pt}{45.4406pt}{5.0pt}{42.67914pt}{5.0pt}\pgfsys@curveto{39.91768pt}{5.0pt}{37.67914pt}{2.76146pt}{37.67914pt}{0.0pt}\pgfsys@curveto{37.67914pt}{-2.76146pt}{39.91768pt}{-5.0pt}{42.67914pt}{-5.0pt}\pgfsys@curveto{45.4406pt}{-5.0pt}{47.67914pt}{-2.76146pt}{47.67914pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{42.67914pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{42.67914pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{38.74771pt}{-16.83331pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$C$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{60.2687pt}{-3.5pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$\ldots$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}{}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{21.33948pt}{5.0pt}\pgfsys@lineto{21.33948pt}{8.5566pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{42.67896pt}{5.0pt}\pgfsys@lineto{42.67896pt}{8.5566pt}\pgfsys@stroke\pgfsys@invoke{ } { {}{}{}}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{{}}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{6.99997pt}{0.0pt}\pgfsys@lineto{16.3395pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}}}}{{}}{}{}{}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{1.6pt}\pgfsys@invoke{ }{}\pgfsys@moveto{26.33945pt}{0.0pt}\pgfsys@lineto{37.67899pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}{}\pgfsys@moveto{47.67894pt}{0.0pt}\pgfsys@lineto{57.01848pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{}{{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;, \ldots A C \ldots \ldots A C \ldots$$ (1)

where the previous step optimized the site to the right of $C$ and shifted the orthogonality center to site $C$. At the current step, site $C$ is the active site and $A$ is the left-orthogonal environment site. As a starting point to expanding the bond dimension between $A$ and $C$, we can consider the action of the Hamiltonian on these two sites (effectively, a single iteration of 2-site DMRG)[5]:

$$\leavevmode\hbox to77.42pt{\vbox to50.49pt{\pgfpicture\makeatletter\hbox{\hskip 28.03944pt\lower-31.0597pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{}}{{}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{-20.33957pt}{19.22638pt}\pgfsys@lineto{-22.33957pt}{19.22638pt}\pgfsys@curveto{-24.54874pt}{19.22638pt}{-26.33957pt}{17.43555pt}{-26.33957pt}{15.22638pt}\pgfsys@lineto{-26.33957pt}{-15.22638pt}\pgfsys@curveto{-26.33957pt}{-17.43555pt}{-24.54874pt}{-19.22638pt}{-22.33957pt}{-19.22638pt}\pgfsys@lineto{-20.33957pt}{-19.22638pt}\pgfsys@curveto{-18.1304pt}{-19.22638pt}{-16.33957pt}{-17.43555pt}{-16.33957pt}{-15.22638pt}\pgfsys@lineto{-16.33957pt}{15.22638pt}\pgfsys@curveto{-16.33957pt}{17.43555pt}{-18.1304pt}{19.22638pt}{-20.33957pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{-26.33957pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{{}}{}{{{}}{}{}{{{{}{}{}{}}}{{}{}{}{}}}{}{}}{{}\pgfsys@moveto{0.0pt}{-9.22638pt}\pgfsys@lineto{0.0pt}{-9.22638pt}\pgfsys@curveto{2.76146pt}{-9.22638pt}{5.0pt}{-11.46492pt}{5.0pt}{-14.22638pt}\pgfsys@curveto{5.0pt}{-16.98784pt}{2.76146pt}{-19.22638pt}{0.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-9.22638pt}\pgfsys@closepath\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.68867pt}{-31.05931pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$A$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{}{{}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{26.33957pt}{-14.22638pt}\pgfsys@curveto{26.33957pt}{-11.46492pt}{24.10103pt}{-9.22638pt}{21.33957pt}{-9.22638pt}\pgfsys@curveto{18.57811pt}{-9.22638pt}{16.33957pt}{-11.46492pt}{16.33957pt}{-14.22638pt}\pgfsys@curveto{16.33957pt}{-16.98784pt}{18.57811pt}{-19.22638pt}{21.33957pt}{-19.22638pt}\pgfsys@curveto{24.10103pt}{-19.22638pt}{26.33957pt}{-16.98784pt}{26.33957pt}{-14.22638pt}\pgfsys@closepath\pgfsys@moveto{21.33957pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{17.40823pt}{-31.0597pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$C$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{-5.0pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{16.33957pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{43.67914pt}{19.22638pt}\pgfsys@lineto{41.67914pt}{19.22638pt}\pgfsys@curveto{39.46997pt}{19.22638pt}{37.67914pt}{17.43555pt}{37.67914pt}{15.22638pt}\pgfsys@lineto{37.67914pt}{-15.22638pt}\pgfsys@curveto{37.67914pt}{-17.43555pt}{39.46997pt}{-19.22638pt}{41.67914pt}{-19.22638pt}\pgfsys@lineto{43.67914pt}{-19.22638pt}\pgfsys@curveto{45.8883pt}{-19.22638pt}{47.67914pt}{-17.43555pt}{47.67914pt}{-15.22638pt}\pgfsys@lineto{47.67914pt}{15.22638pt}\pgfsys@curveto{47.67914pt}{17.43555pt}{45.8883pt}{19.22638pt}{43.67914pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{37.67914pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{42.67914pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{}{{}}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{0.0pt}{-9.22646pt}\pgfsys@lineto{0.0pt}{-5.0pt}\pgfsys@moveto{0.0pt}{5.0pt}\pgfsys@lineto{0.0pt}{9.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{21.3394pt}{-9.22638pt}\pgfsys@lineto{21.33948pt}{-5.0pt}\pgfsys@moveto{21.33948pt}{5.0pt}\pgfsys@lineto{21.33948pt}{9.22638pt}\pgfsys@stroke\pgfsys@invoke{ } { {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{-16.3395pt}{0.0pt}\pgfsys@lineto{-4.99997pt}{0.0pt}\pgfsys@moveto{4.99997pt}{0.0pt}\pgfsys@lineto{16.3395pt}{0.0pt}\pgfsys@moveto{26.33945pt}{0.0pt}\pgfsys@lineto{37.67899pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{{}}{}}{}{{}}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{{}}}}}}{}{}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}}}}{{}}{}{}{}{{ {}{}{}}{}{{}}{}} {}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@moveto{-16.3395pt}{-14.22638pt}\pgfsys@lineto{-4.99997pt}{-14.22638pt}\pgfsys@moveto{4.99997pt}{-14.22638pt}\pgfsys@lineto{16.3395pt}{-14.22638pt}\pgfsys@moveto{26.33945pt}{-14.22638pt}\pgfsys@lineto{37.67899pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{ {}{}{}}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{-16.3395pt}{14.22638pt}\pgfsys@lineto{-11.0046pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{ {}{}{}}{}{ {}{}{}}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{37.67899pt}{14.22638pt}\pgfsys@lineto{32.34409pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{}{{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}=\leavevmode\hbox to45.41pt{\vbox to31.26pt{\pgfpicture\makeatletter\hbox{\hskip 12.03482pt\lower-16.83331pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{}}{{}} {}{{}}{{}} {{{}{{}{}}{}}}{{ {}{}{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{22.33957pt}{5.0pt}\pgfsys@lineto{-1.0pt}{5.0pt}\pgfsys@curveto{-3.20917pt}{5.0pt}{-5.0pt}{3.20917pt}{-5.0pt}{1.0pt}\pgfsys@lineto{-5.0pt}{-1.0pt}\pgfsys@curveto{-5.0pt}{-3.20917pt}{-3.20917pt}{-5.0pt}{-1.0pt}{-5.0pt}\pgfsys@lineto{22.33957pt}{-5.0pt}\pgfsys@curveto{24.54874pt}{-5.0pt}{26.33957pt}{-3.20917pt}{26.33957pt}{-1.0pt}\pgfsys@lineto{26.33957pt}{1.0pt}\pgfsys@curveto{26.33957pt}{3.20917pt}{24.54874pt}{5.0pt}{22.33957pt}{5.0pt}\pgfsys@closepath\pgfsys@moveto{-5.0pt}{-5.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{10.66978pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{6.13503pt}{-16.83331pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$X$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{{}}{}{ {}{}{}}{}}{}{{}}{}{{}} {}{}{}\pgfsys@moveto{0.0pt}{5.0pt}\pgfsys@lineto{0.0pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}}{}{ {}{}{}}{}}{}{{}}{}{{}} {}{}{}\pgfsys@moveto{21.33948pt}{5.0pt}\pgfsys@lineto{21.33948pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{26.33945pt}{0.0pt}\pgfsys@lineto{31.67435pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } { {}{}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{-4.99997pt}{0.0pt}\pgfsys@lineto{-10.33487pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{}{{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;, A C X A C X$$ (2)

where the final tensor is denoted $X$, which we treat as a matrix reshaped to be have dimensions $dD\times dD$. We could do an SVD of $X$ and construct the $D+k$ most important left singular vectors, and use these to construct a new left-orthogonalized tensor $A^{\prime}$, thereby increasing the bond dimension from $D$ to $D+k$. However this is very expensive, $O(d^{3}D^{3})$, plus the cost of constructing $X$, so we look for a way to reduce the computational complexity. Firstly, it will generally be the case that the leading left singular vectors of $X$ have a large overlap with the states already in $A$, so we could keep those vectors and add to them the $k$ largest singular values of $X$ that are orthogonal. To do that, we project out the states in $A$ via $X_{\bar{A}}=(I-AA^{\dagger})X=X-A(A^{\dagger}X)$, treating $A$ as a $dD\times D$ matrix ($X_{\bar{A}}$ is referred to as $C^{\text{tmp}}_{l+1}$ in Ref. [1] Algorithm 1.) Once we have the $k$ largest singular vectors of $X_{\bar{A}}$, we add these as new columns of $A$, increasing its dimensions to $dD\times(D+k)$, and add corresponding environment Hamiltonian $E$-matrix elements at a cost of $O(dwkD^{2})$. The next problem is how to contract the tensor network to obtain $X_{\bar{A}}$ faster than $O(d^{2}wD^{3})$. CBE achieves this by a sequence of three SVDs to obtain an isometry $S$ (referred to as shrewd selection in Ref. [1]) that projects the left hand side of $X_{\bar{A}}$ from $dD$ states to $\hat{D}\simeq D$,

$$SX_{\bar{A}}=\leavevmode\hbox to77.42pt{\vbox to50.49pt{\pgfpicture\makeatletter\hbox{\hskip 28.03944pt\lower-19.42638pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{}}{{}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{-20.33957pt}{19.22638pt}\pgfsys@lineto{-22.33957pt}{19.22638pt}\pgfsys@curveto{-24.54874pt}{19.22638pt}{-26.33957pt}{17.43555pt}{-26.33957pt}{15.22638pt}\pgfsys@lineto{-26.33957pt}{-15.22638pt}\pgfsys@curveto{-26.33957pt}{-17.43555pt}{-24.54874pt}{-19.22638pt}{-22.33957pt}{-19.22638pt}\pgfsys@lineto{-20.33957pt}{-19.22638pt}\pgfsys@curveto{-18.1304pt}{-19.22638pt}{-16.33957pt}{-17.43555pt}{-16.33957pt}{-15.22638pt}\pgfsys@lineto{-16.33957pt}{15.22638pt}\pgfsys@curveto{-16.33957pt}{17.43555pt}{-18.1304pt}{19.22638pt}{-20.33957pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{-26.33957pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{}{{}}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{{}}{}{{{}}{}{}{{{{}{}{}{}}}{{}{}{}{}}}{}{}}{{}\pgfsys@moveto{0.0pt}{19.22638pt}\pgfsys@lineto{0.0pt}{19.22638pt}\pgfsys@curveto{2.76146pt}{19.22638pt}{5.0pt}{16.98784pt}{5.0pt}{14.22638pt}\pgfsys@curveto{5.0pt}{11.46492pt}{2.76146pt}{9.22638pt}{0.0pt}{9.22638pt}\pgfsys@lineto{-5.0pt}{9.22638pt}\pgfsys@lineto{-5.0pt}{19.22638pt}\pgfsys@closepath\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.35416pt}{24.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$S$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.09279pt}{11.6112pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$\hat{D}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{{}}{}{{{}}{}{}{{{{}{}{}{}}}{{}{}{}{}}}{}{}}{{}\pgfsys@moveto{0.0pt}{-9.22638pt}\pgfsys@lineto{0.0pt}{-9.22638pt}\pgfsys@curveto{2.76146pt}{-9.22638pt}{5.0pt}{-11.46492pt}{5.0pt}{-14.22638pt}\pgfsys@curveto{5.0pt}{-16.98784pt}{2.76146pt}{-19.22638pt}{0.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-9.22638pt}\pgfsys@closepath\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{26.33957pt}{-14.22638pt}\pgfsys@curveto{26.33957pt}{-11.46492pt}{24.10103pt}{-9.22638pt}{21.33957pt}{-9.22638pt}\pgfsys@curveto{18.57811pt}{-9.22638pt}{16.33957pt}{-11.46492pt}{16.33957pt}{-14.22638pt}\pgfsys@curveto{16.33957pt}{-16.98784pt}{18.57811pt}{-19.22638pt}{21.33957pt}{-19.22638pt}\pgfsys@curveto{24.10103pt}{-19.22638pt}{26.33957pt}{-16.98784pt}{26.33957pt}{-14.22638pt}\pgfsys@closepath\pgfsys@moveto{21.33957pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{-5.0pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{16.33957pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{43.67914pt}{19.22638pt}\pgfsys@lineto{41.67914pt}{19.22638pt}\pgfsys@curveto{39.46997pt}{19.22638pt}{37.67914pt}{17.43555pt}{37.67914pt}{15.22638pt}\pgfsys@lineto{37.67914pt}{-15.22638pt}\pgfsys@curveto{37.67914pt}{-17.43555pt}{39.46997pt}{-19.22638pt}{41.67914pt}{-19.22638pt}\pgfsys@lineto{43.67914pt}{-19.22638pt}\pgfsys@curveto{45.8883pt}{-19.22638pt}{47.67914pt}{-17.43555pt}{47.67914pt}{-15.22638pt}\pgfsys@lineto{47.67914pt}{15.22638pt}\pgfsys@curveto{47.67914pt}{17.43555pt}{45.8883pt}{19.22638pt}{43.67914pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{37.67914pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{42.67914pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{}{{}}}}}}{{}}{}{}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{0.0pt}{-9.22646pt}\pgfsys@lineto{0.0pt}{-5.0pt}\pgfsys@moveto{0.03317pt}{5.0pt}\pgfsys@lineto{0.06133pt}{9.22676pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{21.3394pt}{-9.22638pt}\pgfsys@lineto{21.33948pt}{-5.0pt}\pgfsys@moveto{21.33948pt}{5.0pt}\pgfsys@lineto{21.33948pt}{9.22638pt}\pgfsys@stroke\pgfsys@invoke{ } { {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{-16.3395pt}{0.0pt}\pgfsys@lineto{-4.99997pt}{0.0pt}\pgfsys@moveto{4.99997pt}{0.0pt}\pgfsys@lineto{16.3395pt}{0.0pt}\pgfsys@moveto{26.33945pt}{0.0pt}\pgfsys@lineto{37.67899pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{{}}{}}{}{{}}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{{}}}}}}{}{}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}}}}{{}}{}{}{}{{ {}{}{}}{}{{}}{}} {}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@moveto{-16.3395pt}{-14.22638pt}\pgfsys@lineto{-4.99997pt}{-14.22638pt}\pgfsys@moveto{4.99997pt}{-14.22638pt}\pgfsys@lineto{16.3395pt}{-14.22638pt}\pgfsys@moveto{26.33945pt}{-14.22638pt}\pgfsys@lineto{37.67899pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{{}}{}}{}{{}}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{{}}}}}}{}{}{}{}\pgfsys@moveto{-16.3395pt}{14.22638pt}\pgfsys@lineto{-4.99997pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{4.99997pt}{14.22638pt}\pgfsys@lineto{10.33487pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{ {}{}{}}{}{ {}{}{}}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{37.67899pt}{14.22638pt}\pgfsys@lineto{32.34409pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{}{{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;. S \hat{D} S \hat{D}$$ (3)

At this point the contraction can be done in cost $O(dwD^{3})$, which is the same scaling as an iteration of single-site DMRG.

2-site tangent space – The CBE algorithm includes a second projection, to project out the states used in the $C$ tensor from $X_{\bar{A}}$ prior to the SVD. This can be expressed as another projection $X_{\bar{A}}(I-VV^{\dagger})$, where $V^{\dagger}$ is the matrix of right singular vectors of $C$ ($V^{\dagger}$ is referred to as $X^{\text{orth}}_{l+1}$ in Ref. [1] Algorithm 1). This requires another SVD to obtain $V^{\dagger}$. This projection appears to be of little benefit, in fact it is not hard to construct examples where this could cause a problem. For example, consider an iteration of DMRG starting from an initial vacuum state $\ket{\cdots 00\cdots}$, and suppose on the two sites corresponding to $A$ and $C$ the Hamiltonian contains some matrix element $\ket{10}\bra{00}$. In expanding the bond dimension of the $A$ site, the state $\ket{1}$ is clearly a good choice, and the state $\ket{10}$ will be represented in the $X_{\bar{A}}$ tensor. However the CBE algorithm projects out the kept states on the right site as well, meaning any state of the form $\ket{x0}$ will be projected out, since $\ket{0}$ will be one of the right singular vectors of $C$. Hence the state $\ket{10}$ that should be included in the expanded basis will be missing. Important states that would appear in a 2-site DMRG update are therefore excluded. This is not a contrived example, as similar constructions appears in many constrained models such as gauge theories and quantum link models, typically with terms that span 3 sites (e.g., creation of a particle-antiparticle pair with an accompanying change in flux at a gauge site). In some cases the missing basis vectors would be included with additional DMRG sweeps, but this is less efficient and a real problem with TDVP[6] where such additional sweeps are not usually performed[7]. This problem can be avoided by simply omitting the projection onto the null space of $C$, and incidentally saving several $O(dD^{3})$ operations. We have not found any numerical examples where omitting this projection has a detrimental effect on the calculation.

Throughout the Letter, the authors refer to selecting expansion vectors from the orthogonal two-site tangent space, referred to as ${\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}$, the space of states that are discarded both on the $A$ site and the $C$ site. However the left singular vectors of $X_{\bar{A}}$ cannot be separated into components that are kept (or not) with respect to site $C$. In order to split the tensor $X$ into 2-site spaces ${\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{K}}$, ${\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{D}}$, ${\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{K}}$, ${\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}$, we need to first shift the orthogonality center onto the bond, by factorizing $C=\Lambda B$, where $B$ is right orthogonalized and $\Lambda$ lives on the bond between $A$ and $B$. Extending this bond basis with the null spaces $\bar{A}$ and $\bar{B}$ gives the 2-site tangent spaces as the four different combinations of $(A\oplus\bar{A})\otimes(B\oplus\bar{B})$. Expansion vectors from $\bar{A}$ cannot be selected by whether they belong (or not) to $\bar{B}$, since they are separate Hilbert spaces and that information is lost when taking the tensor product. The Hilbert space that will be optimized in the DMRG iterations is the $(D+k)\times dD$-dimensional space of $(A\oplus\tilde{A})\otimes(B\oplus\bar{B})$, where $\tilde{A}\subseteq\bar{A}$ is the carrier space of the $k$ expansion vectors. Writing the expansion tensor $X$ in partitioned form,

$$X=\begin{pmatrix}X_{{\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{K}}}&X_{{\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{D}}}\\ X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{K}}}&X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}}\end{pmatrix}\;,$$ (4)

the states already contained in $A$ are the top rows of $X$, $\begin{pmatrix}X_{{\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{K}}}&X_{{\scriptstyle\mathrm{K}}{\scriptstyle\mathrm{D}}}\end{pmatrix}$. Candidate expansion vectors come from the bottom rows, being the null space $\bar{A}$. This corresponds to the SVD of the $(d-1)D\times dD$ matrix $\begin{pmatrix}X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{K}}}&X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}}\end{pmatrix}$. CBE chooses to only consider the component $X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}}$ by projecting onto the discarded states $\bar{B}$, which sets $X_{{\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{K}}}$ to zero prior to the SVD. This throws away potentially useful information that could be used to improve the basis, and does not have the effect that the authors suggest.

Randomized SVD – Instead of using shrewd selection[1] to approximate the contraction of Eq. (2), a more efficient and accurate way to obtain the $k$ dominant singular values of $X_{\bar{A}}$ is via the Randomized SVD (RSVD)[3]. the RSVD has already been used to accelerate 2-site DMRG[9], and other algorithms[10, 11, 12]. The core of the RSVD is the range finding algorithm for finding an approximate span of the dominant singular vectors of a matrix. To find the span of the dominant $k$ left singular vectors of $X_{\bar{A}}$, we multiply it by a Gaussian random matrix $\Omega$ of dimension $dD\times k$. We then perform a $QR$ decomposition,

$$QR=X_{\bar{A}}\Omega\;,$$ (5)

where $Q$ is a $dD\times k$ left-orthogonal matrix and $R$ is $k\times k$ upper triangular. $Q$ contains an approximation of the dominant left singular vectors of $X_{\bar{A}}$. If we only need $k$ approximate singular vectors then we can stop here. But while the leading singular vectors should be represented quite accurately, the tail of the spectrum will not be accurate. To obtain an accurate spectrum for all $k$ singular values, the RSVD algorithm uses the range finding algorithm to firstly find a larger set of vectors $k+p$, where $p$ is the oversampling parameter ($p=10$ is typical)[3]. Once we have $Q$ as a $dD\times(k+p)$ dimensional matrix, we project the original matrix $X_{\bar{A}}$ into this space and perform an SVD,

$$UDV^{\dagger}=Q^{\dagger}X_{\bar{A}}\;,$$ (6)

where $D$ is the diagonal matrix of singular values, $U$ is left-orthogonal, and $V^{\dagger}$ is right-orthogonal. Projecting $U$ onto the $k$ largest singular values, $QU$ now contains the $k$ dominant singular vectors of $X_{\bar{A}}$ with high accuracy. However we have found that this step is often not necessary, and in our numerical experiments the range finding algorithm works quite well and is very cheap as it requires just one tensor network contraction and one $QR$ decomposition. The reason that the range finding works so well is that the selection of states to use as expansion vectors is not very sensitive, and in fact random vectors work reasonably well, as shown below, and also used to good effect by Oseledets and Dolgov[13]. Range finding can be seen as a semi-random algorithm that improves upon purely random vectors. RSVD interpolates between range finding for $p\rightarrow 0$, becoming equivalent to the exact SVD when $k+p$ is equal to the rank of $X_{\bar{A}}$. Importantly, the computational complexity of the RSVD and range finding algorithms are dominated by the matrix-matrix multiply of $\Omega$, which only has $k$ columns. Inserting $\Omega$ into the tensor network results in a contraction that has a computational complexity of $O(dwkD^{2})+O(d^{2}w^{2}kD)$,

$$X\Omega=\leavevmode\hbox to77.42pt{\vbox to50.49pt{\pgfpicture\makeatletter\hbox{\hskip 28.03944pt\lower-19.42638pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{}}{{}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{-20.33957pt}{19.22638pt}\pgfsys@lineto{-22.33957pt}{19.22638pt}\pgfsys@curveto{-24.54874pt}{19.22638pt}{-26.33957pt}{17.43555pt}{-26.33957pt}{15.22638pt}\pgfsys@lineto{-26.33957pt}{-15.22638pt}\pgfsys@curveto{-26.33957pt}{-17.43555pt}{-24.54874pt}{-19.22638pt}{-22.33957pt}{-19.22638pt}\pgfsys@lineto{-20.33957pt}{-19.22638pt}\pgfsys@curveto{-18.1304pt}{-19.22638pt}{-16.33957pt}{-17.43555pt}{-16.33957pt}{-15.22638pt}\pgfsys@lineto{-16.33957pt}{15.22638pt}\pgfsys@curveto{-16.33957pt}{17.43555pt}{-18.1304pt}{19.22638pt}{-20.33957pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{-26.33957pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}} {{{}{}{{}}{}}}{{{}}}{{{{}}{{}}}}{{}}{{{ }}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{26.33957pt}{14.22638pt}\pgfsys@curveto{26.33957pt}{16.98784pt}{24.10103pt}{19.22638pt}{21.33957pt}{19.22638pt}\pgfsys@curveto{18.57811pt}{19.22638pt}{16.33957pt}{16.98784pt}{16.33957pt}{14.22638pt}\pgfsys@curveto{16.33957pt}{11.46492pt}{18.57811pt}{9.22638pt}{21.33957pt}{9.22638pt}\pgfsys@curveto{24.10103pt}{9.22638pt}{26.33957pt}{11.46492pt}{26.33957pt}{14.22638pt}\pgfsys@closepath\pgfsys@moveto{21.33957pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{17.72836pt}{24.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$\Omega$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{{}}{}{{{}}{}{}{{{{}{}{}{}}}{{}{}{}{}}}{}{}}{{}\pgfsys@moveto{0.0pt}{-9.22638pt}\pgfsys@lineto{0.0pt}{-9.22638pt}\pgfsys@curveto{2.76146pt}{-9.22638pt}{5.0pt}{-11.46492pt}{5.0pt}{-14.22638pt}\pgfsys@curveto{5.0pt}{-16.98784pt}{2.76146pt}{-19.22638pt}{0.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-19.22638pt}\pgfsys@lineto{-5.0pt}{-9.22638pt}\pgfsys@closepath\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{26.33957pt}{-14.22638pt}\pgfsys@curveto{26.33957pt}{-11.46492pt}{24.10103pt}{-9.22638pt}{21.33957pt}{-9.22638pt}\pgfsys@curveto{18.57811pt}{-9.22638pt}{16.33957pt}{-11.46492pt}{16.33957pt}{-14.22638pt}\pgfsys@curveto{16.33957pt}{-16.98784pt}{18.57811pt}{-19.22638pt}{21.33957pt}{-19.22638pt}\pgfsys@curveto{24.10103pt}{-19.22638pt}{26.33957pt}{-16.98784pt}{26.33957pt}{-14.22638pt}\pgfsys@closepath\pgfsys@moveto{21.33957pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{-14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{-5.0pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {{}\pgfsys@rect{16.33957pt}{-5.0pt}{10.0pt}{10.0pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{21.33957pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{14.22638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}}{}{}{}{}{} {{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{}\pgfsys@moveto{43.67914pt}{19.22638pt}\pgfsys@lineto{41.67914pt}{19.22638pt}\pgfsys@curveto{39.46997pt}{19.22638pt}{37.67914pt}{17.43555pt}{37.67914pt}{15.22638pt}\pgfsys@lineto{37.67914pt}{-15.22638pt}\pgfsys@curveto{37.67914pt}{-17.43555pt}{39.46997pt}{-19.22638pt}{41.67914pt}{-19.22638pt}\pgfsys@lineto{43.67914pt}{-19.22638pt}\pgfsys@curveto{45.8883pt}{-19.22638pt}{47.67914pt}{-17.43555pt}{47.67914pt}{-15.22638pt}\pgfsys@lineto{47.67914pt}{15.22638pt}\pgfsys@curveto{47.67914pt}{17.43555pt}{45.8883pt}{19.22638pt}{43.67914pt}{19.22638pt}\pgfsys@closepath\pgfsys@moveto{37.67914pt}{-19.22638pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{42.67914pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{}{{}}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{0.0pt}{-9.22646pt}\pgfsys@lineto{0.0pt}{-5.0pt}\pgfsys@moveto{0.0pt}{5.0pt}\pgfsys@lineto{0.0pt}{9.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}{ {}{}{}{}{}} {{{{{}}{ {}{}}{{}{}}{}{{}{}}}{}{}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{21.3394pt}{-9.22638pt}\pgfsys@lineto{21.33948pt}{-5.0pt}\pgfsys@moveto{21.33948pt}{5.0pt}\pgfsys@lineto{21.33948pt}{9.22638pt}\pgfsys@stroke\pgfsys@invoke{ } { {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}\pgfsys@moveto{-16.3395pt}{0.0pt}\pgfsys@lineto{-4.99997pt}{0.0pt}\pgfsys@moveto{4.99997pt}{0.0pt}\pgfsys@lineto{16.3395pt}{0.0pt}\pgfsys@moveto{26.33945pt}{0.0pt}\pgfsys@lineto{37.67899pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{{}}{}}{}{{}}{}{{}} {{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{{}{}{}{}}{{}{}{}{}}{}{}{{}}}}}}{}{}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{{{{{}}{}{}{}{{}{}{}}{{}{}}{{}{{}{}{}{}}}{{}{{}{}{}{}}}{{}{}}{{{}{}}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{{}{{}{}}}{}{}{}}}}}{{}}{}{}{}{{ {}{}{}}{}{{}}{}} {}{{{{{}}{}{}{}{}{{}}}}}{{}}{}{}{}\pgfsys@moveto{-16.3395pt}{-14.22638pt}\pgfsys@lineto{-4.99997pt}{-14.22638pt}\pgfsys@moveto{4.99997pt}{-14.22638pt}\pgfsys@lineto{16.3395pt}{-14.22638pt}\pgfsys@moveto{26.33945pt}{-14.22638pt}\pgfsys@lineto{37.67899pt}{-14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{{}{}{}}{}{ {}{}{}}{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{-16.3395pt}{14.22638pt}\pgfsys@lineto{-11.0046pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}{}{{}}{} {{}{}}{}{}\pgfsys@moveto{16.3395pt}{14.22638pt}\pgfsys@lineto{11.0046pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{3.6415pt}{10.75417pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$k$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{ {}{}{}}{}{{}}{}}{}{{}}{}{{}} {{{{{}}{}{}{}{}{{}}}}}{}{}{}{}\pgfsys@moveto{37.67899pt}{14.22638pt}\pgfsys@lineto{26.33945pt}{14.22638pt}\pgfsys@stroke\pgfsys@invoke{ } {{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{}{{}{}{}}{}{{}}{}{{{}{}}{}}{{}}{} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;. \Omega k \Omega k$$ (7)

Projecting $X\Omega$ onto the null space of $A$ can now be done at a cost of $O(dkD^{2})$. So finally the most expensive operation is $O(dwkD^{2})$. If we choose $k$ to scale with $D$, e.g. $k=\delta D$, then formally the scaling is still $O(D^{3})$, but in practice the prefactor $\delta$ is usually much smaller than 1 (the authors use $\delta=0.1$, which we also find works quite well), so the cost is negligible compared with the other stages of the DMRG algorithm.

Variational properties – There are several statements in [1] about the variational properties of the CBE algorithm that are incorrect, and the variational properties are largely the same as existing algorithms including 2-site DMRG and single-site subspace expansion (3S)[4]. The authors state (bottom of page 3), “The energy minimization based on $H^{1\text{s,ex}}_{l+1}$ is variational, hence each CBE update strictly lowers the GS energy”. This is a non sequitur. The sentence immediately preceding, CBE update step (iv), is a truncation of the bond dimension from $D+\tilde{D}$ back to $D$, which involves a loss of fidelity of the wavefunction and hence an inevitable increase in the energy[15]. Similar incorrect claims are repeated in the Supplementary Information section S-3. While there is no generally accepted meaning of the phrase “fully variational”, and it is not clear exactly in what sense the authors use this phrase, the authors repeatedly imply that the CBE algorithm necessarily lowers the energy of the MPS with each step. It is important to be precise in the language on this point, because if this was really true then it would put the CBE algorithm within the class of algorithms known as Hill Climbing[16], named since every iteration of the algorithm, if the energy changes at all then it necessarily moves up hill (lower energy). Pure single-site DMRG is an example of a hill climbing algorithm. Such algorithms are notorious for slow convergence, since if the system gets into a local minima then it cannot temporarily increase the energy to escape. This explains why single-site DMRG is notoriously slow to converge, and if good quantum numbers are used then it will usually never reach the variational minimum unless the starting state is already very close, because there is no way for the algorithm to adapt the relative sizes of each quantum number sector; every choice of quantum number distribution leads to a different local minima. Fortunately, the CBE algorithm, similarly to 2-site DMRG, single-site mixing[17], 3S[4], AMEn[18] etc., is not a hill climbing algorithm, due to the truncation step (CBE update step (iv)). Just like other bond expansion schemes, the singular value decomposition at the end of the optimization phase of each step truncates the bond dimension from a larger space back to $D$ states. In 2-site DMRG and density matrix mixing (including 3S), this is a truncation from up to $dD$ non-zero singular values down to $D$. In CBE, it is a truncation from $D+k=D+\tilde{D}$ to $D$. Because of the truncation at each step, these algorithms never converge to a unique fixed point: the bond expansion breaks translation symmetry and the energy of the wavefunction is different at each active site of the lattice. The best that can be achieved with a fixed number of expansion vectors is a limit cycle, where the wavefunction returns to the same point after two complete half-sweeps; the kept and discarded states are slightly different at every step and the 2-site discarded space ${\scriptstyle\mathrm{D}}{\scriptstyle\mathrm{D}}$ is never empty. Thus the entirety of Ref. [1] endnote [29] is based on a false premise, and to the contrary the expansion vectors added to the basis drive the calculation away from the optimal fixed point of a bond dimension $D$ MPS. This is readily seen by examining the energy as a function of position in the lattice during the DMRG sweeping, as the bond expansion causes a shallow bound state across the expanded bond[19, 20], with a bond energy that can even be below that of the ground state. A close examination of the figures in Ref. [1] shows that the energy is not monotonically decreasing, although it is difficult to see as the variation is smaller than the line widths of the plots. So while the upwards fluctuations in energy in CBE might be much smaller than in 2-site DMRG, the fact that they can exist is very important for the convergence properties of the algorithm.

Figure 1: Convergence in the relative error for non-interacting fermions with PBC, otherwise same parameters as used in Ref. [1] Fig. S-6 with bond dimension $D=600$. 3S result uses a mixing parameter of $\alpha=10^{-4}$. $\text{SVD}^{\perp}$ is a baseline comparison against CBE, using a full SVD of the tensor $X_{\bar{A}}$ projected onto the 2-site tangent space. Random uses $k=\delta D$ Haar-distributed random vectors. RSVD and range finding are the $O(dwkD^{2})$ algorithms discussed in the main text. RSVD post-expansion uses a new variant of 3S where $k=\eta D$ expansion vectors are added to the MPS after the truncation step. The variational bound is the best energy we could obtain for fixed bond dimension $D=600$. In the first half of the calculation, the 2-site and $\text{SVD}^{\perp}$ lines are hidden by that of RSVD, which in turn is mainly hidden by the range finding curve.

Comparison to 3S – The idea behind the single-site density matrix mixing scheme[17] and the 3S algorithm[4] is to incorporate degrees of freedom into the kept states that are likely to be useful to represent terms in the Hamiltonian, even if they do not (yet) do anything at the active site. By adding such states to the basis throughout a sweep, matrix elements corresponding to long range interactions can be incorporated even if they never appear at any order in perturbation of 2-site projected Hamiltonians. Perhaps the clearest example is models with an inhomogeneous unit cell, such as a lattice gauge theory where gauge sites do not contain any matter degrees of freedom, and hence a 2-site update that consists of one matter and one gauge site cannot incorporate new matter quantum numbers. The mixing term overcomes this problem, but the cost is that some proportion of the kept states are ultimately not so useful with respect to the optimal ground state. The magnitude of the perturbations is controlled by the mixing parameter $\alpha$, and although one could try to gradually reduce $\alpha$ all the way to zero throughout the calculation, this is not an efficient process and there is still no guarantee that this leads to the variational minima. It is a generic feature of fixed bond dimension MPS calculations that converging much beyond the leading digit of the relative error is an inefficient use of computational resources anyway, and if you want to improve the relative error then it is much better to simply increase the bond dimension a few percent. That is, the best performance is obtained by continuing to increase the bond dimension, only stopping once the desired error is achieved. For most purposes the resulting wavefunction is good enough, but if a specific calculation requires something closer to the variational bound for a lower bond dimension, then it is often better to approach the variational energy from below, starting from a larger bond dimension and progressively reducing it, perhaps combined with a small mixing term (or bond expansion) that is smoothly taken to zero at the same time. The authors own comparisons of CBE versus 3S in figures S-7 and S-8 demonstrate this: the most efficient way to get to a given relative error is to continue increasing the bond dimension until the desired energy error is achieved. It is in this sense that 3S is much more efficient than 2-site DMRG[4]. It is surely true that CBE is more efficient than 3S in systems with short-range interactions, but the authors own calculation in figure S-7 shows that the difference is small: although there is a long tail for 3S to converge for a fixed bond dimension, this is entirely expected and the important message of figure S-7 is that the most efficient way to lower the variational energy is to continue increasing the bond dimension. Indeed, the figure shows that this only takes one additional sweep with increased bond dimension, with computational resources that are only slightly higher than CBE. We note that the authors recommend using 3S-style mixing in the early stages of a calculation anyway, but in some cases this isn’t enough and the mixing is required throughout.

None of the example calculations presented in Ref. [1] contain interactions beyond 6 sites. It has been understood for a long time[17, 4] that 2-site DMRG (and hence also CBE and other algorithms based on expanding the environment bond prior to the optimization step; we refer to this class of algorithm as local pre-expansion) can have convergence problems with long range interactions. To illustrate this, we take an example from Ref. [1], for spinful non-interacting fermions on an $L=100$ site chain but now making the boundary condition periodic, and pre-expansion algorithms struggle to converge. This is shown in Fig. [1], where we compare several variants of pre-expansion including 2-site DMRG and the randomized SVD approach described above, and density matrix mixing via the 3S algorithm[4]. All of the algorithms get temporarily stuck in a metastable excited state, which corresponds to the open boundary condition wavefunction. It takes some time for the basis to grow sufficiently to contain Hamiltonian matrix elements that connect the periodic boundary, and thereafter the convergence resumes. But even after overcoming the metastable state, local bond expansion algorithms have trouble, often encountering additional metastable plateaus, and none of the local expansion algorithms get the relative error closer than a factor $\sim 2$ of the variational bound for $D=600$ after 30 sweeps. However it is notable that all of the presented algorithms, including random expansion vectors, converge very well in the initial steps, indicating that in the absence of long range interactions the choice of algorithm is not a decisive factor. This is not a detail of our particular implementation of DMRG, and we verified that the behavior of the 2-site DMRG code in the iTensor toolkit[23] is consistent with these results. The algorithms escape from the metastable state rather chaotically, and this depends very sensitively on the numerical details. For example, decreasing the $\epsilon$ tolerance of the eigensolver might help the calculation to escape the metastability, or it might cause the calculation to fall even deeper into the metastable state. We don’t know of any systematic way to tune these parameters in this situation, so we simply used the recommended default parameters used by the Matrix Product Toolkit[25]. The random, RSVD, and range finding algorithms use randomized sketching and are therefore non-deterministic; for these cases we ran 10 instances of the same calculation and chose a sample that best represents the median convergence. The separate runs produce near-identical results for the first half of the calculation, and only differ once they escape the metastable state. However the distribution of the results for each algorithm is quite distinct such that the depiction in Fig. [1] is representative of the expected relative performance. Fig. [1] also includes a calculation using post-expansion, which is a term we use to describe the class of algorithms that expand the basis after the optimisation step. This class includes the 3S algorithm as well as a new algorithm that uses the randomized SVD of the 3S mixing term to construct $k$ expansion vectors in the null space of $C$, which are added basis after the truncation giving an MPS with a bond dimension of $D+k$. This can also be done in computation time $O(dwkD^{2})$, and has an appealing physical interpretation. Full details of these algorithms will be published separately[26].

Appendix A Appendix A: Additional details and benchmarks

Figure 2: All runs of the randomized algorithms from Fig. [1].

In Fig. [1], four of the algorithms are based on randomized sketching (random, RSVD, range finding, RSVD post-expansion). While the performance of these algorithms is generally not random, the escape from a metastable state is rather chaotic and small changes can affect the calculation, even for notionally deterministic algorithms. To obtain Fig. [1] for the randomized algorithms, we ran each calculation 10 times and selected one run that we thought best represented the median performance. We show all of the runs in Fig. [2]. Four out of the ten runs using the range finding algorithm actually found a lower energy state than 2-site DMRG, showing that some randomness can help escape metastability, even though the 2-site algorithm is exploring a larger local Hilbert space.

Figure 3: Log-log plot of the relative error in the energy versus CPU time in seconds for the $3S$, pre-expansion and post-expansion algorithms from Fig. [1]. Solid lines are the bond expansion phase, as the bond dimension $D$ is increased to $600$. Dashed lines are a further 11 sweeps with $D=600$.

Fig. [3] shows the CPU time for the $3S$, pre-expansion and post-expansion algorithms. Calculations were performed on a single core Intel Core i7-12700KF CPU, with double precision complex arithmetic. With the local Hilbert space dimension of 3 multiplets, the $O(d)$ vs $O(d^{2})$ performance improvement of the new algorithms is not large. We argue in the main text that the most relevant performance comparison is during the bond growth phase of the calculation, which is indicated in solid lines, transitioning to dashed lines for the final part of the calculation with fixed $D$.

Appendix B Appendix B: Hubbard-Holstein model

Figure 4: Convergence of the Hubbard-Holstein model, energy versus number of half-sweeps. The random, range finding, and RSVD algorithms all use $\delta=0.1$ bond expansion. Post-exp is post-expansion using RSVD with $p=10$ oversampling. The inset shows the detailed convergence over the last two sweeps where the bond dimension is fixed at $D=600$. The black dashed line is an estimated variational bound for $D=600$. Post-expansion ads $\eta=0.1$ expansion states after the truncation phase, leading to an MPS with a final bond dimension of $D(1+\eta)=660$.

As a second benchmark, we examine the Hubbard-Holstein model, using the same parameters as Fig. 3 of Ref. [1], with maximum phonon occupancy of 3. Focusing on bond dimension $D=600$, Fig. [4] shows the relative energy error per sweep for a selection of different local pre-expansion algorithms. Full SVD is a baseline comparison using the full SVD of the expansion tensor $X_{\bar{A}}$ (note we did not project out the kept states of the $C$ tensor). Random vectors do not work so well in this model, probably because of the large local Hilbert space dimension of 12 states means that the $k=0.1D=60$ expansion states out of a possible 660 has a relatively low probability of making a good selection. The range finding algorithm works very well though. Note that for the random and range finding algorithms, we force a minimum of 1 state per quantum number sector into the expansion basis. Without sampling all of the available symmetry sectors, the random and range finding algorithms are ineffective. We also show three settings for oversampling of the RSVD; the baseline $p=10$ oversampling suggested in Ref. [3], and multiplicative oversampling with $p=\min(k+10,\kappa k)$, for $\kappa=1$ and $\kappa=3$. Multiplicative oversampling has a theoretical truncation error of a factor $\simeq 1+1/\kappa$ larger than the exact SVD[3], and $\kappa=1$ is more than sufficient for this calculation. We also show 3S and 2-site DMRG results for comparison. Aside from the random and 3S curves, the remaining curves are basically on top of each other. The inset shows the detail of the convergence in the final sweeps. For pre-expansion algorithms, the energy can drop below the variational bound for $D=600$ because of the larger Hilbert space during the bond expansion, which leads to a shallow bound state and a lower energy at that bond[19, 20]. At the end of the sweep, when all of the MPS bonds are dimension $\leq 600$, the energy is above the estimated variational bound. Because the RSVD post-expansion algorithm results in an MPS with an enlarged bond dimension $D(1+\eta)$, it is allowed that the final energy is below the estimated variational bound and this actually occurs at the end of the calculation.

Figure 5: CPU time (seconds) for the Hubbard-Holstein model.

For completeness, we show the CPU time for the Hubbard-Holstein calculation in Fig. [5]. All of the pre-expansion and post-expansion algorithms perform similarly. $3S$ suffers from the larger local Hilbert space due to the $O(d^{2})$ scaling. Random pre-expansion has a somewhat worse energy, and post-expansion is somewhat slower because both sides of the active tensor have an expanded bond dimension, i.e., in pre-expansion on a right-to-left sweep the $C$ tensor has dimension $(D+k)\times d\times D$, but for post-expansion it has dimension $(D+k)\times d\times(D+k)$. However, post-expansion has a significant advantage, in that it works well even when there are long-range interactions. For the Hubbard-Holstein model this means that it is possible to factorize the unit cell of the lattice into separate fermion and phonon sites, which enables a significant increase the number of phonon modes. With $d_{\text{ph}}$ phonon modes (corresponding to a maximum phonon occupancy of $d_{\text{ph}}-1$), the coarse-grained lattice site has a local Hilbert space of $3d_{\text{ph}}$ multiplets ($4d_{\text{ph}}$ without using $SU(2)$ symmetry for the fermions), whereas splitting the unit cell into two lattice sites gives one site for 3 fermion states and one site for the $d_{\text{ph}}$ phonon modes. In principle the CPU time is then proportional to $3+d_{\text{ph}}$, rather than $3d_{\text{ph}}$. To tackle even larger values of $d_{\text{ph}}$, the phonon modes can themselves be factorized into multiple sites[29]. Unfortunately, this procedure fails completely for local pre-expansion algorithms, including 2-site DMRG, because on every pair of neighboring lattice sites there is only at most one fermion site, and hence expanding the bond cannot introduce new fermionic quantum numbers.

Figure 6: Convergence of the Hubbard-Holstein model increasing the maximum phonon number to $8$ ($d_{\text{ph}}=9$ phonon modes). For comparison purposes, the $D=600$ result for 3 phonons ($d_{\text{ph}}=4$) is included showing that the limiting factor affecting the ground state energy is the number of phonon modes. The coarse-grained curve uses the basis of the combined fermion+phonon. All pre-expansion algorithms fail completely and the energy stays at 0.

On increasing the maximum number of phonons, the advantage of using a fine-grained unit cell becomes very clear. Fig. [6] shows the convergence for the same calculation as Fig. [4] but now with up to 8 phonons per site ($d_{\text{ph}}=9$). Using a unit cell of two sites, there are now a total of 100 sites in the lattice (50 fermion sites and 50 phonon sites), and post-expansion works extremely well, producing an energy that is nearly as good as for the coarse-grained lattice. Splitting the fermion and phonon sites slightly restricts the entanglement allowed between them compared with the coarse-grained lattice, which affects the energy slightly. This can be compensated by increasing the bond dimension, and $D=680$ is sufficient here. Pre-expansion algorithms, including 2-site DMRG, fail completely with the factorized basis.

Figure 7: CPU time for the calculation of Fig. [6].

The CPU time for this calculation is shown in Fig. [7], which shows the expected $\simeq 2$ speedup from using a factorized basis, going from local Hilbert space sizes of $3\times 9=27$ to $3+9=12$. This advantage clearly gets bigger for even larger phonon numbers needed for the Holstein model when the zero mode is not removed from the basis[29, 30].

Appendix C Appendix C: Random sketching versus shrewd selection

Figure 8: Comparison using randomized sketching for the expansion vectors using the same parameters as figure S-5 of Ref. [1]. (a) the singular values of the expansion matrix $X$ without projecting onto the two-site tangent space and $X_{\bar{A}}$ with the projection. (b) difference in the variational energy from the exact solution, starting from a $D=1$ valence bond state increasing to $D=300$. The curve for the range finding algorithm is almost indistinguishable from (and hidden by) the RSVD curve. For the RSVD, we used oversampling factor $\kappa=1$, with a minimum oversample of 10.

In this section, we compare in detail the effectiveness of the algorithms based on random sketching. In Fig. [8] (a), we compare the singular values of the expansion tensor $X$ with and without the projection onto the 2-site tangent space, for bond dimension $D=300$ at the center of the $L=20$ site chain. The projection makes very little difference to the top part of the spectrum, except to give very low weight to some states that would otherwise appear around the middle of the spectrum. The bulk of the spectrum has a very steady exponential decay, which explains why random expansion vectors work so well: a Haar random vector has an expected weight of nearly 10% of the leading singular value. Fig. [8] (b) shows the convergence of the energy for the different algorithms. The full SVD is included as a baseline. As expected from the unchanged top part of the spectrum in Fig. [8] (a), it makes no difference in this case whether the expansion tensor is projected onto the two-site tangent space or not. As best as we can tell, random vectors are already very close to the ‘moderate pre-selection’ line of figure S-5 of Ref. [1], and range-finding exeeds it. However random vectors are much less effective when only a few states from the local Hibert space are used, such as the Hubbard-Holstein model. In that case, the phonon number distribution is far from uniform, and only a small subset of the null space are effective expansion vectors and we recommend that at least range-finding is used, and preferably the RSVD. When using good quantum numbers, there is a question of how to choose the quantum number sectors of the expanded states. Our approach is to choose sectors such that the distribution of quantum numbers matches the the states already kept, and in addition we require at least one expansion vector in each available quantum number sector. This typically results in slightly more than $\delta D$ expansion vectors, but the cost of this is negligible as the computation time is dominated by the matrix multiplies in the largest quantum number sectors. For the post-expansion algorithm, this requirement to explore all candidate quantum number sectors means that random vectors or random range-finding algorithms are not so effective, since there is no singular value truncation to control the number of quantum number sectors, which can therefore get very large. For this reason, we recommend using the RSVD with post-expansion, since the singular value selection serves to prevent the number of quantum number sectors from growing out of control. But even without quantum number symmetries, it is best to use the RSVD with post expansion. This is because in post-expansion the vectors are used for the entire sweep so it is more important that the vectors are chosen as to have a high probability of contributing to the converged wavefunction. In contrast, in the pre-expansion algorithm the vectors are selected from the null space of the environment tensor and they are discarded at the end of the current iteration, so a single iteration with an unlucky or poor selection of expansion vectors is of no major consequence, as long as there is a good average selection throughout multiple sweeps.