arXiv:2108.09575 · Physical Review Research

Automatic differentiable numerical renormalization group

Jonas B. Rigo, Andrew K. Mitchell

Published Mar 25, 2022• 14 citations

Abstract

Machine learning techniques have recently gained prominence in physics, yielding a host of new results and insights. One key concept is that of backpropagation, which computes the exact gradient of any output of a program with respect to any input. This is achieved efficiently within the differentiable programming paradigm, which utilizes automatic differentiation (AD) of each step of a computer program and the chain rule. A classic application is in training neural networks. Here, we apply this methodology instead to the numerical renormalization group (NRG), a powerful technique in computational quantum many-body physics. We demonstrate how derivatives of NRG outputs with respect to Hamiltonian parameters can be accurately and efficiently obtained. Physical properties can be calculated using this differentiable NRG scheme---for example, thermodynamic observables from derivatives of the free energy. Susceptibilities can be computed by adding source terms to the Hamiltonian, but still evaluated with AD at precisely zero field. As an outlook, we briefly discuss the derivatives of dynamical quantities and a possible route to the vertex.

DOI: 10.1103/physrevresearch.4.013227

Related papers

Automatic differentiation for second renormalization of tensor networks

Bin-Bin Chen, Yuan Gao, Yibin Guo +7 more

Tensor renormalization group (TRG) constitutes an important methodology for accurate simulations of strongly correlated lattice models. Facilitated by the automatic differentiation technique widely used in deep learning, we propose a uniform framework of differentiable TRG ($\ensuremath{\partial}\mathrm{TRG}$) that can be applied to improve various TRG methods, in an automatic fashion. $\ensuremath{\partial}\mathrm{TRG}$ systematically extends the essential concept of second renormalization [Phys. Rev. Lett. 103, 160601 (2009)] where the tensor environment is computed recursively in the backward iteration. Given the forward TRG process, $\ensuremath{\partial}\mathrm{TRG}$ automatically finds the gradient of local tensors through backpropagation, with which one can deeply ``train'' the tensor networks. We benchmark $\ensuremath{\partial}\mathrm{TRG}$ in solving the square-lattice Ising model, and we demonstrate its power by simulating one- and two-dimensional quantum systems at finite temperature. The global optimization as well as GPU acceleration renders $\ensuremath{\partial}\mathrm{TRG}$ a highly efficient and accurate many-body computation approach.

arXiv:1912.02780Physical review. B./Physical review. BJun 23, 202032 citations

Differentiable Programming Tensor Networks

Haijun Liao, Jin-Guo Liu, Lei Wang +1 more

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search.The concept emerges from deep learning but is not limited to training neural networks.We present the theory and practice of programming tensor network algorithms in a fully differentiable way.By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation.We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations.As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation.Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-ofthe-art variational energy and magnetization with moderate efforts.Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.

arXiv:1903.09650Physical Review XSep 5, 2019251 citations

Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

Yoshiyuki Sugimoto

We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group methods. In this approach, evaluating the derivatives of the partition function up to the order of k increases the matrix-multiplication cost by a factor of ( k + 1 ) ( k + 2 ) / 2 compared to computing the free energy alone, and the memory footprint is only k + 1 times that of the original calculation. In the limit where the derivatives of the singular value decomposition are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in tensor renormalization group calculations in both two and three dimensions. In addition, we discuss how to efficiently differentiate an arbitrary tensor network.

arXiv:2602.08987Physical review. D/Physical review. D.Apr 10, 20261 citation

Tensor networks and the numerical renormalization group

Andreas Weichselbaum

The full-density-matrix numerical renormalization group has evolved as a systematic and transparent setting for the calculation of thermodynamical quantities at arbitrary temperatures within the numerical renormalization group (NRG) framework. It directly evaluates the relevant Lehmann representations based on the complete basis sets introduced by Anders and Schiller [Phys. Rev. Lett. 95, 196801 (2005)]. In addition, specific attention is given to the possible feedback from low-energy physics to high energies by the explicit and careful construction of the full thermal density matrix, naturally generated over a distribution of energy shells. Specific examples are given in terms of spectral functions (fdmNRG), time-dependent NRG (tdmNRG), Fermi-golden-rule calculations (fgrNRG) as well as the calculation of plain thermodynamic expectation values. Furthermore, based on the very fact that, by its iterative nature, the NRG eigenstates are naturally described in terms of matrix product states, the language of tensor networks has proven enormously convenient in the description of the underlying algorithmic procedures. This paper therefore also provides a detailed introduction and discussion of the prototypical NRG calculations in terms of their corresponding tensor networks.

arXiv:1209.2062Physical Review BDec 20, 201278 citations

Scheme for automatic differentiation of complex loss functions with applications in quantum physics

Chu Guo, Dario Poletti

The past few years have seen a significant transfer of tools from machine learning to solve quantum physics problems. Automatic differentiation is one standard algorithm used to efficiently compute gradients of loss functions for generic neural networks. In this work we show how to extend automatic differentiation to the case of complex loss function in a way that can be readily implemented in existing frameworks and which is compatible with the common case of real loss functions. We then combine this tool with matrix product states and apply it to compute the ground state and the steady state of a close and an open quantum system. Compared to the traditional density matrix renormalization group algorithm, complex automatic differentiation allows both straightforward GPU accelerations as well as generalizations to different types of tensor and neural networks.

arXiv:2003.04295Physical review. EJan 15, 202124 citations

Automatic differentiation in quantum chemistry

Tamayo Mendoza, Teresa

Automatic differentiation (AD) is a well-established tool that allows us to compute exact derivatives of implemented functions with respect to all their parameters. It does so without requiring any extra coding or knowledge of the analytical derivatives. Derivatives of quantum chemistry algorithms are generally very hard to obtain and often cumbersome to implement. AD shows great potential to solve this problem and to facilitate the implementation of gradients in the context of electronic structure methods. In this dissertation, we have applied AD in a broader context and demonstrate that AD tools can compute arbitrary gradients of quantum chemistry methods with respect to any input parameter and any intermediate function. To this end, we have developed the open-source code DiffiQult, a fully differentiable Hartree-Fock (HF) software package. DiffiQult is a python package that uses AD tools using the external libraries JAX and Algopy. As a proof of concept, DiffiQult is an example of an end-to-end differentiable software package of a quantum chemistry algorithm based solely on AD tools. This application has helped us understand the challenges of AD in quantum chemistry. We outline the details, benefits, and caveats of AD for the HF algorithm in this work. We have chosen HF as a starting point since it contains all the functions included in correlated electronic structure methods. In this way, it opens the door for the development of AD for more accurate algorithms. As an application, we describe DiffiQult within the framework of the fully variational HF method, where we use a gradient-based optimization method to compute system-specific basis functions. As a result, the wavefunction can capture more molecular features, and we obtain a set of basis functions for correlated methods in quantum and classical algorithms. This proves that these functions improve variationally energy while keeping a compact representation in correlated methods by interfacing with the quantum chemistry library PySCF, and with Tequila, a platform for rapid development of quantum algorithms. First, we present optimization results of minimal basis sets with respect to all parameters of the molecules H2O, CH4, NH3, HF. Later we include optimization of the potential energy surface of two isomerization mechanisms of diazene and hydrogens chains. Here, we use a minimal optimized basis set to compute FCI when possible with PySCF and VQE energies of hydrogen chains with Tequila. We obtain similar results compared to a bigger basis set with considerably fewer configurations. Also, we show how this approach improves the estimation of the thermodynamic limit in the hydrogen chain at the HF level. Finally, we apply this approach to the optimization of all atomic orbitals of benzene and compute electronic correlations restricting the active space to the 6 π orbitals of benzene using the UpCCGSD ansatz implemented in Tequila.

Jan 1, 20210 citations