Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2311.11894 · Physical Review Research

Stable and efficient differentiation of tensor network algorithms

Anna Francuz, Norbert Schuch, Bram Vanhecke

Published Mar 4, 2025• 13 citations
arXivHTMLPDF

Abstract

Gradient-based optimization methods are the established state-of-the-art paradigm to study strongly entangled quantum systems in two dimensions with projected entangled pair states. However, the key ingredient, the gradient itself, has proven challenging to calculate accurately and reliably in the case of a corner transfer matrix-based approach. Automatic differentiation (AD), which is the best known tool for calculating the gradient, still suffers some crucial shortcomings. Some of these are known, like the problem of excessive memory usage and the divergences which may arise when differentiating a singular value decomposition (SVD). Importantly, we find that there is also a fundamental inaccuracy in the currently used backpropagation of SVD that had not been noted before. In this paper, we describe all these problems and provide them with compact and easy to implement solutions. We analyze the impact of these changes and find that the last problem—the use of the correct gradient—is by far the dominant one and thus should be considered a crucial patch to any AD application that makes use of an SVD for truncation.

DOI
10.1103/physrevresearch.7.013237

Related papers

Implicit differentiation of tensor network algorithms

Lander Burgelman, Anna Francuz, Paul Brehmer +4 more

The current leading approach to the variational optimization of projected entangled-pair states (PEPS) is based on automatic differentiation, which allows for a convenient evaluation of the energy gradient with respect to the local variational degrees of freedom. However, evaluating the energy gradient not only remains a major computational bottleneck of the optimization procedure, but also suffers from frequent numerical instabilities. In this work, we adopt recent advances in implicit differentiation techniques to address these challenges in PEPS optimization. By reformulating the core step of the gradient computation in terms of a single characteristic equation for the contraction environment, we reduce the cost of the gradient computation and improve its scaling with the problem size. By choosing a suitable parametrization of this characteristic equation based on the intrinsic symmetries of the contraction environment, we can directly remove instabilities from the global gradient computation that would otherwise arise from the derivatives of subroutines of the contraction algorithm. Finally, we demonstrate how this approach drastically simplifies the practical implementation of stable gradient-based PEPS optimization.

quant-pharXiv:2607.15030•Jul 17, 2026•0 citations

Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

Yoshiyuki Sugimoto

We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group methods. In this approach, evaluating the derivatives of the partition function up to the order of k increases the matrix-multiplication cost by a factor of ( k + 1 ) ( k + 2 ) / 2 compared to computing the free energy alone, and the memory footprint is only k + 1 times that of the original calculation. In the limit where the derivatives of the singular value decomposition are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in tensor renormalization group calculations in both two and three dimensions. In addition, we discuss how to efficiently differentiate an arbitrary tensor network.

arXiv:2602.08987•Physical review. D/Physical review. D.•Apr 10, 2026•1 citation

Differentiable Programming Tensor Networks

Haijun Liao, Jin-Guo Liu, Lei Wang +1 more

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search.The concept emerges from deep learning but is not limited to training neural networks.We present the theory and practice of programming tensor network algorithms in a fully differentiable way.By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation.We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations.As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation.Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-ofthe-art variational energy and magnetization with moderate efforts.Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.

arXiv:1903.09650•Physical Review X•Sep 5, 2019•251 citations

Towards next-generation methods to optimize two-dimensional tensor networks: Algorithmic differentiation and applications to quantum magnets

Juraj Hašík

Infinite Projected entangled-pair state (iPEPS) is a two-dimensional tensor network (TN) ansatz targeting the ground states of lattice models directly in the thermodynamic limit. It is the most successful among the true two-dimensional TN methods, challenging the more common density matrix renormalization group computations conducted on a finite-width cylinders. iPEPS, being a variational method does not suffer from the infamous sign problem, and hence could be readily applied to both fermionic problems off the half-filling and frustrated spin system. Using the imaginary-time optimization, the so called Full Update, led to the celebrated results of iPEPS such as the analysis of stripes in doped t-J and Hubbard models or the insight into the magnetization plateaus in Shastry-Sutherland model. However, the physics of the continuous phase transitions from the symmetry-broken Neel phase into the valence-bond solids or spin-liquids in frustrated magnets is still eluding iPEPS. The main difficulty lies in the optimization, since Full Update is just not precise enough. In the pursuit of the best variational iPEPS states, the gradient methods were put forward in 2016 by Corboz and Vanderstraeten. However, due to their complex nature, they have not been widely adopted especially for iPEPS ansatze with larger unit cells. We will introduce an alternative approach first applied in the context of iPEPS by Liao et al., based on the well established method of Algorithmic differentiation (AD) which allows us to evaluate gradients for iPEPS in a robust and conceptually simple way. Finally, we will present the results of our investigation into the physics of frustrated magnets putting AD to work.

•Oct 25, 2019•3 citations

An introduction to infinite projected entangled-pair state methods for variational ground state simulations using automatic differentiation

Jan Naumann, Erik Lennart Weerda, Matteo Rizzi +2 more

Tensor networks capture large classes of ground states of phases of quantum matter faithfully and efficiently. Their manipulation and contraction has remained a challenge over the years, however. For most of the history, ground state simulations of two-dimensional quantum lattice systems using (infinite) projected entangled pair states have relied on what is called a time-evolving block decimation. In recent years, multiple proposals for the variational optimization of the quantum state have been put forward, overcoming accuracy and convergence problems of previously known methods. The incorporation of automatic differentiation in tensor networks algorithms has ultimately enabled a new, flexible way for variational simulation of ground states and excited states. In this work we review the state-of-the-art of the variational iPEPS framework, providing a detailed introduction to automatic differentiation, a description of a general foundation into which various two-dimensional lattices can be conveniently incorporated, and demonstrative benchmarking results.

arXiv:2308.12358•SciPost Physics Lecture Notes•Sep 10, 2024•18 citations

Automatic differentiation applied to excitations with projected entangled pair states

Boris Ponsioen, Fakher F. Assaad, Philippe Corboz

The excitation ansatz for tensor networks is a powerful tool for simulating the low-lying quasiparticle excitations above ground states of strongly correlated quantum many-body systems. Recently, the two-dimensional tensor network class of infinite projected entangled-pair states gained new ground state optimization methods based on automatic differentiation, which are at the same time highly accurate and simple to implement. Naturally, the question arises whether these new ideas can also be used to optimize the excitation ansatz, which has recently been implemented in two dimensions as well. In this paper, we describe a straightforward way to reimplement the framework for excitations using automatic differentiation, and demonstrate its performance for the Hubbard model at half filling.

arXiv:2107.03399•SciPost Physics•Jan 6, 2022•48 citations
Gradient-based optimization methods are the established state-of-the-art paradigm to study strongly entangled quantum systems in two dimensions with projected entangled pair states. However, the key ingredient, the gradient itself, has proven challenging to calculate accurately and reliably in the case of a corner transfer matrix-based approach. Automatic differentiation (AD), which is the best known tool for calculating the gradient, still suffers some crucial shortcomings. Some of these are known, like the problem of excessive memory usage and the divergences which may arise when differentiating a singular value decomposition (SVD). Importantly, we find that there is also a fundamental inaccuracy in the currently used backpropagation of SVD that had not been noted before. In this paper, we describe all these problems and provide them with compact and easy to implement solutions. We analyze the impact of these changes and find that the last problem—the use of the correct gradient—is by far the dominant one and thus should be considered a crucial patch to any AD application that makes use of an SVD for truncation.
The current leading approach to the variational optimization of projected entangled-pair states (PEPS) is based on automatic differentiation, which allows for a convenient evaluation of the energy gradient with respect to the local variational degrees of freedom. However, evaluating the energy gradient not only remains a major computational bottleneck of the optimization procedure, but also suffers from frequent numerical instabilities. In this work, we adopt recent advances in implicit differentiation techniques to address these challenges in PEPS optimization. By reformulating the core step of the gradient computation in terms of a single characteristic equation for the contraction environment, we reduce the cost of the gradient computation and improve its scaling with the problem size. By choosing a suitable parametrization of this characteristic equation based on the intrinsic symmetries of the contraction environment, we can directly remove instabilities from the global gradient computation that would otherwise arise from the derivatives of subroutines of the contraction algorithm. Finally, we demonstrate how this approach drastically simplifies the practical implementation of stable gradient-based PEPS optimization.
We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group methods. In this approach, evaluating the derivatives of the partition function up to the order of k increases the matrix-multiplication cost by a factor of ( k + 1 ) ( k + 2 ) / 2 compared to computing the free energy alone, and the memory footprint is only k + 1 times that of the original calculation. In the limit where the derivatives of the singular value decomposition are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in tensor renormalization group calculations in both two and three dimensions. In addition, we discuss how to efficiently differentiate an arbitrary tensor network.
Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search.The concept emerges from deep learning but is not limited to training neural networks.We present the theory and practice of programming tensor network algorithms in a fully differentiable way.By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation.We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations.As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation.Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-ofthe-art variational energy and magnetization with moderate efforts.Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.
Infinite Projected entangled-pair state (iPEPS) is a two-dimensional tensor network (TN) ansatz targeting the ground states of lattice models directly in the thermodynamic limit. It is the most successful among the true two-dimensional TN methods, challenging the more common density matrix renormalization group computations conducted on a finite-width cylinders. iPEPS, being a variational method does not suffer from the infamous sign problem, and hence could be readily applied to both fermionic problems off the half-filling and frustrated spin system. Using the imaginary-time optimization, the so called Full Update, led to the celebrated results of iPEPS such as the analysis of stripes in doped t-J and Hubbard models or the insight into the magnetization plateaus in Shastry-Sutherland model. However, the physics of the continuous phase transitions from the symmetry-broken Neel phase into the valence-bond solids or spin-liquids in frustrated magnets is still eluding iPEPS. The main difficulty lies in the optimization, since Full Update is just not precise enough. In the pursuit of the best variational iPEPS states, the gradient methods were put forward in 2016 by Corboz and Vanderstraeten. However, due to their complex nature, they have not been widely adopted especially for iPEPS ansatze with larger unit cells. We will introduce an alternative approach first applied in the context of iPEPS by Liao et al., based on the well established method of Algorithmic differentiation (AD) which allows us to evaluate gradients for iPEPS in a robust and conceptually simple way. Finally, we will present the results of our investigation into the physics of frustrated magnets putting AD to work.
Tensor networks capture large classes of ground states of phases of quantum matter faithfully and efficiently. Their manipulation and contraction has remained a challenge over the years, however. For most of the history, ground state simulations of two-dimensional quantum lattice systems using (infinite) projected entangled pair states have relied on what is called a time-evolving block decimation. In recent years, multiple proposals for the variational optimization of the quantum state have been put forward, overcoming accuracy and convergence problems of previously known methods. The incorporation of automatic differentiation in tensor networks algorithms has ultimately enabled a new, flexible way for variational simulation of ground states and excited states. In this work we review the state-of-the-art of the variational iPEPS framework, providing a detailed introduction to automatic differentiation, a description of a general foundation into which various two-dimensional lattices can be conveniently incorporated, and demonstrative benchmarking results.
The excitation ansatz for tensor networks is a powerful tool for simulating the low-lying quasiparticle excitations above ground states of strongly correlated quantum many-body systems. Recently, the two-dimensional tensor network class of infinite projected entangled-pair states gained new ground state optimization methods based on automatic differentiation, which are at the same time highly accurate and simple to implement. Naturally, the question arises whether these new ideas can also be used to optimize the excitation ansatz, which has recently been implemented in two dimensions as well. In this paper, we describe a straightforward way to reimplement the framework for excitations using automatic differentiation, and demonstrate its performance for the Hubbard model at half filling.