Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1903.09650 · Physical Review X

Differentiable Programming Tensor Networks

Haijun Liao, Jin-Guo Liu, Lei Wang, Tao Xiang

Published Sep 5, 2019• 251 citations
arXivHTMLPDF

Abstract

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search.The concept emerges from deep learning but is not limited to training neural networks.We present the theory and practice of programming tensor network algorithms in a fully differentiable way.By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation.We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations.As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation.Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-ofthe-art variational energy and magnetization with moderate efforts.Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.

DOI
10.1103/physrevx.9.031041

Related papers

Automatic differentiation for second renormalization of tensor networks

Bin-Bin Chen, Yuan Gao, Yibin Guo +7 more

Tensor renormalization group (TRG) constitutes an important methodology for accurate simulations of strongly correlated lattice models. Facilitated by the automatic differentiation technique widely used in deep learning, we propose a uniform framework of differentiable TRG ($\ensuremath{\partial}\mathrm{TRG}$) that can be applied to improve various TRG methods, in an automatic fashion. $\ensuremath{\partial}\mathrm{TRG}$ systematically extends the essential concept of second renormalization [Phys. Rev. Lett. 103, 160601 (2009)] where the tensor environment is computed recursively in the backward iteration. Given the forward TRG process, $\ensuremath{\partial}\mathrm{TRG}$ automatically finds the gradient of local tensors through backpropagation, with which one can deeply ``train'' the tensor networks. We benchmark $\ensuremath{\partial}\mathrm{TRG}$ in solving the square-lattice Ising model, and we demonstrate its power by simulating one- and two-dimensional quantum systems at finite temperature. The global optimization as well as GPU acceleration renders $\ensuremath{\partial}\mathrm{TRG}$ a highly efficient and accurate many-body computation approach.

arXiv:1912.02780•Physical review. B./Physical review. B•Jun 23, 2020•32 citations

Implicit differentiation of tensor network algorithms

Lander Burgelman, Anna Francuz, Paul Brehmer +4 more

The current leading approach to the variational optimization of projected entangled-pair states (PEPS) is based on automatic differentiation, which allows for a convenient evaluation of the energy gradient with respect to the local variational degrees of freedom. However, evaluating the energy gradient not only remains a major computational bottleneck of the optimization procedure, but also suffers from frequent numerical instabilities. In this work, we adopt recent advances in implicit differentiation techniques to address these challenges in PEPS optimization. By reformulating the core step of the gradient computation in terms of a single characteristic equation for the contraction environment, we reduce the cost of the gradient computation and improve its scaling with the problem size. By choosing a suitable parametrization of this characteristic equation based on the intrinsic symmetries of the contraction environment, we can directly remove instabilities from the global gradient computation that would otherwise arise from the derivatives of subroutines of the contraction algorithm. Finally, we demonstrate how this approach drastically simplifies the practical implementation of stable gradient-based PEPS optimization.

quant-pharXiv:2607.15030•Jul 17, 2026•0 citations

Differentiable programming of isometric tensor networks

Chenhua Geng, Hong-Ye Hu, Yijian Zou

Abstract Differentiable programming is a new programming paradigm which enables large scale optimization through automatic calculation of gradients also known as auto-differentiation. This concept emerges from deep learning, and has also been generalized to tensor network optimizations. Here, we extend the differentiable programming to tensor networks with isometric constraints with applications to multiscale entanglement renormalization ansatz (MERA) and tensor network renormalization (TNR). By introducing several gradient-based optimization methods for the isometric tensor network and comparing with Evenbly–Vidal method, we show that auto-differentiation has a better performance for both stability and accuracy. We numerically tested our methods on 1D critical quantum Ising spin chain and 2D classical Ising model. We calculate the ground state energy for the 1D quantum model and internal energy for the classical model, and scaling dimensions of scaling operators and find they all agree with the theory well.

arXiv:2110.03898•Machine Learning Science and Technology•Jan 6, 2022•12 citations

Automatic differentiable numerical renormalization group

Jonas B. Rigo, Andrew K. Mitchell

Machine learning techniques have recently gained prominence in physics, yielding a host of new results and insights. One key concept is that of backpropagation, which computes the exact gradient of any output of a program with respect to any input. This is achieved efficiently within the differentiable programming paradigm, which utilizes automatic differentiation (AD) of each step of a computer program and the chain rule. A classic application is in training neural networks. Here, we apply this methodology instead to the numerical renormalization group (NRG), a powerful technique in computational quantum many-body physics. We demonstrate how derivatives of NRG outputs with respect to Hamiltonian parameters can be accurately and efficiently obtained. Physical properties can be calculated using this differentiable NRG scheme---for example, thermodynamic observables from derivatives of the free energy. Susceptibilities can be computed by adding source terms to the Hamiltonian, but still evaluated with AD at precisely zero field. As an outlook, we briefly discuss the derivatives of dynamical quantities and a possible route to the vertex.

arXiv:2108.09575•Physical Review Research•Mar 25, 2022•14 citations

Differentiable Quantum Programming with Unbounded Loops

Wang Sheng Fang, Mingsheng Ying, Xiaodi Wu

The emergence of variational quantum applications has led to the development of automatic differentiation techniques in quantum computing. Existing work has formulated differentiable quantum programming with bounded loops, providing a framework for scalable gradient calculation by quantum means for training quantum variational applications. However, promising parameterized quantum applications, e.g., quantum walk and unitary implementation, cannot be trained in the existing framework due to the natural involvement of unbounded loops. To fill in the gap, we provide the first differentiable quantum programming framework with unbounded loops, including a newly designed differentiation rule, code transformation, and their correctness proof. Technically, we introduce a randomized estimator for derivatives to deal with the infinite sum in the differentiation of unbounded loops, whose applicability in classical and probabilistic programming is also discussed. We implement our framework with Python and Q# and demonstrate a reasonable sample efficiency. Through extensive case studies, we showcase an exciting application of our framework in automatically identifying close-to-optimal parameters for several parameterized quantum applications.

arXiv:2211.04507•ACM Transactions on Software Engineering and Methodology•Aug 24, 2023•7 citations

Quantum Circuit Optimization using Differentiable Programming of Tensor Network States

David Rogerson, Ananda Roy

Efficient quantum circuit optimization schemes are central to quantum simulation of strongly interacting quantum many body systems. Here, we present an optimization algorithm which combines machine learning techniques and tensor network methods. The said algorithm runs on classical hardware and finds shallow, accurate quantum circuits by minimizing scalar cost functions. The gradients relevant for the optimization process are computed using the reverse mode automatic differentiation technique implemented on top of the time-evolved block decimation algorithm for matrix product states. A variation of the ADAM optimizer is utilized to perform a gradient descent on the manifolds of charge conserving unitary operators to find the optimal quantum circuit. The efficacy of this approach is demonstrated by finding the ground states of spin chain Hamiltonians for the Ising, three-state Potts and the massive Schwinger models for system sizes up to L=100. The first ten excited states of these models are also obtained for system sizes L=24. All circuits achieve high state fidelities within reasonable CPU time and modest memory requirements.

arXiv:2408.12583•Aug 22, 2024•0 citations
Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search.The concept emerges from deep learning but is not limited to training neural networks.We present the theory and practice of programming tensor network algorithms in a fully differentiable way.By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation.We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations.As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation.Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-ofthe-art variational energy and magnetization with moderate efforts.Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.
Tensor renormalization group (TRG) constitutes an important methodology for accurate simulations of strongly correlated lattice models. Facilitated by the automatic differentiation technique widely used in deep learning, we propose a uniform framework of differentiable TRG ($\ensuremath{\partial}\mathrm{TRG}$) that can be applied to improve various TRG methods, in an automatic fashion. $\ensuremath{\partial}\mathrm{TRG}$ systematically extends the essential concept of second renormalization [Phys. Rev. Lett. 103, 160601 (2009)] where the tensor environment is computed recursively in the backward iteration. Given the forward TRG process, $\ensuremath{\partial}\mathrm{TRG}$ automatically finds the gradient of local tensors through backpropagation, with which one can deeply ``train'' the tensor networks. We benchmark $\ensuremath{\partial}\mathrm{TRG}$ in solving the square-lattice Ising model, and we demonstrate its power by simulating one- and two-dimensional quantum systems at finite temperature. The global optimization as well as GPU acceleration renders $\ensuremath{\partial}\mathrm{TRG}$ a highly efficient and accurate many-body computation approach.
The current leading approach to the variational optimization of projected entangled-pair states (PEPS) is based on automatic differentiation, which allows for a convenient evaluation of the energy gradient with respect to the local variational degrees of freedom. However, evaluating the energy gradient not only remains a major computational bottleneck of the optimization procedure, but also suffers from frequent numerical instabilities. In this work, we adopt recent advances in implicit differentiation techniques to address these challenges in PEPS optimization. By reformulating the core step of the gradient computation in terms of a single characteristic equation for the contraction environment, we reduce the cost of the gradient computation and improve its scaling with the problem size. By choosing a suitable parametrization of this characteristic equation based on the intrinsic symmetries of the contraction environment, we can directly remove instabilities from the global gradient computation that would otherwise arise from the derivatives of subroutines of the contraction algorithm. Finally, we demonstrate how this approach drastically simplifies the practical implementation of stable gradient-based PEPS optimization.
Abstract Differentiable programming is a new programming paradigm which enables large scale optimization through automatic calculation of gradients also known as auto-differentiation. This concept emerges from deep learning, and has also been generalized to tensor network optimizations. Here, we extend the differentiable programming to tensor networks with isometric constraints with applications to multiscale entanglement renormalization ansatz (MERA) and tensor network renormalization (TNR). By introducing several gradient-based optimization methods for the isometric tensor network and comparing with Evenbly–Vidal method, we show that auto-differentiation has a better performance for both stability and accuracy. We numerically tested our methods on 1D critical quantum Ising spin chain and 2D classical Ising model. We calculate the ground state energy for the 1D quantum model and internal energy for the classical model, and scaling dimensions of scaling operators and find they all agree with the theory well.
Machine learning techniques have recently gained prominence in physics, yielding a host of new results and insights. One key concept is that of backpropagation, which computes the exact gradient of any output of a program with respect to any input. This is achieved efficiently within the differentiable programming paradigm, which utilizes automatic differentiation (AD) of each step of a computer program and the chain rule. A classic application is in training neural networks. Here, we apply this methodology instead to the numerical renormalization group (NRG), a powerful technique in computational quantum many-body physics. We demonstrate how derivatives of NRG outputs with respect to Hamiltonian parameters can be accurately and efficiently obtained. Physical properties can be calculated using this differentiable NRG scheme---for example, thermodynamic observables from derivatives of the free energy. Susceptibilities can be computed by adding source terms to the Hamiltonian, but still evaluated with AD at precisely zero field. As an outlook, we briefly discuss the derivatives of dynamical quantities and a possible route to the vertex.
The emergence of variational quantum applications has led to the development of automatic differentiation techniques in quantum computing. Existing work has formulated differentiable quantum programming with bounded loops, providing a framework for scalable gradient calculation by quantum means for training quantum variational applications. However, promising parameterized quantum applications, e.g., quantum walk and unitary implementation, cannot be trained in the existing framework due to the natural involvement of unbounded loops. To fill in the gap, we provide the first differentiable quantum programming framework with unbounded loops, including a newly designed differentiation rule, code transformation, and their correctness proof. Technically, we introduce a randomized estimator for derivatives to deal with the infinite sum in the differentiation of unbounded loops, whose applicability in classical and probabilistic programming is also discussed. We implement our framework with Python and Q# and demonstrate a reasonable sample efficiency. Through extensive case studies, we showcase an exciting application of our framework in automatically identifying close-to-optimal parameters for several parameterized quantum applications.
Efficient quantum circuit optimization schemes are central to quantum simulation of strongly interacting quantum many body systems. Here, we present an optimization algorithm which combines machine learning techniques and tensor network methods. The said algorithm runs on classical hardware and finds shallow, accurate quantum circuits by minimizing scalar cost functions. The gradients relevant for the optimization process are computed using the reverse mode automatic differentiation technique implemented on top of the time-evolved block decimation algorithm for matrix product states. A variation of the ADAM optimizer is utilized to perform a gradient descent on the manifolds of charge conserving unitary operators to find the optimal quantum circuit. The efficacy of this approach is demonstrated by finding the ground states of spin chain Hamiltonians for the Ising, three-state Potts and the massive Schwinger models for system sizes up to L=100. The first ten excited states of these models are also obtained for system sizes L=24. All circuits achieve high state fidelities within reasonable CPU time and modest memory requirements.