© 2026 Aqora Quantum S.A.S.

Terms Privacy Legal Notice

arXiv:2111.14046

Neural Tangent Kernel of Matrix Product States: Convergence and Applications

Erdong Guo, David Draper

Published Nov 28, 2021• 0 citations

Abstract

In this work, we study the Neural Tangent Kernel (NTK) of Matrix Product States (MPS) and the convergence of its NTK in the infinite bond dimensional limit. We prove that the NTK of MPS asymptotically converges to a constant matrix during the gradient descent (training) process (and also the initialization phase) as the bond dimensions of MPS go to infinity by the observation that the variation of the tensors in MPS asymptotically goes to zero during training in the infinite limit. By showing the positive-definiteness of the NTK of MPS, the convergence of MPS during the training in the function space (space of functions represented by MPS) is guaranteed without any extra assumptions of the data set. We then consider the settings of (supervised) Regression with Mean Square Error (RMSE) and (unsupervised) Born Machines (BM) and analyze their dynamics in the infinite bond dimensional limit. The ordinary differential equations (ODEs) which describe the dynamics of the responses of MPS in the RMSE and BM are derived and solved in the closed-form. For the Regression, we consider Mercer Kernels (Gaussian Kernels) and find that the evolution of the mean of the responses of MPS follows the largest eigenvalue of the NTK. Due to the orthogonality of the kernel functions in BM, the evolution of different modes (samples) decouples and the "characteristic time" of convergence in training is obtained.

DOI: 10.48550/arxiv.2111.14046

Related papers

Analyzing Convergence in Quantum Neural Networks: Deviations from Neural Tangent Kernels

Xuchen You, Shouvanik Chakrabarti, Boyang Chen +1 more

A quantum neural network (QNN) is a parameterized mapping efficiently implementable on near-term Noisy Intermediate-Scale Quantum (NISQ) computers. It can be used for supervised learning when combined with classical gradient-based optimizers. Despite the existing empirical and theoretical investigations, the convergence of QNN training is not fully understood. Inspired by the success of the neural tangent kernels (NTKs) in probing into the dynamics of classical neural networks, a recent line of works proposes to study over-parameterized QNNs by examining a quantum version of tangent kernels. In this work, we study the dynamics of QNNs and show that contrary to popular belief it is qualitatively different from that of any kernel regression: due to the unitarity of quantum operations, there is a non-negligible deviation from the tangent kernel regression derived at the random initialization. As a result of the deviation, we prove the at-most sublinear convergence for QNNs with Pauli measurements, which is beyond the explanatory power of any kernel regression dynamics. We then present the actual dynamics of QNNs in the limit of over-parameterization. The new dynamics capture the change of convergence rate during training and implies that the range of measurements is crucial to the fast QNN convergence.

arXiv:2303.14844•Mar 26, 2023•3 citations

Evaluating the Convergence Limit of Quantum Neural Tangent Kernel

Duong, Trong

Quantum variational algorithms have been one of major applications of quantum computing with current quantum devices. There are recent attempts to establish the foundation for these algorithms. A possible approach is to characterize the training dynamics with quantum neural tangent kernel. In this work, we construct the kernel for two models, Quantun Ensemble and Quantum Neural Network, and show the convergence of these models in the limit of infinitely many qubits. We also show applications of the kernel limit in regression tasks.

arXiv:2312.02451•Dec 5, 2023•0 citations

Infinite neural network quantum states: entanglement and training dynamics

Di Luo, James Halverson

Abstract We study infinite limits of neural network quantum states ( ∞ -NNQS), which exhibit representation power through ensemble statistics, and also tractable gradient descent dynamics. Ensemble averages of entanglement entropies are expressed in terms of neural network correlators, and architectures that exhibit volume-law entanglement are presented. The analytic calculations of entanglement entropy bound are tractable because the ensemble statistics are simplified in the Gaussian process limit. A general framework is developed for studying the gradient descent dynamics of neural network quantum states (NNQS), using a quantum state neural tangent kernel (QS-NTK). For ∞ -NNQS the training dynamics is simplified, since the QS-NTK becomes deterministic and constant. An analytic solution is derived for quantum state supervised learning, which allows an ∞ -NNQS to recover any target wavefunction. Numerical experiments on finite and infinite NNQS in the transverse field Ising model and Fermi Hubbard model demonstrate excellent agreement with theory. ∞ -NNQS opens up new opportunities for studying entanglement and training dynamics in other physics applications, such as in finding ground states.

arXiv:2112.00723•Machine Learning Science and Technology•Jun 1, 2023•7 citations

Tangent-space methods for truncating uniform MPS

Bram Vanhecke, Maarten Van Damme, Jutho Haegeman +2 more

An essential primitive in quantum tensor network simulations is the problem of approximating a matrix product state with one of a smaller bond dimension. This problem forms the central bottleneck in algorithms for time evolution and for contracting projected entangled pair states. We formulate a tangent-space based variational algorithm to achieve this goal for uniform (infinite) matrix product states. The algorithm exhibits a favourable scaling of the computational cost, and we demonstrate its usefulness by several examples involving the multiplication of a matrix product state with a matrix product operator.

arXiv:2001.11882•SciPost Physics Core•Feb 19, 2021•2 citations

Tangent-space methods for truncating uniform MPS

Bram Vanhecke

A central primitive in quantum tensor network simulations is the problem of approximating a matrix product state with one of a lower bond dimension. This problem forms the central bottleneck in algorithms for time evolution and for contracting projected entangled pair states. We formulate a tangent-space based variational algorithm to achieve this for uniform (infinite) matrix product states. The algorithm exhibits a favourable scaling of the computational cost, and we demonstrate its usefulness by several examples involving the multiplication of a matrix product state with a matrix product operator.

•Feb 1, 2021•19 citations

Quantum-classical hybrid neural networks in the neural tangent kernel regime

Kouhei Nakaji, Hiroyuki Tezuka, Naoki Yamamoto

Abstract Recently, quantum neural networks or quantum–classical neural networks (qcNN) have been actively studied, as a possible alternative to the conventional classical neural network (cNN), but their practical and theoretically-guaranteed performance is still to be investigated. In contrast, cNNs and especially deep cNNs, have acquired several solid theoretical basis; one of those basis is the neural tangent kernel (NTK) theory, which can successfully explain the mechanism of various desirable properties of cNNs, particularly the global convergence in the training process. In this paper, we study a class of qcNN composed of a quantum data-encoder followed by a cNN. The quantum part is randomly initialized according to unitary 2-designs, which is an effective feature extraction process for quantum states, and the classical part is also randomly initialized according to Gaussian distributions; then, in the NTK regime where the number of nodes of the cNN becomes infinitely large, the output of the entire qcNN becomes a nonlinear function of the so-called projected quantum kernel. That is, the NTK theory is used to construct an effective quantum kernel, which is in general nontrivial to design. Moreover, NTK defined for the qcNN is identical to the covariance matrix of a Gaussian process, which allows us to analytically study the learning process. These properties are investigated in thorough numerical experiments; particularly, we demonstrate that the qcNN shows a clear advantage over fully classical NNs and qNNs for the problem of learning the quantum data-generating process.

arXiv:2109.03786•Quantum Science and Technology•Dec 7, 2023•12 citations

In this work, we study the Neural Tangent Kernel (NTK) of Matrix Product States (MPS) and the convergence of its NTK in the infinite bond dimensional limit. We prove that the NTK of MPS asymptotically converges to a constant matrix during the gradient descent (training) process (and also the initialization phase) as the bond dimensions of MPS go to infinity by the observation that the variation of the tensors in MPS asymptotically goes to zero during training in the infinite limit. By showing the positive-definiteness of the NTK of MPS, the convergence of MPS during the training in the function space (space of functions represented by MPS) is guaranteed without any extra assumptions of the data set. We then consider the settings of (supervised) Regression with Mean Square Error (RMSE) and (unsupervised) Born Machines (BM) and analyze their dynamics in the infinite bond dimensional limit. The ordinary differential equations (ODEs) which describe the dynamics of the responses of MPS in the RMSE and BM are derived and solved in the closed-form. For the Regression, we consider Mercer Kernels (Gaussian Kernels) and find that the evolution of the mean of the responses of MPS follows the largest eigenvalue of the NTK. Due to the orthogonality of the kernel functions in BM, the evolution of different modes (samples) decouples and the "characteristic time" of convergence in training is obtained.

A quantum neural network (QNN) is a parameterized mapping efficiently implementable on near-term Noisy Intermediate-Scale Quantum (NISQ) computers. It can be used for supervised learning when combined with classical gradient-based optimizers. Despite the existing empirical and theoretical investigations, the convergence of QNN training is not fully understood. Inspired by the success of the neural tangent kernels (NTKs) in probing into the dynamics of classical neural networks, a recent line of works proposes to study over-parameterized QNNs by examining a quantum version of tangent kernels. In this work, we study the dynamics of QNNs and show that contrary to popular belief it is qualitatively different from that of any kernel regression: due to the unitarity of quantum operations, there is a non-negligible deviation from the tangent kernel regression derived at the random initialization. As a result of the deviation, we prove the at-most sublinear convergence for QNNs with Pauli measurements, which is beyond the explanatory power of any kernel regression dynamics. We then present the actual dynamics of QNNs in the limit of over-parameterization. The new dynamics capture the change of convergence rate during training and implies that the range of measurements is crucial to the fast QNN convergence.

Quantum variational algorithms have been one of major applications of quantum computing with current quantum devices. There are recent attempts to establish the foundation for these algorithms. A possible approach is to characterize the training dynamics with quantum neural tangent kernel. In this work, we construct the kernel for two models, Quantun Ensemble and Quantum Neural Network, and show the convergence of these models in the limit of infinitely many qubits. We also show applications of the kernel limit in regression tasks.

Abstract We study infinite limits of neural network quantum states ( ∞ -NNQS), which exhibit representation power through ensemble statistics, and also tractable gradient descent dynamics. Ensemble averages of entanglement entropies are expressed in terms of neural network correlators, and architectures that exhibit volume-law entanglement are presented. The analytic calculations of entanglement entropy bound are tractable because the ensemble statistics are simplified in the Gaussian process limit. A general framework is developed for studying the gradient descent dynamics of neural network quantum states (NNQS), using a quantum state neural tangent kernel (QS-NTK). For ∞ -NNQS the training dynamics is simplified, since the QS-NTK becomes deterministic and constant. An analytic solution is derived for quantum state supervised learning, which allows an ∞ -NNQS to recover any target wavefunction. Numerical experiments on finite and infinite NNQS in the transverse field Ising model and Fermi Hubbard model demonstrate excellent agreement with theory. ∞ -NNQS opens up new opportunities for studying entanglement and training dynamics in other physics applications, such as in finding ground states.

An essential primitive in quantum tensor network simulations is the problem of approximating a matrix product state with one of a smaller bond dimension. This problem forms the central bottleneck in algorithms for time evolution and for contracting projected entangled pair states. We formulate a tangent-space based variational algorithm to achieve this goal for uniform (infinite) matrix product states. The algorithm exhibits a favourable scaling of the computational cost, and we demonstrate its usefulness by several examples involving the multiplication of a matrix product state with a matrix product operator.

A central primitive in quantum tensor network simulations is the problem of approximating a matrix product state with one of a lower bond dimension. This problem forms the central bottleneck in algorithms for time evolution and for contracting projected entangled pair states. We formulate a tangent-space based variational algorithm to achieve this for uniform (infinite) matrix product states. The algorithm exhibits a favourable scaling of the computational cost, and we demonstrate its usefulness by several examples involving the multiplication of a matrix product state with a matrix product operator.

Abstract Recently, quantum neural networks or quantum–classical neural networks (qcNN) have been actively studied, as a possible alternative to the conventional classical neural network (cNN), but their practical and theoretically-guaranteed performance is still to be investigated. In contrast, cNNs and especially deep cNNs, have acquired several solid theoretical basis; one of those basis is the neural tangent kernel (NTK) theory, which can successfully explain the mechanism of various desirable properties of cNNs, particularly the global convergence in the training process. In this paper, we study a class of qcNN composed of a quantum data-encoder followed by a cNN. The quantum part is randomly initialized according to unitary 2-designs, which is an effective feature extraction process for quantum states, and the classical part is also randomly initialized according to Gaussian distributions; then, in the NTK regime where the number of nodes of the cNN becomes infinitely large, the output of the entire qcNN becomes a nonlinear function of the so-called projected quantum kernel. That is, the NTK theory is used to construct an effective quantum kernel, which is in general nontrivial to design. Moreover, NTK defined for the qcNN is identical to the covariance matrix of a Gaussian process, which allows us to analytically study the learning process. These properties are investigated in thorough numerical experiments; particularly, we demonstrate that the qcNN shows a clear advantage over fully classical NNs and qNNs for the problem of learning the quantum data-generating process.