arXiv:2303.17452

Theory on variational high-dimensional tensor networks

Zidu Liu, Qi Ye, Li-Wei Yu, L. -M. Duan, Dong-Ling Deng

Published Mar 30, 2023• 0 citations

Abstract

Tensor network methods are powerful tools for studying quantum many-body systems. In this paper, we investigate the emergent statistical properties of random high-dimensional tensor-network states and the trainability of variational tensor networks. We utilize diagrammatic methods and map our problems to the calculations of different partition functions for high-dimensional Ising models with special structures. To address the notorious difficulty in cracking these models, we develop a combinatorial method based on solving the ``puzzle of polyominoes". With this method, we are able to rigorously study statistical properties of the high dimensional random tensor networks. We prove: (a) the entanglement entropy approaches the maximal volume law, except for a small probability that is bounded by an inverse polynomial of the bond dimension; (b) the typicality occurs for the expectation value of a local observable when the bond dimension increases. In addition, we investigate the barren plateaus (i.e., exponentially vanishing gradients) for the high-dimensional tensor network models. We prove that such variational models suffer from barren plateaus for global loss functions, rendering their training processes inefficient in general. Whereas, for local loss functions, we prove that the gradient is independent of the system size (thus no barren plateau occurs), but decays exponentially with the distance between the region where the local observable acts and the site that hosts the derivative parameter. Our results uncover in a rigorous fashion some fundamental properties for variational high-dimensional tensor networks, which paves a way for their future theoretical studies and practical applications.

DOI: 10.48550/arxiv.2303.17452

Related papers

Presence and Absence of Barren Plateaus in Tensor-Network Based Machine Learning

Zidu Liu, Li-Wei Yu, Luming Duan +1 more

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

arXiv:2108.08312Physical Review LettersDec 28, 202249 citations

No-Free-Lunch Theories for Tensor-Network Machine Learning Models

Wu, Jing-Chuan, Qi Ye, Dong-Ling Deng +1 more

Tensor network machine learning models have shown remarkable versatility in tackling complex data-driven tasks, ranging from quantum many-body problems to classical pattern recognitions. Despite their promising performance, a comprehensive understanding of the underlying assumptions and limitations of these models is still lacking. In this work, we focus on the rigorous formulation of their no-free-lunch theorem -- essential yet notoriously challenging to formalize for specific tensor network machine learning models. In particular, we rigorously analyze the generalization risks of learning target output functions from input data encoded in tensor network states. We first prove a no-free-lunch theorem for machine learning models based on matrix product states, i.e., the one-dimensional tensor network states. Furthermore, we circumvent the challenging issue of calculating the partition function for two-dimensional Ising model, and prove the no-free-lunch theorem for the case of two-dimensional projected entangled-pair state, by introducing the combinatorial method associated to the "puzzle of polyominoes". Our findings reveal the intrinsic limitations of tensor network-based learning models in a rigorous fashion, and open up an avenue for future analytical exploration of both the strengths and limitations of quantum-inspired machine learning frameworks.

arXiv:2412.05674Dec 7, 20240 citations

No-Free-Lunch Theorems for Tensor Network Machine Learning Models

Jing-Chuan Wu, Qi Ye, Dong-Ling Deng +1 more

Tensor network machine learning models have shown remarkable versatility in tackling complex data-driven tasks. Despite their promising performance, a comprehensive understanding of the underlying assumptions and limitations of these models is still lacking. Here we focus on the rigorous formulation of their no-free-lunch theorem, which is essential yet notoriously challenging to formalize for specific tensor network learning models. In particular, we rigorously analyze the generalization risks of learning target output functions from input data encoded in tensor network states. We first prove a no-free-lunch theorem for machine learning models based on matrix product states. Then we circumvent the challenging issue of calculating the partition function for a two-dimensional Ising model, and prove the no-free-lunch theorem for the case of two-dimensional projected entangled-pair states, by introducing the combinatorial method associated to the "puzzle of polyominoes." In addition, we rigorously establish an adversarial theorem that emerges naturally as a practical consequence of our no-free-lunch theorem. Our findings reveal the intrinsic limitations of tensor-network-based learning models in a rigorous fashion, and open up an avenue for future analytical exploration of both the strengths and limitations of quantum-inspired machine learning frameworks.

Physical Review LettersNov 12, 20250 citations

Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

Thomas Barthel, Qiang Miao

Abstract Vanishing gradients can pose substantial obstacles for high-dimensional optimization problems. Here we consider energy minimization problems for quantum many-body systems with extensive Hamiltonians and finite-range interactions, which can be studied on classical computers or in the form of variational quantum eigensolvers on quantum computers. Barren plateaus correspond to scenarios where the average amplitude of the energy gradient decreases exponentially with increasing system size. This occurs, for example, for quantum neural networks and for brickwall quantum circuits when the depth increases polynomially in the system size. Here we prove that the variational optimization problems for matrix product states, tree tensor networks, and the multiscale entanglement renormalization ansatz are free of barren plateaus. The derived scaling properties for the gradient variance provide an analytical guarantee for the trainability of randomly initialized tensor network states (TNS) and motivate certain initialization schemes. In a suitable representation, unitary tensors that parametrize the TNS are sampled according to the uniform Haar measure. We employ a Riemannian formulation of the gradient based optimizations which simplifies the analytical evaluation.

arXiv:2304.00161Communications in Mathematical PhysicsMar 19, 202511 citations

Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits

Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen +1 more

Over-parameterization is commonly used to increase the expressivity of variational quantum circuits (VQCs), yet deeper and more highly parameterized circuits often exhibit poor trainability and limited generalization. In this work, we provide a theoretical explanation for this phenomenon from a function-class perspective. We show that sufficiently expressive, unstructured variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size. As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions, a phenomenon we term simplicity bias, with barren plateaus arising as a consequence rather than the root cause. Using tools from random matrix theory and concentration of measure, we rigorously characterize this universality class and establish uniform hypothesis-class collapse over finite datasets. We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class. By restricting the accessible unitary ensemble through bounded tensor rank or bond dimension, these architectures prevent concentration of measure, preserve output variability for local observables, and retain non-degenerate gradient signals even in over-parameterized regimes. Together, our results unify barren plateaus, expressivity limits, and generalization collapse under a single structural mechanism rooted in random-matrix universality, highlighting the central role of architectural inductive bias in variational quantum algorithms.

quant-pharXiv:2601.01877Jan 6, 20260 citations

Barren plateaus in quantum tensor network optimization

Enrique Cervero-Martín, Kirill Plekhanov, Michael Lubasch

We analyze the barren plateau phenomenon in the variational optimization of quantum circuits inspired by matrix product states (qMPS), tree tensor networks (qTTN), and the multiscale entanglement renormalization ansatz (qMERA). We consider as the cost function the expectation value of a Hamiltonian that is a sum of local terms. For randomly chosen variational parameters we show that the variance of the cost function gradient decreases exponentially with the distance of a Hamiltonian term from the canonical centre in the quantum tensor network. Therefore, as a function of qubit count, for qMPS most gradient variances decrease exponentially and for qTTN as well as qMERA they decrease polynomially. We also show that the calculation of these gradients is exponentially more efficient on a classical computer than on a quantum computer.

arXiv:2209.00292QuantumApr 13, 202369 citations