Physical Review Letters

No-Free-Lunch Theorems for Tensor Network Machine Learning Models

Jing-Chuan Wu, Qi Ye, Dong-Ling Deng, Li-Wei Yu

Published Nov 12, 2025• 0 citations

Abstract

Tensor network machine learning models have shown remarkable versatility in tackling complex data-driven tasks. Despite their promising performance, a comprehensive understanding of the underlying assumptions and limitations of these models is still lacking. Here we focus on the rigorous formulation of their no-free-lunch theorem, which is essential yet notoriously challenging to formalize for specific tensor network learning models. In particular, we rigorously analyze the generalization risks of learning target output functions from input data encoded in tensor network states. We first prove a no-free-lunch theorem for machine learning models based on matrix product states. Then we circumvent the challenging issue of calculating the partition function for a two-dimensional Ising model, and prove the no-free-lunch theorem for the case of two-dimensional projected entangled-pair states, by introducing the combinatorial method associated to the "puzzle of polyominoes." In addition, we rigorously establish an adversarial theorem that emerges naturally as a practical consequence of our no-free-lunch theorem. Our findings reveal the intrinsic limitations of tensor-network-based learning models in a rigorous fashion, and open up an avenue for future analytical exploration of both the strengths and limitations of quantum-inspired machine learning frameworks.

DOI: 10.1103/by9g-5bkm

Related papers

No-Free-Lunch Theories for Tensor-Network Machine Learning Models

Wu, Jing-Chuan, Qi Ye, Dong-Ling Deng +1 more

Tensor network machine learning models have shown remarkable versatility in tackling complex data-driven tasks, ranging from quantum many-body problems to classical pattern recognitions. Despite their promising performance, a comprehensive understanding of the underlying assumptions and limitations of these models is still lacking. In this work, we focus on the rigorous formulation of their no-free-lunch theorem -- essential yet notoriously challenging to formalize for specific tensor network machine learning models. In particular, we rigorously analyze the generalization risks of learning target output functions from input data encoded in tensor network states. We first prove a no-free-lunch theorem for machine learning models based on matrix product states, i.e., the one-dimensional tensor network states. Furthermore, we circumvent the challenging issue of calculating the partition function for two-dimensional Ising model, and prove the no-free-lunch theorem for the case of two-dimensional projected entangled-pair state, by introducing the combinatorial method associated to the "puzzle of polyominoes". Our findings reveal the intrinsic limitations of tensor network-based learning models in a rigorous fashion, and open up an avenue for future analytical exploration of both the strengths and limitations of quantum-inspired machine learning frameworks.

arXiv:2412.05674Dec 7, 20240 citations

Theory on variational high-dimensional tensor networks

Zidu Liu, Qi Ye, Li-Wei Yu +2 more

Tensor network methods are powerful tools for studying quantum many-body systems. In this paper, we investigate the emergent statistical properties of random high-dimensional tensor-network states and the trainability of variational tensor networks. We utilize diagrammatic methods and map our problems to the calculations of different partition functions for high-dimensional Ising models with special structures. To address the notorious difficulty in cracking these models, we develop a combinatorial method based on solving the ``puzzle of polyominoes". With this method, we are able to rigorously study statistical properties of the high dimensional random tensor networks. We prove: (a) the entanglement entropy approaches the maximal volume law, except for a small probability that is bounded by an inverse polynomial of the bond dimension; (b) the typicality occurs for the expectation value of a local observable when the bond dimension increases. In addition, we investigate the barren plateaus (i.e., exponentially vanishing gradients) for the high-dimensional tensor network models. We prove that such variational models suffer from barren plateaus for global loss functions, rendering their training processes inefficient in general. Whereas, for local loss functions, we prove that the gradient is independent of the system size (thus no barren plateau occurs), but decays exponentially with the distance between the region where the local observable acts and the site that hosts the derivative parameter. Our results uncover in a rigorous fashion some fundamental properties for variational high-dimensional tensor networks, which paves a way for their future theoretical studies and practical applications.

arXiv:2303.17452Mar 30, 20230 citations

Presence and Absence of Barren Plateaus in Tensor-Network Based Machine Learning

Zidu Liu, Li-Wei Yu, Luming Duan +1 more

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

arXiv:2108.08312Physical Review LettersDec 28, 202249 citations

Generative learning of continuous data by tensor networks

Alex Meiburg, Jing Chen, Jacob Miller +3 more

Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervised generative learning. While possessing many desirable features arising from their quantum-inspired nature, tensor network generative models have previously been largely restricted to binary or categorical data, limiting their utility in real-world modeling problems. We overcome this by introducing a new family of tensor network generative models for continuous data, which are capable of learning from distributions containing continuous random variables. We develop our method in the setting of matrix product states, first deriving a universal expressivity theorem proving the ability of this model family to approximate any reasonably smooth probability density function with arbitrary precision. We then benchmark the performance of this model on several synthetic and real-world datasets, finding that the model learns and generalizes well on distributions of continuous and discrete variables. We develop methods for modeling different data domains, and introduce a trainable compression layer which is found to increase model performance given limited memory or computational resources. Overall, our methods give important theoretical and empirical evidence of the efficacy of quantum-inspired methods for the rapidly growing field of generative learning.

arXiv:2310.20498SciPost PhysicsMar 17, 20254 citations

No Free Lunch for Quantum Machine Learning

Kyle Poland, Kerstin Beer, Tobias J. Osborne

The ultimate limits for the quantum machine learning of quantum data are investigated by obtaining a generalisation of the celebrated No Free Lunch (NFL) theorem. We find a lower bound on the quantum risk (the probability that a trained hypothesis is incorrect when presented with a random input) of a quantum learning algorithm trained via pairs of input and output states when averaged over training pairs and unitaries. The bound is illustrated using a recently introduced QNN architecture.

arXiv:2003.14103Mar 31, 202026 citations

Assessing the advantages and limitations of quantum neural networks in regression tasks

Gubio G. de Lima, Tiago de Souza Farias, Alexandre C. Ricardo +1 more

The development of quantum neural networks (QNNs) has attracted considerable attention due to their potential to surpass classical models in certain machine learning tasks. Nonetheless, it remains unclear under which conditions QNNs provide concrete benefits over classical neural networks (CNNs). This study addresses this question by performing both qualitative and quantitative analyses of classical and quantum models applied to regression problems, using two target functions with contrasting properties. Additionally, the work explores the methodological difficulties inherent in making fair comparisons between QNNs and CNNs. The findings reveal a distinct advantage of QNNs in a specific quantum machine learning context. In particular, QNNs excelled at approximating the sinusoidal function, achieving errors up to seven orders of magnitude lower than their classical counterparts. However, their performance was limited in other cases, emphasizing that QNNs are highly effective for certain tasks but not universally superior. These results reinforce the principles of the “No Free Lunch” theorem, highlighting that no single model outperforms all others across every problem domain.

Quantum Machine IntelligenceMar 3, 20260 citations