Abstract The interpretation of representations and generalization powers has been a long-standing challenge in the fields of machine learning (ML) and artificial intelligence. This study contributes to understanding the emergence of universal scaling laws in quantum-probabilistic ML. We consider the generative tensor network (GTN) in the form of a matrix-product state as an example and show that with an untrained GTN (such as a random TN state), the negative logarithmic likelihood (NLL) L generally increases linearly with the number of features M, that is, L ≃ kM + const. This is a consequence of the so-called “catastrophe of orthogonality,” which states that quantum many-body states tend to become exponentially orthogonal to each other as M increases. This study reveals that, while gaining information through training, the linear-scaling law is suppressed by a negative quadratic correction, leading to L ≃ βM – αM 2 + const. The scaling coefficients exhibit logarithmic relationships with the number of training samples and quantum channels χ. The emergence of a quadratic correction term in the NLL for the testing (training) set can be regarded as evidence of the generalization (representation) power of the GTN. Over-parameterization can be identified by the deviation in the values of α between the training and testing sets while increasing χ. We further investigate how orthogonality in the quantum-feature map relates to the satisfaction of quantum-probabilistic interpretation and the representation and generalization powers of the GTN. Unveiling universal scaling laws in quantum-probabilistic ML would be a valuable step toward establishing a white-box ML scheme interpreted within the quantum-probabilistic framework.
Abstract The interpretation of representations and generalization powers has been a long-standing challenge in the fields of machine learning (ML) and artificial intelligence. This study contributes to understanding the emergence of universal scaling laws in quantum-probabilistic ML. We consider the generative tensor network (GTN) in the form of a matrix-product state as an example and show that with an untrained GTN (such as a random TN state), the negative logarithmic likelihood (NLL) L generally increases linearly with the number of features M, that is, L ≃ kM + const. This is a consequence of the so-called “catastrophe of orthogonality,” which states that quantum many-body states tend to become exponentially orthogonal to each other as M increases. This study reveals that, while gaining information through training, the linear-scaling law is suppressed by a negative quadratic correction, leading to L ≃ βM – αM 2 + const. The scaling coefficients exhibit logarithmic relationships with the number of training samples and quantum channels χ. The emergence of a quadratic correction term in the NLL for the testing (training) set can be regarded as evidence of the generalization (representation) power of the GTN. Over-parameterization can be identified by the deviation in the values of α between the training and testing sets while increasing χ. We further investigate how orthogonality in the quantum-feature map relates to the satisfaction of quantum-probabilistic interpretation and the representation and generalization powers of the GTN. Unveiling universal scaling laws in quantum-probabilistic ML would be a valuable step toward establishing a white-box ML scheme interpreted within the quantum-probabilistic framework.
The “black-box” nature of deep learning models fundamentally limits their reliability in critical applications. Tensor networks (TNs), drawing on their interpretable, quantum-probabilistic foundations, offer a promising pathway toward more transparent machine learning (ML). However, a systematic understanding of their capabilities and limitations remains an open challenge. Here, we establish a fundamental connection between a core property of quantum systems, namely, the catastrophe of orthogonality (COO), and the representational power and generalization performance of quantum-probabilistic TN models. Taking the Fashion Modified National Institute of Standards and Technology dataset, we propose 2 multiqubit data encoding schemes and demonstrate that key hyperparameters, such as the number of qubits, feature count, and mapping angle, modulate the onset of COO. This control mechanism directly dictates the model’s capacity, revealing that over-fitting can be interpreted as a specific regime of the quantum state space. Our work provides a theoretical lens, grounded in quantum many-body physics, for interpreting ML models. It positions TNs not merely as efficient computational tools but as a foundational framework for building inherently interpretable and reliable ML models.
Gaussian-process limits are powerful in describing overparameterized machine learning models, yet their validity in structured tensor-network architectures remains unclear. Here we analytically present a moment-based approach that identifies precise conditions for the emergence and breakdown of Gaussian universality in tensor-network learning models, with a focus on matrix product states. We prove that in the large bond dimension limit, the learning models with both local and global observables converge to Gaussian processes, with explicit finite-size bounds on higher-order moment deviations. Whereas in the large physical dimension limit, the Gaussian universality no longer persists: while the models with local observables retain Gaussian-process behavior, those global cases exhibit persistent non-Gaussian corrections. Our results reveal that Gaussian-process behavior in tensor-network learning is controlled not only by parameter number, but also by architectural scaling, observable locality, and the spectral properties.
It is a critical challenge to simultaneously achieve high interpretability and high efficiency with the current schemes of deep machine learning (ML). The tensor network (TN), a well-established mathematical tool originating from quantum mechanics, has shown its unique advantages in developing efficient “white-box” ML schemes. Here, we provide a brief review of the inspiring progress in TN-based ML. On the one hand, the interpretability of TN ML can be accommodated by a solid theoretical foundation based on quantum information and many-body physics. On the other hand, high efficiency can be obtained from powerful TN representations and the advanced computational techniques developed in quantum many-body physics. Keeping pace with the rapid development of quantum computers, TNs are expected to produce novel schemes runnable on quantum hardware in the direction of “quantum artificial intelligence” in the near future.
Tensor networks have emerged as promising tools for machine learning, inspired by their widespread use as variational ansatze in quantum many-body physics. It is well known that the success of a given tensor network ansatz depends in part on how well it can reproduce the underlying entanglement structure of the target state, with different network designs favoring different scaling patterns. We demonstrate here how a related correlation analysis can be applied to tensor network machine learning, and explore whether classical data possess correlation scaling patterns similar to those found in quantum states which might indicate the best network to use for a given dataset. We utilize mutual information as measure of correlations in classical data, and show that it can serve as a lower-bound on the entanglement needed for a probabilistic tensor network classifier. We then develop a logistic regression algorithm to estimate the mutual information between bipartitions of data features, and verify its accuracy on a set of Gaussian distributions designed to mimic different correlation patterns. Using this algorithm, we characterize the scaling patterns in the MNIST and Tiny Images datasets, and find clear evidence of boundary-law scaling in the latter. This quantum-inspired classical analysis offers insight into the design of tensor networks which are best suited for specific learning tasks.
Tensor-network techniques have recently proven useful in machine learning, both as a tool for the formulation of new learning algorithms and for enhancing the mathematical understanding of existing methods.Inspired by these developments, and the natural correspondence between tensor networks and probabilistic graphical models, we provide a rigorous analysis of the expressive power of various tensornetwork factorizations of discrete multivariate probability distributions.These factorizations include non-negative tensor-trains/MPS, which are in correspondence with hidden Markov models, and Born machines, which are naturally related to the probabilistic interpretation of quantum circuits.When used to model probability distributions, they exhibit tractable likelihoods and admit efficient learning algorithms.Interestingly, we prove that there exist probability distributions for which there are unbounded separations between the resource requirements of some of these tensor-network factorizations.Of particular interest, using complex instead of real tensors can lead to an arbitrarily large reduction in the number of parameters of the network.Additionally, we introduce locally purified states (LPS), a new factorization inspired by techniques for the simulation of quantum systems, with provably better expressive power than all other representations considered.The ramifications of this result are explored through numerical experiments.
Abstract Machine learning is a promising application of quantum computing, but challenges remain for implementation today because near-term devices have a limited number of physical qubits and high error rates. Motivated by the usefulness of tensor networks for machine learning in the classical context, we propose quantum computing approaches to both discriminative and generative learning, with circuits based on tree and matrix product state tensor networks, that could already have benefits with such near-term devices. The result is a unified framework in which classical and quantum computing can benefit from the same theoretical and algorithmic developments, and the same model can be trained classically then transferred to the quantum setting for additional optimization. Tensor network circuits can also provide qubit-efficient schemes in which, depending on the architecture, the number of physical qubits required scales only logarithmically with, or independently of the input or output data sizes. We demonstrate our proposals with numerical experiments, training a discriminative model to perform handwriting recognition using a hybrid quantum-classical optimization procedure that could be carried out on quantum hardware today, and testing the noise resilience of the trained model.