Tensor networks developed in the context of condensed matter physics try to approximate order- N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b=4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.
Tensor networks developed in the context of condensed matter physics try to approximate order- N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b=4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.
Tensor networks are used to efficiently approximate states of strongly-correlated quantum many-body systems. More generally, tensor network approximations may allow to reduce the costs for operating on an order-N tensor from exponential to polynomial in N, and this has become a popular approach for machine learning. We introduce the idea of imposing low-rank constraints on the tensors that compose the tensor network. With this modification, the time and space complexities for the network optimization can be substantially reduced while maintaining high accuracy. We detail this idea for tree tensor network states (TTNS) and projected entangled-pair states. Simulations of spin models on Cayley trees with low-rank TTNS exemplify the effect of rank constraints on the expressive power. We find that choosing the tensor rank r to be on the order of the bond dimension m, is sufficient to obtain high-accuracy groundstate approximations and to substantially outperform standard TTNS computations. Thus low-rank tensor networks are a promising route for the simulation of quantum matter and machine learning on large data sets.
We describe a quantum-assisted machine learning method in which multivariate data are encoded into quantum states in a Hilbert space whose dimension is exponentially large in the length of the data vector. Learning in this space occurs through applying a low-depth quantum circuit with a tree-tensor-network (TTN) topology acting as an unsupervised feature extractor to identify the most relevant quantum states in a data-driven fashion and then applying a supervised linear classifier encoding the class decision in a small-dimensional quantum register. We present tools for making TTN classifiers amenable to implementation on gate-based quantum computing devices, including an embedding map with accuracy similar to the recently defined exponential machines (A. Novikov et al., arXiv:1605.03795) but which produces valid quantum state embeddings of classical data vectors, and the use of manifold-based gradient optimization schemes to produce isometric operations mapping quantum states to a register of qubits defining a class decision. We detail methods for efficiently obtaining one-point and two-point correlation functions of the vectors defining the decision boundary of the quantum model, which can be used for model interpretability, as well as methods for obtaining classification decisions from partial data vectors. Further, we show that the use of isometric tensors can significantly aid in the human interpretability of the correlation functions extracted from the decision weights and may produce models that are less susceptible to adversarial perturbations. Finally, we discuss in detail the problem of compiling classically optimized isometric TTN models into unitary operations to be run on quantum computers and how isometric models requiring postselection on quantum hardware can be used to precondition variational Ans\"atze for models without postselection. We demonstrate our methodologies in applications utilizing the MNIST database of handwritten digits and a multivariate time-series data set of human activity recognition.
Tensor network contractions are widely used in statistical physics, quantum computing, and computer science. We introduce a method to efficiently approximate tensor network contractions using low-rank approximations, where each intermediate tensor generated during the contractions is approximated as a low-rank binary tree tensor network. The proposed algorithm has the flexibility to incorporate a large portion of the environment when performing low-rank approximations, which can lead to high accuracy for a given rank. Here, the environment refers to the remaining set of tensors in the network, and low-rank approximations with larger environments can generally provide higher accuracy. For contracting tensor networks defined on lattices, the proposed algorithm can be viewed as a generalization of the standard boundary-based algorithms. In addition, the algorithm includes a cost-efficient density matrix algorithm for approximating a tensor network with a general graph structure into a tree structure, whose computational cost is asymptotically upper-bounded by that of the standard algorithm that uses canonicalization. Experimental results indicate that the proposed technique outperforms previously proposed approximate tensor network contraction algorithms for multiple problems in terms of both accuracy and efficiency.
We introduce the Tree Tensor Network-enhanced Quantum-Classical Neural Network (TTN-QNet), a hybrid architecture that leverages the hierarchical structure of Tree Tensor Networks for efficient parameter representation and Variational Quantum Circuits (VQC) for expressive modeling. Unlike Tensor Ring Networks, TTNs reduce parameter redundancy through a tree-based topology, enabling scalable and interpretable computation. The proposed TTN-QNet is evaluated on the Iris, MNIST, and CIFAR-10 datasets, achieving classification accuracies of 93.2%, 85.24%, and 81.67%, respectively, on binary classification tasks. TTN-QNet demonstrates rapid convergence and robustness against barren plateaus, offering a promising direction for deep quantum learning.
Tensor network methods have been a key ingredient of advances in condensed matter physics and have recently sparked interest in the machine learning community for their ability to compactly represent very high-dimensional objects. Tensor network methods can for example be used to efficiently learn linear models in exponentially large feature spaces [Stoudenmire and Schwab, 2016]. In this work, we derive upper and lower bounds on the VC dimension and pseudo-dimension of a large class of tensor network models for classification, regression and completion. Our upper bounds hold for linear models parameterized by arbitrary tensor network structures, and we derive lower bounds for common tensor decomposition models~(CP, Tensor Train, Tensor Ring and Tucker) showing the tightness of our general upper bound. These results are used to derive a generalization bound which can be applied to classification with low rank matrices as well as linear classifiers based on any of the commonly used tensor decomposition models. As a corollary of our results, we obtain a bound on the VC dimension of the matrix product state classifier introduced in [Stoudenmire and Schwab, 2016] as a function of the so-called bond dimension~(i.e. tensor train rank), which answers an open problem listed by Cirac, Garre-Rubio and Pérez-García in [Cirac et al., 2019].
Tensor network states (TNSs) offer an efficient representation for the ground state of quantum many-body systems and play an important role in the simulations of them. Numerous TNSs have been proposed in the past few decades. However, due to the high cost of TNS for two-dimensional systems, a balance between the encoded entanglement and computational complexity of TNS is yet to be reached. In this work we introduce a new tree tensor network-based (TTN-based) TNS dubbed a ``fully augmented tree tensor network (FATTN) by releasing the constraint in augmented tree tensor network (ATTN). When disentanglers are augmented in the physical layer of TTN, FATTN can provide more entanglement than TTN and ATTN. At the same time, FATTN maintains the scaling of computational cost with bond dimension in TTN and ATTN. Benchmark results on the ground-state energy for the transverse Ising model are provided to demonstrate the improvement of accuracy of FATTN over TTN and ATTN. Moreover, FATTN is quite flexible, which can be constructed as an interpolation between TTN and the multiscale entanglement renormalization ansatz to reach a balance between the encoded entanglement and the computational cost.