Tensor networks are a compressed format for multi-dimensional data. One dimensional tensor networks—often referred to as tensor trains (TT) or matrix product states (MPS)—are increasingly being used as a numerical Ansatz for continuum functions by ``quantizing’’ the inputs into discrete binary digits. Here we demonstrate the power of more general tree tensor networks (TTNs) for this purpose. We provide direct constructions of a number of elementary functions as generic tree tensor networks and interpolative constructions for more complicated functions via a generalization of the tensor cross interpolation algorithm. For a range of multi-dimensional functions we show how more structured tree tensor networks offer a significantly more efficient Ansatz than the commonly used tensor train. Finally, we demonstrate how the methods introduced in this work can be used to realize a TTN-based solver for multi-dimensional, non-linear Fredholm equations.
Tensor networks are a compressed format for multi-dimensional data. One dimensional tensor networks—often referred to as tensor trains (TT) or matrix product states (MPS)—are increasingly being used as a numerical Ansatz for continuum functions by ``quantizing’’ the inputs into discrete binary digits. Here we demonstrate the power of more general tree tensor networks (TTNs) for this purpose. We provide direct constructions of a number of elementary functions as generic tree tensor networks and interpolative constructions for more complicated functions via a generalization of the tensor cross interpolation algorithm. For a range of multi-dimensional functions we show how more structured tree tensor networks offer a significantly more efficient Ansatz than the commonly used tensor train. Finally, we demonstrate how the methods introduced in this work can be used to realize a TTN-based solver for multi-dimensional, non-linear Fredholm equations.
Matrix product states (MPSs), a tensor network designed for one-dimensional quantum systems, were recently proposed for generative modeling of natural data (such as images) in terms of the ``Born machine.'' However, the exponential decay of correlation in MPSs restricts its representation power heavily for modeling complex data such as natural images. In this work, we push forward the effort of applying tensor networks to machine learning by employing the tree tensor network (TTN), which exhibits balanced performance in expressibility and efficient training and sampling. We design the tree tensor network to utilize the two-dimensional prior of the natural images and develop sweeping learning and sampling algorithms which can be efficiently implemented utilizing graphical processing units. We apply our model to random binary patterns and the binary MNIST data sets of handwritten digits. We show that the TTN is superior to MPSs for generative modeling in keeping the correlation of pixels in natural images, as well as giving better log-likelihood scores in standard data sets of handwritten digits. We also compare its performance with state-of-the-art generative models such as variational autoencoders, restricted Boltzmann machines, and PixelCNN. Finally, we discuss the future development of tensor network states in machine learning problems.
In this paper we review basic and emerging models and associated algorithms for large-scale tensor networks, especially Tensor Train (TT) decompositions using novel mathematical and graphical representations. We discus the concept of tensorization (i.e., creating very high-order tensors from lower-order original data) and super compression of data achieved via quantized tensor train (QTT) networks. The purpose of a tensorization and quantization is to achieve, via low-rank tensor approximations "super" compression, and meaningful, compact representation of structured data. The main objective of this paper is to show how tensor networks can be used to solve a wide class of big data optimization problems (that are far from tractable by classical numerical methods) by applying tensorization and performing all operations using relatively small size matrices and tensors and applying iteratively optimized and approximative tensor contractions. Keywords: Tensor networks, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, tensorization, distributed representation od data optimization problems for very large-scale problems: generalized eigenvalue decomposition (GEVD), PCA/SVD, canonical correlation analysis (CCA).
Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)
We have developed TTNOpt, a software package that utilizes tree tensor networks (TTNs) for quantum spin systems and high-dimensional data analysis. TTNOpt provides efficient and powerful TTN computations by locally optimizing the network structure, guided by the entanglement pattern of the target tensors. For quantum spin systems, TTNOpt searches for the ground state of Hamiltonians with bilinear spin interactions and magnetic fields, and computes physical properties of these states, including the variational energy, bipartite entanglement entropy (EE), single-site expectation values, and two-site correlation functions. Additionally, TTNOpt can target the lowest-energy state within a specified subspace, provided that the Hamiltonian conserves total magnetization. For high-dimensional data analysis, TTNOpt factorizes complex tensors into TTN states that maximize fidelity to the original tensors by optimizing the tensors and the network. When a TTN is provided as input, TTNOpt reconstructs the network based on the EE without referencing the fidelity of the original state. We present three demonstrations of TTNOpt:(1) Ground-state search for the hierarchical chain model with a system size of 256. The entanglement patterns of the ground state manifest themselves in a tree structure, and TTNOpt successfully identifies the tree.(2) Factorization of a quantic tensor of the 224 dimensions representing a three-variable function where each variant has a weak bit-wise correlation. The optimized TTN shows that its structure isolates the variables from each other.(3) Reconstruction of the matrix product network representing a 16-variable normal distribution characterized by a tree-like correlation structure. TTNOpt can reveal hidden correlation structures of the covariance matrix.
Quantics tensor trains are attracting intense interest for quantum-inspired computing and quantum state preparation. These methods, which approximate continuum functions by representing their amplitude encoding as a matrix product state (MPS), are exceedingly powerful for univariate functions but rapidly become challenging when handling multivariate functions, since the linear chain topology leads to a large distance between highly-entangled qubits. We overcome this limitation by introducing SCENT (Spectral Clustering for Entanglement miNimizing Trees). SCENT is a protocol that utilizes efficiently-computable pairwise entanglement metrics to determine a suitable tree tensor network (TTN) structure, which can then be efficiently approximated using tensor cross-interpolation (TCI); we find that it substantially outperforms MPS methods and improves upon previous TTN methods. We then apply these optimized TTNs to state preparation, introducing an approximate circuit compilation method based on environment-tensor methods without significantly conceding overall accuracy. Importantly, the inherent gauge freedom of TTNs can be directly exploited in this method, resulting in higher fidelity at a given circuit depth. We demonstrate this quantum state-preparation pipeline on archetypal state-preparation problems in quantum chemistry and financial portfolio optimization. In our flagship demonstration, we encode a 20-variable probability distribution with long-ranged, non-nearest neighbor inter-variable correlations in a 200-qubit state-preparation circuit with infidelity 7.44×10−9 using only 43284 CNOTs; depth and fidelity can be traded, allowing the same distribution to be prepared to infidelity 10−3 with as few as 5584 CNOTs.
We study the tree-tensor-network-state (TTNS) method with variable tensor orders for quantum chemistry. TTNS is a variational method to efficiently approximate complete active space (CAS) configuration interaction (CI) wave functions in a tensor product form. TTNS can be considered as a higher order generalization of the matrix product state (MPS) method. The MPS wave function is formulated as products of matrices in a multiparticle basis spanning a truncated Hilbert space of the original CAS-CI problem. These matrices belong to active orbitals organized in a one-dimensional array, while tensors in TTNS are defined upon a tree-like arrangement of the same orbitals. The tree-structure is advantageous since the distance between two arbitrary orbitals in the tree scales only logarithmically with the number of orbitals N, whereas the scaling is linear in the MPS array. It is found to be beneficial from the computational costs point of view to keep strongly correlated orbitals in close vicinity in both arrangements; therefore, the TTNS ansatz is better suited for multireference problems with numerous highly correlated orbitals. To exploit the advantages of TTNS a novel algorithm is designed to optimize the tree tensor network topology based on quantum information theory and entanglement. The superior performance of the TTNS method is illustrated on the ionic-neutral avoided crossing of LiF. It is also shown that the avoided crossing of LiF can be localized using only ground state properties, namely one-orbital entanglement.