Learning tensor networks with tensor cross interpolation: New algorithms and libraries
Y. Núñez-Fernández, Marc K. Ritter, Matthieu Jeannin, Jheng-Wei Li, Thomas Kloss, Thibaud Louvet, Satoshi Terasaki, Olivier Parcollet, Jan von Delft, Hiroshi Shinaoka, Xavier Waintal
The tensor cross interpolation (TCI) algorithm is a rank-revealing algorithm for decomposing low-rank, high-dimensional tensors into tensor trains/matrix product states (MPS). TCI learns a compact MPS representation of the entire object from a tiny training data set. Once obtained, the large existing MPS toolbox provides exponentially fast algorithms for performing a large set of operations. We discuss several improvements and variants of TCI. In particular, we show that replacing the cross interpolation by the partially rank-revealing LU decomposition yields a more stable and more flexible algorithm than the original algorithm. We also present two open source libraries, xfac in Python/C++ and TensorCrossInterpolation.jl in Julia, that implement these improved algorithms, and illustrate them on several applications. These include sign-problem-free integration in large dimension, the “superhigh-resolution” quantics representation of functions, the solution of partial differential equations, the superfast Fourier transform, the computation of partition functions, and the construction of matrix product operators.
The tensor cross interpolation (TCI) algorithm is a rank-revealing algorithm for decomposing low-rank, high-dimensional tensors into tensor trains/matrix product states (MPS). TCI learns a compact MPS representation of the entire object from a tiny training data set. Once obtained, the large existing MPS toolbox provides exponentially fast algorithms for performing a large set of operations. We discuss several improvements and variants of TCI. In particular, we show that replacing the cross interpolation by the partially rank-revealing LU decomposition yields a more stable and more flexible algorithm than the original algorithm. We also present two open source libraries, xfac in Python/C++ and TensorCrossInterpolation.jl in Julia, that implement these improved algorithms, and illustrate them on several applications. These include sign-problem-free integration in large dimension, the “superhigh-resolution” quantics representation of functions, the solution of partial differential equations, the superfast Fourier transform, the computation of partition functions, and the construction of matrix product operators.
Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as O(χ4), where χ is the TT rank. If the rank of the output is smaller than χ2, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in O(χ3), while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.
This is a set of lectures on tensor networks with a strong emphasis on the core algorithms involving Matrix Product States (MPS) and Matrix Product Operators (MPO). Compared to other presentations, particular care has been given to disentangle aspects of tensor networks from the quantum many-body problem: MPO/MPS algorithms are presented as a way to deal with linear algebra on extremely (exponentially) large matrices and vectors, regardless of any particular application. The lectures include well-known algorithms to find eigenvectors of MPOs (the celebrated DMRG), solve linear problems, and recent learning algorithms that allow one to map a known function into an MPS (the Tensor Cross Interpolation, or TCI, algorithm). The lectures end with a discussion of how to represent functions and perform calculus with tensor networks using the "quantics" representation. They include the detailed analytical construction of important MPOs such as those for differentiation, indefinite integration, convolution, and the quantum Fourier transform. Three concrete applications are discussed in detail: the simulation of a quantum computer (either exactly or with compression), the simulation of a quantum annealer, and techniques to solve partial differential equations (e.g. Poisson, diffusion, or Gross-Pitaevskii) within the "quantics" representation. The lectures have been designed to be accessible to a first-year PhD student and include detailed proofs of all statements.
A defining feature of quantum many-body systems is the exponential scaling of the Hilbert space with the number of degrees of freedom. This exponential complexity naïvely renders a complete state characterization, for instance via the complete set of bipartite Renyi entropies for all disjoint regions, a challenging task. Recently, a compact way of storing subregions' purities by encoding them as amplitudes of a fictitious quantum wave function, known as entanglement feature, was proposed. Notably, the entanglement feature can be a simple object even for highly entangled quantum states. However the complexity and practical usage of the entanglement feature for general quantum states has not been explored. In this work, we demonstrate that the entanglement feature can be efficiently learned using only a polynomial amount of samples in the number of degrees of freedom through the so-called tensor cross interpolation (TCI) algorithm, assuming it is expressible as a finite bond dimension MPS. We benchmark this learning process on Haar and random MPS states, confirming analytic expectations. Applying the TCI algorithm to quantum eigenstates of various one dimensional quantum systems, we identify cases where eigenstates have entanglement feature learnable with TCI. We conclude with possible applications of the learned entanglement feature, such as quantifying the distance between different entanglement patterns and finding the optimal one-dimensional ordering of physical indices in a given state, highlighting the potential utility of the proposed purity interpolation method.
We apply the tensor cross interpolation (TCI) algorithm to solve equilibrium quantum impurity problems with high precision based on the weak-coupling expansion. The TCI algorithm, a kind of active learning method, factorizes high-dimensional integrals that appear in the perturbative expansion into a product of low-dimensional ones, enabling us to evaluate higher-order terms efficiently. This method is free from the sign problem which quantum Monte Carlo methods sometimes suffer from, and allows one to directly calculate the free energy. We benchmark the TCI impurity solver on an exactly solvable impurity model and find good agreement with the exact solutions. We also incorporate the TCI impurity solver into the dynamical mean-field theory to solve the Hubbard model and show that the metal-to-Mott insulator transition is correctly described with comparable accuracy to the Monte Carlo methods. Behind the effectiveness of the TCI approach for quantum impurity problems lies the fact that the integrands in the weak-coupling expansion naturally have a low-rank structure in the tensor-train representation.
Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)
Tensor cross interpolation (TCI) is a powerful technique for learning a tensor train (TT) by adaptively sampling a target tensor based on an interpolation formula. However, when the tensor evaluations contain random noise, optimizing the TT is more advantageous than interpolating the noise. Here, we propose a new method that starts with an initial guess of TT and optimizes it using non-linear least-squares by fitting it to measured points obtained from TCI. We use quantics TCI (QTCI) in this method and demonstrate its effectiveness on sine and two-time correlation functions, with each evaluated with random noise. The resulting QTT exhibits increased robustness against noise compared to the QTCI method. Furthermore, we employ this optimized QTT of the correlation function in quantum simulation based on pseudo-imaginary-time evolution, resulting in ground-state energy with higher accuracy than the QTCI or Monte Carlo methods.