We present an alternative approach to decompose non-negative tensors, called many-body approximation. Traditional decomposition methods assume low-rankness in the representation, resulting in difficulties in global optimization and target rank selection. We avoid these problems by energy-based modeling of tensors, where a tensor and its mode correspond to a probability distribution and a random variable, respectively. Our model can be globally optimized in terms of the KL divergence minimization by taking the interaction between variables (that is, modes), into account that can be tuned more intuitively than ranks. Furthermore, we visualize interactions between modes as tensor networks and reveal a nontrivial relationship between many-body approximation and low-rank approximation. We demonstrate the effectiveness of our approach in tensor completion and approximation.
We present an alternative approach to decompose non-negative tensors, called many-body approximation. Traditional decomposition methods assume low-rankness in the representation, resulting in difficulties in global optimization and target rank selection. We avoid these problems by energy-based modeling of tensors, where a tensor and its mode correspond to a probability distribution and a random variable, respectively. Our model can be globally optimized in terms of the KL divergence minimization by taking the interaction between variables (that is, modes), into account that can be tuned more intuitively than ranks. Furthermore, we visualize interactions between modes as tensor networks and reveal a nontrivial relationship between many-body approximation and low-rank approximation. We demonstrate the effectiveness of our approach in tensor completion and approximation.
It is well known that tensor decompositions show separations, that is, constraints on local terms (such as positivity) may entail an arbitrarily high cost in their representation. Here, we show that many of these separations disappear in the approximate case. Specifically, for every approximation error ɛ and norm, we define the approximate rank as the minimum rank of an element in the ɛ-ball with respect to that norm. For positive semidefinite matrices, we show that the separations between rank, purification rank, and separable rank disappear for a large class of Schatten p-norms. For non-negative tensors, we show that the separations between rank, positive semidefinite rank, and non-negative rank disappear for all ℓp-norms with p > 1. For the trace norm (p = 1), we obtain upper bounds that depend on the ambient dimension. We also provide a deterministic algorithm to obtain the approximate decomposition attaining our bounds. Our main tool is an approximate version of the Carathéodory theorem. Our results imply that many separations are not robust under small perturbations of the tensor, with implications in quantum many-body systems and communication complexity.
Tensor networks are used to efficiently approximate states of strongly-correlated quantum many-body systems. More generally, tensor network approximations may allow to reduce the costs for operating on an order-N tensor from exponential to polynomial in N, and this has become a popular approach for machine learning. We introduce the idea of imposing low-rank constraints on the tensors that compose the tensor network. With this modification, the time and space complexities for the network optimization can be substantially reduced while maintaining high accuracy. We detail this idea for tree tensor network states (TTNS) and projected entangled-pair states. Simulations of spin models on Cayley trees with low-rank TTNS exemplify the effect of rank constraints on the expressive power. We find that choosing the tensor rank r to be on the order of the bond dimension m, is sufficient to obtain high-accuracy groundstate approximations and to substantially outperform standard TTNS computations. Thus low-rank tensor networks are a promising route for the simulation of quantum matter and machine learning on large data sets.
We investigate a novel approach to approximate tensor-network contraction via the exact, matrix-free decomposition of full tensor-networks. We study this method as a means to eliminate the propagation of error in the approximation of tensor-networks. Importantly, this decomposition-based approach is generic, i.e. it does not depend on a specific tensor-network, the tensor index (physical) ordering, or the choice of tensor decomposition. Careful consideration should be made to determine the best decomposition strategy. Furthermore, this method does not rely on robust cancellation of errors (i.e. the Taylor expansion). As a means to study the effectiveness of the approach, we replace the exact contraction of the particle particle ladder (PPL) tensor diagram in the popular coupled-cluster with single and double excitation (CCSD) method with a low-rank tensor decomposition, namely the canonical polyadic decomposition (CPD). With this approach, we replace an O(N6) tensor contractions with a potentially reduced-scaling O(N4R) optimization problem, where R is the CP rank, and we reduce the computational storage of the PPL tensor from O(N4) to O(NR), although we do not take advantage of this compression in this study. To minimize the cost of the CPD optimization, we utilize the iterative structure of CCSD to efficiently initialize the CPD optimization. We show that accurate chemically-relevant energy values can be computed with an error of less than 1 kcal/mol using a relatively low CP rank.
Background: The computational resources needed to generate the ab initio solution of the nuclear many-body problem for increasing mass number and/or accuracy necessitates innovative developments to improve upon (i) the storage of many-body operators and (ii) the scaling of many-body methods used to evaluate nuclear observables. The storing and efficient handling of many-body operators with high particle ranks is currently one of the major bottlenecks limiting the applicability range of ab initio studies with respect to mass number and accuracy. Recently, the application of tensor decomposition techniques to many-body tensors has proven highly beneficial to reduce the computational cost of ab initio calculations in quantum chemistry and solid-state physics.Purpose: The impact of applying state-of-the-art tensor factorization techniques to modern nuclear Hamiltonians derived from chiral effective field theory is investigated. Subsequently, the error induced by the tensor decomposition of the input Hamiltonian on ground-state energies of closed-shell nuclei calculated via second-order many-body perturbation theory is benchmarked.Methods: The first proof-of-principles application of tensor-decomposition techniques to the nuclear Hamiltonian is performed. Two different tensor formats are investigated by systematically benchmarking the approximation error on matrix elements stored in various bases of interest. The analysis is achieved while including normal-ordered three-nucleon interactions that are nowadays used as input to the most advanced ab initio calculations in medium-mass nuclei. With the aid of the factorized Hamiltonian, the second-order perturbative correction to ground-state energies is decomposed and the scaling properties of the underlying tensor network are discussed.Results: The employed tensor formats are found to lead to an efficient data compression of two-body matrix elements of the nuclear Hamiltonian. In particular, the sophisticated tensor hypercontraction scheme yields low tensor ranks with respect to both harmonic-oscillator and Hartree-Fock single-particle bases. It is found that the tensor rank depends on the two-body total angular momentum J for which one performs the decomposition, which is itself directly related to the sparsity of the corresponding tensor. Furthermore, including normal-ordered two-body contributions originating from three-body interactions does not compromise the efficient data compression. Ultimately, the use of factorized matrix elements authorizes controlled approximations of the exact second-order ground-state energy corrections. In particular, a small enough error is obtained from low-rank factorizations in 4He,16O, and 40Ca.Conclusions: It is presently demonstrated that tensor-decomposition techniques can be efficiently applied to systematically approximate the nuclear many-body Hamiltonian in terms of lower-rank tensors. Beyond the input Hamiltonian, tensor-decomposition techniques can be envisioned to scale down the cost of state-of-the-art nonperturbative many-body methods in order to extend ab initio studies to (i) higher precisions, (ii) larger masses, and (iii) nuclei of doubly open-shell character.
We investigate a novel approach to approximate tensor-network contraction via the exact, matrix-free decomposition of full tensor-networks. We study this method as a means to eliminate the propagation of error in the approximation of tensor-networks. Importantly, this decomposition-based approach is generic, i.e., it does not depend on a specific tensor-network, the tensor index (physical) ordering, or the choice of tensor decomposition. Careful consideration should be made to determine the best decomposition strategy. Furthermore, this method does not rely on robust cancellation of errors (i.e., the Taylor expansion). As a means to study the effectiveness of the approach, we replace the exact contraction of the particle–particle ladder (PPL) tensor diagram in the popular coupled-cluster with single and double excitation (CCSD) method with a low-rank tensor decomposition, namely the canonical polyadic decomposition (CPD). With this approach, we replace an O ( N 6 ) tensor contractions with a potentially reduced-scaling O ( N 4 R ) optimization problem, where R is the CP rank, and we reduce the computational storage of the PPL tensor from O ( N 4 ) to O ( N R ), although we do not take advantage of this compression in this study. To minimize the cost of the CPD optimization, we utilize the iterative structure of CCSD to efficiently initialize the CPD optimization. We show that accurate chemically relevant energy values can be computed with an error of less than 1 kcal/mol using a relatively low CP rank.
Tensor decomposition techniques have shown great successes in machine learning and data science by extending classical algorithms based on matrix factorization to multi-modal and multi-way data. However, there exist many tensor decomposition models~(CP, Tucker, Tensor Train, etc.) and the rank of such a decomposition is typically a collection of integers rather than a unique number, making model and hyper-parameter selection a tedious and costly task. At the same time, tensor network methods are powerful tools developed in the physics community which have recently shown their potential for machine learning applications and offer a unifying view of the various tensor decomposition models. In this paper, we leverage the tensor network formalism to develop a generic and efficient adaptive algorithm for tensor learning. Our method is based on a simple greedy approach optimizing a differentiable loss function starting from a rank one tensor and successively identifying the most promising tensor network edges for small rank increments. Our algorithm can adaptively identify tensor network structures with small number of parameters that effectively optimize the objective function from data. The framework we introduce is very broad and encompasses many common tensor optimization problems. Experiments on tensor decomposition and tensor completion tasks with both synthetic and real world data demonstrate the effectiveness of the proposed algorithm.