It is well known that tensor decompositions show separations, that is, constraints on local terms (such as positivity) may entail an arbitrarily high cost in their representation. Here, we show that many of these separations disappear in the approximate case. Specifically, for every approximation error ɛ and norm, we define the approximate rank as the minimum rank of an element in the ɛ-ball with respect to that norm. For positive semidefinite matrices, we show that the separations between rank, purification rank, and separable rank disappear for a large class of Schatten p-norms. For non-negative tensors, we show that the separations between rank, positive semidefinite rank, and non-negative rank disappear for all ℓp-norms with p > 1. For the trace norm (p = 1), we obtain upper bounds that depend on the ambient dimension. We also provide a deterministic algorithm to obtain the approximate decomposition attaining our bounds. Our main tool is an approximate version of the Carathéodory theorem. Our results imply that many separations are not robust under small perturbations of the tensor, with implications in quantum many-body systems and communication complexity.
Tensors with unit Frobenius norm are fundamental objects in many fields, including scientific computing and quantum physics, which are able to represent normalized eigenvectors and pure quantum states. While the tensor train decomposition provides a powerful low-rank format for tackling high-dimensional problems, it does not intrinsically enforce the unit-norm constraint. To address this, we introduce the normalized tensor train (NTT) decomposition, which aims to approximate a tensor by unit-norm tensors in tensor train format. The low-rank structure of NTT decomposition not only saves storage and computational cost but also preserves the underlying unit-norm structure. We prove that the set of fixed-rank NTT tensors forms a smooth manifold, and the corresponding Riemannian geometry is derived, paving the way for geometric methods. We propose NTT-based methods for low-rank tensor recovery, high-dimensional eigenvalue problem, estimation of stabilizer rank, and calculation of the minimum output Rényi 2-entropy of quantum channels. Numerical experiments demonstrate the superior efficiency and scalability of the proposed NTT-based methods.
This dissertation reviews classical vector - tensor analysis, building up to the necessary techniques required to decompose a tensor into a tensor train and to reconstruct it back into the original tensor with minimal error. The tensor train decomposition decomposes a tensor of dimensionality d into a train of d third order tensors, whose sizes are dependent upon the rank and chosen error bound. I will be reviewing the required operations of matricization, tensor - matrix, vector and tensor multiplication to be able to compute this decomposition. I then move onto analysing the tensor train decomposition by ap-plying it to different types of tensor, of differing dimensionality with a variety of accuracy bounds to investigate their influence on the time taken to complete the decomposition and the final absolute error. Finally I explore a method to compute a d-dimensional integration from the tensor train, which will allow larger tensors to be integrated with the memory required dramatically reduced after the tensor is decomposed. I will be applying this technique to two tensors with different ranks and compare the efficiency and accuracy of integrating directly from the tensor to that of the tensor train decomposition.
Abstract Often, large, high-dimensional datasets collected across multiple modalities can be organized as a higher-order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low-dimensional structures underlying such data. However, we currently lack a theoretical understanding of the algorithmic behavior of low-rank tensor decompositions. We derive Bayesian approximate message passing (AMP) algorithms for recovering arbitrarily shaped low-rank tensors buried within noise, and we employ dynamic mean field theory to precisely characterize their performance. Our theory reveals the existence of phase transitions between easy, hard and impossible inference regimes, and displays an excellent match with simulations. Moreover it reveals several qualitative surprises compared to the behavior of symmetric, cubic tensor decomposition. Finally, we compare our AMP algorithm to the most commonly used algorithm, alternating least squares (ALS), and demonstrate that AMP significantly outperforms ALS in the presence of noise.
Free tensors are tensors which, after a change of bases, have free support: any two distinct elements of its support differ in at least two coordinates. They play a distinguished role in the theory of bilinear complexity, in particular in Strassen's duality theory for asymptotic rank. Within the context of quantum information theory, where tensors are interpreted as multiparticle quantum states, freeness corresponds to a type of multiparticle Schmidt decomposition. In particular, if a state is free in a given basis, the reduced density matrices are diagonal. Although generic tensors in
Cn⊗Cn⊗Cn are non-free for
n≥4 by parameter counting, no explicit non-free tensors were known until now. We solve this hay in a haystack problem by constructing explicit tensors that are non-free for every
n≥3. In particular, this establishes that non-free tensors exist in
Cn⊗Cn⊗Cn, where they are not generic. To establish non-freeness, we use results from geometric invariant theory and the theory of moment polytopes. In particular, we show that if a tensor
T is free, then there is a tensor
S in the GL-orbit closure of
T, whose support is free and whose moment map image is the minimum-norm point of the moment polytope of
T. This implies a reduction for checking non-freeness from arbitrary basis changes of
T to unitary basis changes of
S. The unitary equivariance of the moment map can then be combined with the fact that tensors with free support have diagonal moment map image, in order to further restrict the set of relevant basis changes.
In problems involving approximation, completion, denoising, dimension reduction, estimation, interpolation, modeling, order reduction, regression, etc, we argue that the near-universal practice of assuming that a function, matrix, or tensor (which we will see are all the same object in this context) has \emph{low rank} may be ill-justified. There are many natural instances where the object in question has high rank with respect to the classical notions of rank: matrix rank, tensor rank, multilinear rank --- the latter two being the most straightforward generalizations of the former. To remedy this, we show that one may vastly expand these classical notions of ranks: Given any undirected graph
G, there is a notion of
G-rank associated with
G, which provides us with as many different kinds of ranks as there are undirected graphs. In particular, the popular tensor network states in physics (e.g., \textsc{mps}, \textsc{ttns}, \textsc{peps}) may be regarded as functions of a specific
G-rank for various choices of
G. Among other things, we will see that a function, matrix, or tensor may have very high matrix, tensor, or multilinear rank and yet very low
G-rank for some
G. In fact the difference is in the orders of magnitudes and the gaps between
G-ranks and these classical ranks are arbitrarily large for some important objects in computer science, mathematics, and physics. Furthermore, we show that there is a
G such that almost every tensor has
G-rank exponentially lower than its rank or the dimension of its ambient space.