Abstract Next-generation mobility services require a huge amount of data with multiple attributes. This data is stored as a multi-dimensional array called a tensor. A tensor network is an effective tool for representing a large composite tensor. As an application of the tensor-network formalism to tensor data processing, we present three research results from statistical physics: tree tensor networks, tensor ring decomposition, and MERA.
Abstract Next-generation mobility services require a huge amount of data with multiple attributes. This data is stored as a multi-dimensional array called a tensor. A tensor network is an effective tool for representing a large composite tensor. As an application of the tensor-network formalism to tensor data processing, we present three research results from statistical physics: tree tensor networks, tensor ring decomposition, and MERA.
A tensor network is a type of decomposition used to express and approximate large arrays of data.A given dataset, quantum state, or higher-dimensional multilinear map is factored and approximated by a composition of smaller multilinear maps.This is reminiscent to how a Boolean function might be decomposed into a gate array: this represents a special case of tensor decomposition, in which the tensor entries are replaced by 0, 1 and the factorisation becomes exact.The associated techniques are called tensor network methods: the subject developed independently in several distinct fields of study, which have more recently become interrelated through the language of tensor networks.The tantamount questions in the field relate to expressability of tensor networks and the reduction of computational overheads.A merger of tensor networks with machine learning is natural.On the one hand, machine learning can aid in determining a factorisation of a tensor network approximating a data set.On the other hand, a given tensor network structure can be viewed as a machine learning model.Herein the tensor network parameters are adjusted to learn or classify a data-set.In this survey we review the basics of tensor networks and explain the ongoing effort to develop the theory of tensor networks in machine learning.1
Many problems in computational neuroscience, neuroinformatics, pattern/image recognition, signal processing and machine learning generate massive amounts of multidimensional data with multiple aspects and high dimensionality. Tensors (i.e., multi-way arrays) provide often a natural and compact representation for such massive multidimensional data via suitable low-rank approximations. Big data analytics require novel technologies to efficiently process huge datasets within tolerable elapsed times. Such a new emerging technology for multidimensional big data is a multiway analysis via tensor networks (TNs) and tensor decompositions (TDs) which represent tensors by sets of factor (component) matrices and lower-order (core) tensors. Dynamic tensor analysis allows us to discover meaningful hidden structures of complex data and to perform generalizations by capturing multi-linear and multi-aspect relationships. We will discuss some fundamental TN models, their mathematical and graphical descriptions and associated learning algorithms for large-scale TDs and TNs, with many potential applications including: Anomaly detection, feature extraction, classification, cluster analysis, data fusion and integration, pattern recognition, predictive modeling, regression, time series analysis and multiway component analysis. Keywords: Large-scale HOSVD, Tensor decompositions, CPD, Tucker models, Hierarchical Tucker (HT) decomposition, low-rank tensor approximations (LRA), Tensorization/Quantization, tensor train (TT/QTT) - Matrix Product States (MPS), Matrix Product Operator (MPO), DMRG, Strong Kronecker Product (SKP).
Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)
High-dimensional data arise naturally in many areas of science and engineering, including machine learning, signal processing, computational physics, and statistics. Such data are often represented as tensors, multi-dimensional generalizations of matrices. While tensors provide a natural representation for multi-modal structure, their direct manipulation quickly becomes challenging as the order grows: the number of parameters increases exponentially, and algebraic expressions involving many indices become difficult to interpret and implement. Tensor networks (TNs) provide an effective framework for addressing these challenges. Originally introduced by Penrose and developed extensively in quantum physics, the graphical language of tensor networks encodes contractions as edges in a graph, reducing notational overhead and revealing structural properties obscured by index notation. Despite the central role of high-dimensional tensors in modern machine learning and numerical analysis, tensor network diagrams remain underutilized outside quantum computing, partly due to the lack of a self-contained mathematical reference accessible to a broad technical audience. This manuscript provides a self-contained guide to tensor networks and their use in tensor algebra. We present the main operations on tensors, contractions, products, and reshaping through, graphical notation, and show how classical tensor decompositions and related computations are naturally expressed in this framework. We also illustrate how tensor networks simplify the derivation of gradients and the manipulation of high-dimensional probability distributions. Throughout, we show that the diagrammatic approach yields genuinely shorter and more transparent proofs of classical identities, rank bounds, and gradient formulas that would otherwise require laborious index manipulation.
In this thesis, we use the new numerical method multiscale entanglement
renormalization ansatz (MERA) to study the one dimensional spin system. This
method, MERA, utilizes a network of tensors to represent quantum many-body
systems on a lattice. Other examples of tensor network state are matrix product
states (MPSs) for 1D systems, tree tensor networks (TTNs) for systems with a
tree shape, and projected entangled-pair states (PEPSs) for 2D systems and be-
yond. The three structures dier in the graph that denes how the tensors are
interconnected into a network: The graphs for MPSs, TTNs, and 2D PEPSs are,
respectively, a chain, a tree, and a 2D lattice. Importantly, from these tensor
networks the expectation value of local observables can be computed eciently.
Developing algorithms to simulate quantum many-body systems is important
for understanding and studying the physical principles. And these may become
powerful tools for the analysis of quantum computation and quantum information
theory. But in general ecient simulation is dicult, due to the fundamental
quantum physical laws.
The main method we have studied and worked on is multiscale entangle-
ment renormalization ansatz (MERA) which is a variational ansatz for many-
body states. MERA is a tree tensor network for 1D, 2D systems and beyond.
We have developed code of ternary-MERA to compute the ground state energy,
expected values of local observables, and correlators for 1D lattice systems. In
a nutshell, MERA is an algorithm for entanglement renormalization which is a
numerical technique based on locally reorganizing the Hilbert space of a quantum
many-body system with the aim to reduce the amount of entanglement in its wave
function. We use this method to study 1D Ising model, XXZ model, and J1 - J2
model, and then comparing the numerical results with exact diagonalization (ED)
and density matrix renormalization group (DMRG).
Tensor networks can compactly represent high-dimensional data in the form of low-rank tensor decompositions, which helps capture complex structures and correlations within the data. However, their inherently multilinear nature limits their potential as universal approximators. To address this limitation, tensor networks are combined with classical neural network layers, introducing nonlinear elements to enhance the model’s expressive power. The Hybrid Tensor Networks (HTN) system encompasses two prevalent tensor network configurations, Matrix Product States and Tree Tensor Networks, which facilitates the adaptable design of models customized for particular machine learning applications. This framework has been validated on classification and image reconstruction tasks, demonstrating its potential to handle complex data structures and its applicability within the machine learning field. Employing this approach, the goal is to establish a foundation for future research on hybrid quantum-classical models, thereby opening up avenues for the realm of quantum machine learning.