Relational tensor decomposition aims to uncover the underlying relationships between entities represented in a multi-dimensional tensor. Traditional methods often struggle with scalability and interpretability, particularly when dealing with complex relational structures. This paper presents a novel approach to relational tensor decomposition that leverages the efficiency of Tensor Trains (TTs) combined with a sparsity-inducing constraint based on relational dependencies. We decompose relational tensors using TT decomposition, and then introduce a sparsity pattern that prioritizes connections between related entities. This process effectively builds a sparse TT representation that explicitly encodes the relational structure of the data. The resulting method demonstrates improved efficiency and enhanced interpretability compared to standard TT decomposition, offering a promising solution for analyzing complex relational data. The core claim is improved efficiency and interpretability of relational tensor decomposition by sparsifying the Tensor Train representation based on relational dependencies. The core mechanism is the application of Tensor Train decomposition to relational tensors, coupled with a sparsity constraint prioritizing connections between related entities.
Relational tensor decomposition aims to uncover the underlying relationships between entities represented in a multi-dimensional tensor. Traditional methods often struggle with scalability and interpretability, particularly when dealing with complex relational structures. This paper presents a novel approach to relational tensor decomposition that leverages the efficiency of Tensor Trains (TTs) combined with a sparsity-inducing constraint based on relational dependencies. We decompose relational tensors using TT decomposition, and then introduce a sparsity pattern that prioritizes connections between related entities. This process effectively builds a sparse TT representation that explicitly encodes the relational structure of the data. The resulting method demonstrates improved efficiency and enhanced interpretability compared to standard TT decomposition, offering a promising solution for analyzing complex relational data. The core claim is improved efficiency and interpretability of relational tensor decomposition by sparsifying the Tensor Train representation based on relational dependencies. The core mechanism is the application of Tensor Train decomposition to relational tensors, coupled with a sparsity constraint prioritizing connections between related entities.
The tensor train (TT) is a popular way of representing high-dimensional hyper-rectangular data structures called tensors. It is widely used, for example, in quantum chemistry under the name “matrix product state”. The complexity of the TT model mainly depends on the bond dimensions that connect TT cores, constituting the model. Unlike canonical polyadic decomposition, the TT model complexity may depend on the order of the modes/indices in the data structures or the order of the core tensors in the TT, in general. This letter aims to provide methods for optimizing the order of the modes to reduce the bond dimensions. Since the number of possible orderings of the cores is exponentially high, we propose a greedy algorithm that provides a suboptimal solution. We consider three problem setups, i.e., specifications of the tensor: tensor given by a list of all its elements, tensor described by a TT model with some default order of the modes, and tensor obtained by sampling a multivariate function.
In the context of big data, high-order tensor decompositions have to face a new challenge in terms of storage and computational costs. The tensor train (TT) decomposition provides a very useful graph-based model reduction, whose storage cost grows linearly with the tensor order D. The computation of the TT-core tensors and TT-ranks can be done in a stable sequential (i.e., noniterative) way thanks to the popular TT-SVD algorithm. In this paper, we exploit the ideas developed for the hierarchical/tree Tucker decomposition in the context of the TT decomposition. Specifically, a new efficient estimation scheme, called TT-HSVD (Tensor-Train Hierarchical SVD), is proposed as a solution to compute the TT decomposition of a high-order tensor. The new algorithm simultaneously delivers the TT-core tensors and their TT-ranks in a hierarchical way. It is a stable (i.e., noniterative) and computationally more efficient algorithm than TT-SVD, which is very important when dealing with large-scale data. The TT-HSVD algorithm uses a new reshaping strategy and a tailored partial SVD, which allows us to deal with smaller matrices compared to those of the TT-SVD. In addition, TT-HSVD is well suited for a parallel processing architecture. An algebraic analysis of the two algorithms is carried out, showing that TT-SVD and TT-HSVD compute the same TT-ranks and TT-core tensors up to specific bases. Simulation results for different tensor orders and dimensions corroborate the effectiveness of the proposed algorithm.
Cyber-physical-social big data generated from ubiquitous devices and diverse spaces generally are multi-source, heterogeneous, and deeply intertwined. To efficiently analyze and handle the ubiquitous cyber-physical-social big data, tensor is considered as an effective tool, but the curse of dimensionality is still the main bottleneck of tensor-based big data analysis. Tensor networks can considerably alleviate or overcome it through the tensor approximate theory. Therefore, this paper focuses on developing an efficient big data processing framework based on tensor networks and providing an incremental tensor train decomposition approach for the streaming big data. Concretely, this paper first presents a hierarchical cyber-physical-social big data processing framework composed of three planes, namely, data representation and decomposition, data storage and processing, and data analysis and service, in which tensor train (TT) and quantized TT decompositions are particularly introduced to remarkably overcome the curse of dimensionality. Besides, to efficiently handle the continuous streaming big data and avoid the repeated decomposition for the history data, an incremental tensor train decomposition (ITTD) approach is proposed and the complexities are further analyzed in detail. Experimental results demonstrate that ITTD demonstrably outperforms the nonincremental TT decomposition in execution time on the precise of guaranteeing the nearly equal approximation error.
Tensor networks have in recent years emerged as the powerful tools for solving the large-scale optimization problems. One of the most popular tensor network is tensor train (TT) decomposition that acts as the building blocks for the complicated tensor networks. However, the TT decomposition highly depends on permutations of tensor dimensions, due to its strictly sequential multilinear products over latent cores, which leads to difficulties in finding the optimal TT representation. In this paper, we introduce a fundamental tensor decomposition model to represent a large dimensional tensor by a circular multilinear products over a sequence of low dimensional cores, which can be graphically interpreted as a cyclic interconnection of 3rd-order tensors, and thus termed as tensor ring (TR) decomposition. The key advantage of TR model is the circular dimensional permutation invariance which is gained by employing the trace operation and treating the latent cores equivalently. TR model can be viewed as a linear combination of TT decompositions, thus obtaining the powerful and generalized representation abilities. For optimization of latent cores, we present four different algorithms based on the sequential SVDs, ALS scheme, and block-wise ALS techniques. Furthermore, the mathematical properties of TR model are investigated, which shows that the basic multilinear algebra can be performed efficiently by using TR representaions and the classical tensor decompositions can be conveniently transformed into the TR representation. Finally, the experiments on both synthetic signals and real-world datasets were conducted to evaluate the performance of different algorithms.
Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)
This paper investigates the application of quantum-inspired principles to enhance anomaly detection using tensor decomposition. Traditional tensor decomposition methods, while effective, often struggle to fully capture the complex, non-linear relationships inherent in high-dimensional data, particularly when dealing with subtle anomalies. We propose a novel approach that leverages concepts from quantum mechanics, specifically tensor train (TT) representations with entanglement-like constraints, to improve anomaly detection performance. The core idea is that anomalies, by their very nature, represent deviations from the expected tensor structure. By incorporating constraints mimicking quantum entanglement, we aim to create tensor representations that are more sensitive to these deviations. We demonstrate, through theoretical analysis and conceptual development, the potential of this approach to improve anomaly detection accuracy. The methodology presented here offers a new perspective on tensor decomposition, moving beyond purely classical approaches and opening avenues for more robust and sensitive anomaly detection in diverse datasets. The focus is on the conceptual framework and the potential benefits rather than empirical validation due to the absence of experimental data.