Abstract. We present the tensor train multiplication (TTM) algorithm for the elementwise multiplication of two tensor trains with bond dimension [Formula: see text]. The computational complexity and memory requirements of the TTM algorithm scale as [Formula: see text] and [Formula: see text], respectively. This represents a significant improvement compared with the conventional approach, where the computational complexity scales as [Formula: see text] and memory requirements scale as [Formula: see text]. We benchmark the TTM algorithm using flows obtained from artificial turbulence generation and numerically demonstrate its improved runtime and memory scaling compared with the conventional approach. The TTM algorithm paves the way toward GPU accelerated tensor network simulations of computational fluid dynamics problems with large bond dimensions due to its dramatic improvement in memory scaling.
Abstract. We present the tensor train multiplication (TTM) algorithm for the elementwise multiplication of two tensor trains with bond dimension [Formula: see text]. The computational complexity and memory requirements of the TTM algorithm scale as [Formula: see text] and [Formula: see text], respectively. This represents a significant improvement compared with the conventional approach, where the computational complexity scales as [Formula: see text] and memory requirements scale as [Formula: see text]. We benchmark the TTM algorithm using flows obtained from artificial turbulence generation and numerically demonstrate its improved runtime and memory scaling compared with the conventional approach. The TTM algorithm paves the way toward GPU accelerated tensor network simulations of computational fluid dynamics problems with large bond dimensions due to its dramatic improvement in memory scaling.
We present the Tensor Train Multiplication (TTM) algorithm for the elementwise multiplication of two tensor trains with bond dimension χ. The computational complexity and memory requirements of the TTM algorithm scale as χ3 and χ2, respectively. This represents a significant improvement compared with the conventional approach, where the computational complexity scales as χ4 and memory requirements scale as χ3.We benchmark the TTM algorithm using flows obtained from artificial turbulence generation and numerically demonstrate its improved runtime and memory scaling compared with the conventional approach. The TTM algorithm paves the way towards GPU accelerated tensor network simulations of computational fluid dynamics problems with large bond dimensions due to its dramatic improvement in memory scaling.
Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as O(χ4), where χ is the TT rank. If the rank of the output is smaller than χ2, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in O(χ3), while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.
The Hadamard product of two tensors in the tensor-train (TT) format is a fundamental operation across various applications, such as TT-based function multiplication for nonlinear differential equations or convolutions. However, conventional methods for computing this product typically scale as at least O(χ4) with respect to the TT bond dimension (TT-rank) χ, creating a severe computational bottleneck in practice. By combining randomized tensor-train sketching with slice selection via interpolative decomposition, we introduce Recursive Sketched Interpolation (RSI), a ``scale product'' algorithm that computes the Hadamard product of TTs at a computational cost of O(χ3). Benchmarks across various TT scenarios demonstrate that RSI offers superior scalability compared to traditional methods while maintaining comparable accuracy. We generalize RSI to compute more complex operations, including Hadamard products of multiple TTs and other element-wise nonlinear mappings, without increasing the complexity beyond O(χ3).
Tensor network algorithms can efficiently simulate complex quantum many-body systems by utilizing knowledge of their structure and entanglement. These methodologies have been adapted recently for solving the Navier-Stokes equations, which describe a spectrum of fluid phenomena, from the aerodynamics of vehicles to weather patterns. Within this quantum-inspired paradigm, velocity is encoded as matrix product states (MPS), effectively harnessing the analogy between interscale correlations of fluid dynamics and entanglement in quantum many-body physics. This particular tensor structure is also called quantics tensor train (QTT). By utilizing NVIDIA's cuQuantum library to perform parallel tensor computations on GPUs, our adaptation speeds up simulations by up to 12.1 times. This allows us to study the algorithm in terms of its applicability, scalability, and performance. By simulating two qualitatively different but commonly encountered 2D flow problems at high Reynolds numbers up to 1 × 10 7 using a fourth-order time stepping scheme, we find that the algorithm has a potential advantage over direct numerical simulations in the turbulent regime as the requirements for grid resolution increase drastically. In addition, we derive the scaling χ = O ( poly ( 1 / ε ) ) for the maximum bond dimension χ of MPS representing turbulent flow fields, with an error ε , based on the spectral distribution of turbulent kinetic energy. Our findings motivate further exploration of related quantum algorithms and other tensor network methods.
High performance computers produce extreme-scale data sets that require sampling or compression if they are to be used to their full potential. Existing data compression techniques typically exploit features such as sparsity in the data, homogeneity in the data, or {\it a priori} knowledge of what subsets of data are of most interest. Fluid dynamics data in general do not exhibit these features and so are attractive test beds for generic compression techniques that are objective, robust, and tuneable with respect to information lost due to compression. Presented here is a method based on tensor networks, specifically matrix product states or tensor trains, that meets these requirements. The method is demonstrated for compression in one-dimension and is extensible to higher dimensionality. Lossless compression is demonstrated for random Fourier series for sufficiently high bond dimension of the tensor network, with the memory required to store the tensor network scaling directly proportional to the bond dimension. The lossy compression exhibited at lower bond dimension can be well within the relative error of many fluid simulations. The compression algorithm is tested for the time evolution of Burger's equation with excellent results. We additionally demonstrate the capability to perform computations in the compressed form through a tensor network periodic convolution that can be orders of magnitude faster than using fast Fourier transforms and the convolution theorem. In addition to being an attractive method for working with data sets generated by existing computers, the tensor network methods utilised are directly translatable to the emerging paradigm of quantum computing.
The tensor train (TT) is a popular way of representing high-dimensional hyper-rectangular data structures called tensors. It is widely used, for example, in quantum chemistry under the name “matrix product state”. The complexity of the TT model mainly depends on the bond dimensions that connect TT cores, constituting the model. Unlike canonical polyadic decomposition, the TT model complexity may depend on the order of the modes/indices in the data structures or the order of the core tensors in the TT, in general. This letter aims to provide methods for optimizing the order of the modes to reduce the bond dimensions. Since the number of possible orderings of the cores is exponentially high, we propose a greedy algorithm that provides a suboptimal solution. We consider three problem setups, i.e., specifications of the tensor: tensor given by a list of all its elements, tensor described by a TT model with some default order of the modes, and tensor obtained by sampling a multivariate function.