Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2602.17974

Recursive Sketched Interpolation: Efficient Hadamard Products of Tensor Trains

Zhaonan Meng, Yuehaw Khoo, Jiajia Li, E. Miles Stoudenmire

quant-phmath.NASubmitted Feb 20, 2026• Announced Feb 23, 2026• 0 citations• Revised Oct 6, 2026 (v2)
v1 · Feb 20, 2026v2 · Oct 6, 2026
arXivHTMLPDF

Abstract

The Hadamard product of two tensors in the tensor-train (TT) format is a fundamental operation across various applications, such as TT-based function multiplication for nonlinear differential equations or convolutions. However, conventional methods for computing this product typically scale as at least $\mathcal{O}(χ^4)$ with respect to the TT bond dimension (TT-rank) $χ$, creating a severe computational bottleneck in practice. By combining randomized tensor-train sketching with slice selection via interpolative decomposition, we introduce Recursive Sketched Interpolation (RSI), a ``scale product'' algorithm that computes the Hadamard product of TTs at a computational cost of $\mathcal{O}(χ^3)$. Benchmarks across various TT scenarios demonstrate that RSI offers superior scalability compared to traditional methods while maintaining comparable accuracy. We generalize RSI to compute more complex operations, including Hadamard products of multiple TTs and other element-wise nonlinear mappings, without increasing the complexity beyond $\mathcal{O}(χ^3)$.

Comments
20 pages, 15 figures

Related papers

Fast elementwise operations on tensor trains with alternating cross interpolation

Marc K. Ritter

Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as $O(χ^4)$, where $χ$ is the TT rank. If the rank of the output is smaller than $χ^2$, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in $O(χ^3)$, while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.

math.NACross-listedarXiv:2604.00037•Mar 26, 2026•0 citations

HaTT: Hadamard avoiding TT recompression

Zhonghao Sun, Jizu Huang, Chuanfu Xiao +1 more

The Hadamard product of tensor train (TT) tensors is a fundamental nonlinear operation in scientific computing and data analysis. However, due to its tendency to significantly increase TT ranks, the Hadamard product poses a major computational challenge in TT tensor-based algorithms. To address this, it is crucial to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage costs. In this work, we propose a Hadamard avoiding TT recompression (HaTT) algorithm, which reduces both computational complexity and storage requirements. By leveraging the structure of the Hadamard product in TT tensors and exploiting its Hadamard product-free property, the HaTT algorithm achieves significantly lower complexity compared to existing TT recompression methods. This is confirmed through both complexity analysis and numerical experiments. Furthermore, the HaTT algorithm is applied to solve the Allen--Cahn equation, achieving substantial speedup over existing TT recompression algorithms without sacrificing accuracy.

arXiv:2410.04385•Oct 6, 2024•0 citations

Element-wise Multiplication of Tensor Trains

Alexios A. Michailidis, Christian Fenton, Martin Kiffner

Abstract. We present the tensor train multiplication (TTM) algorithm for the elementwise multiplication of two tensor trains with bond dimension [Formula: see text]. The computational complexity and memory requirements of the TTM algorithm scale as [Formula: see text] and [Formula: see text], respectively. This represents a significant improvement compared with the conventional approach, where the computational complexity scales as [Formula: see text] and memory requirements scale as [Formula: see text]. We benchmark the TTM algorithm using flows obtained from artificial turbulence generation and numerically demonstrate its improved runtime and memory scaling compared with the conventional approach. The TTM algorithm paves the way toward GPU accelerated tensor network simulations of computational fluid dynamics problems with large bond dimensions due to its dramatic improvement in memory scaling.

SIAM Journal on Scientific Computing•Oct 7, 2025•1 citation

Fundamental Tensor Operations for Large-Scale Data Analysis in Tensor Train Formats

Namgil Lee, Andrzej S Cichocki

We discuss extended definitions of linear and multilinear operations such as Kronecker, Hadamard, and contracted products, and establish links between them for tensor calculus. Then we introduce effective low-rank tensor approximation techniques including Candecomp/Parafac (CP), Tucker, and tensor train (TT) decompositions with a number of mathematical and graphical representations. We also provide a brief review of mathematical properties of the TT decomposition as a low-rank approximation technique. With the aim of breaking the curse-of-dimensionality in large-scale numerical analysis, we describe basic operations on large-scale vectors, matrices, and high-order tensors represented by TT decomposition. The proposed representations can be used for describing numerical methods based on TT decomposition for solving large-scale optimization problems such as systems of linear equations and symmetric eigenvalue problems.

arXiv:1405.7786•May 30, 2014•25 citations

Exploring Near-Optimal Contraction Strategies for the Scalar Product in the Tensor-Train Format

Przemysław Dominikowski, Atte Torri, Brice Pointal +3 more

Tensor-train (TT) decomposition has gained tremendous popularity due to its efficiency in handling high-dimensional data arising in scientific and quantum computing as well as machine learning applications. It provides a compact representation of matrices and vectors with a Kronecker product-like low-rank structure, and hence enables efficient matrix-vector operations in this compressed form. The vector scalar product is among such key operations, comprising a series of tensor contractions in a specific TT network where the order of contractions significantly impacts the computational cost. In this work, we introduce fast algorithms for finding a near-optimal contraction ordering for such networks. We show that our algorithms find such contraction orderings efficiently, outperforming the ordering methods for general tensor networks. Compared to our approaches, the best existing method incurs a higher cost of up to 15% for xTy, twice for xTAy, and more than ten times for xTABy scalar products, where A and B are matrices expressed in TT format.

IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum•Jun 3, 2025•0 citations

Learning tensor networks with tensor cross interpolation: New algorithms and libraries

Y. Núñez-Fernández, Marc K. Ritter, Matthieu Jeannin +8 more

The tensor cross interpolation (TCI) algorithm is a rank-revealing algorithm for decomposing low-rank, high-dimensional tensors into tensor trains/matrix product states (MPS). TCI learns a compact MPS representation of the entire object from a tiny training data set. Once obtained, the large existing MPS toolbox provides exponentially fast algorithms for performing a large set of operations. We discuss several improvements and variants of TCI. In particular, we show that replacing the cross interpolation by the partially rank-revealing LU decomposition yields a more stable and more flexible algorithm than the original algorithm. We also present two open source libraries, xfac in Python/C++ and TensorCrossInterpolation.jl in Julia, that implement these improved algorithms, and illustrate them on several applications. These include sign-problem-free integration in large dimension, the “superhigh-resolution” quantics representation of functions, the solution of partial differential equations, the superfast Fourier transform, the computation of partition functions, and the construction of matrix product operators.

arXiv:2407.02454•SciPost Physics•Mar 20, 2025•44 citations
The Hadamard product of two tensors in the tensor-train (TT) format is a fundamental operation across various applications, such as TT-based function multiplication for nonlinear differential equations or convolutions. However, conventional methods for computing this product typically scale as at least O(χ4)\mathcal{O}(χ^4)O(χ4) with respect to the TT bond dimension (TT-rank) χχχ, creating a severe computational bottleneck in practice. By combining randomized tensor-train sketching with slice selection via interpolative decomposition, we introduce Recursive Sketched Interpolation (RSI), a ``scale product'' algorithm that computes the Hadamard product of TTs at a computational cost of O(χ3)\mathcal{O}(χ^3)O(χ3). Benchmarks across various TT scenarios demonstrate that RSI offers superior scalability compared to traditional methods while maintaining comparable accuracy. We generalize RSI to compute more complex operations, including Hadamard products of multiple TTs and other element-wise nonlinear mappings, without increasing the complexity beyond O(χ3)\mathcal{O}(χ^3)O(χ3).
Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as O(χ4)O(χ^4)O(χ4), where χχχ is the TT rank. If the rank of the output is smaller than χ2χ^2χ2, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in O(χ3)O(χ^3)O(χ3), while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.
The Hadamard product of tensor train (TT) tensors is a fundamental nonlinear operation in scientific computing and data analysis. However, due to its tendency to significantly increase TT ranks, the Hadamard product poses a major computational challenge in TT tensor-based algorithms. To address this, it is crucial to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage costs. In this work, we propose a Hadamard avoiding TT recompression (HaTT) algorithm, which reduces both computational complexity and storage requirements. By leveraging the structure of the Hadamard product in TT tensors and exploiting its Hadamard product-free property, the HaTT algorithm achieves significantly lower complexity compared to existing TT recompression methods. This is confirmed through both complexity analysis and numerical experiments. Furthermore, the HaTT algorithm is applied to solve the Allen--Cahn equation, achieving substantial speedup over existing TT recompression algorithms without sacrificing accuracy.
Abstract. We present the tensor train multiplication (TTM) algorithm for the elementwise multiplication of two tensor trains with bond dimension [Formula: see text]. The computational complexity and memory requirements of the TTM algorithm scale as [Formula: see text] and [Formula: see text], respectively. This represents a significant improvement compared with the conventional approach, where the computational complexity scales as [Formula: see text] and memory requirements scale as [Formula: see text]. We benchmark the TTM algorithm using flows obtained from artificial turbulence generation and numerically demonstrate its improved runtime and memory scaling compared with the conventional approach. The TTM algorithm paves the way toward GPU accelerated tensor network simulations of computational fluid dynamics problems with large bond dimensions due to its dramatic improvement in memory scaling.
We discuss extended definitions of linear and multilinear operations such as Kronecker, Hadamard, and contracted products, and establish links between them for tensor calculus. Then we introduce effective low-rank tensor approximation techniques including Candecomp/Parafac (CP), Tucker, and tensor train (TT) decompositions with a number of mathematical and graphical representations. We also provide a brief review of mathematical properties of the TT decomposition as a low-rank approximation technique. With the aim of breaking the curse-of-dimensionality in large-scale numerical analysis, we describe basic operations on large-scale vectors, matrices, and high-order tensors represented by TT decomposition. The proposed representations can be used for describing numerical methods based on TT decomposition for solving large-scale optimization problems such as systems of linear equations and symmetric eigenvalue problems.
Tensor-train (TT) decomposition has gained tremendous popularity due to its efficiency in handling high-dimensional data arising in scientific and quantum computing as well as machine learning applications. It provides a compact representation of matrices and vectors with a Kronecker product-like low-rank structure, and hence enables efficient matrix-vector operations in this compressed form. The vector scalar product is among such key operations, comprising a series of tensor contractions in a specific TT network where the order of contractions significantly impacts the computational cost. In this work, we introduce fast algorithms for finding a near-optimal contraction ordering for such networks. We show that our algorithms find such contraction orderings efficiently, outperforming the ordering methods for general tensor networks. Compared to our approaches, the best existing method incurs a higher cost of up to 15% for xTy, twice for xTAy, and more than ten times for xTABy scalar products, where A and B are matrices expressed in TT format.
The tensor cross interpolation (TCI) algorithm is a rank-revealing algorithm for decomposing low-rank, high-dimensional tensors into tensor trains/matrix product states (MPS). TCI learns a compact MPS representation of the entire object from a tiny training data set. Once obtained, the large existing MPS toolbox provides exponentially fast algorithms for performing a large set of operations. We discuss several improvements and variants of TCI. In particular, we show that replacing the cross interpolation by the partially rank-revealing LU decomposition yields a more stable and more flexible algorithm than the original algorithm. We also present two open source libraries, xfac in Python/C++ and TensorCrossInterpolation.jl in Julia, that implement these improved algorithms, and illustrate them on several applications. These include sign-problem-free integration in large dimension, the “superhigh-resolution” quantics representation of functions, the solution of partial differential equations, the superfast Fourier transform, the computation of partition functions, and the construction of matrix product operators.