Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2606.01852

Parallelizing Large-Scale Tensor Network Contraction on Multiple GPUs

Feng Pan, Hanfeng Gu, Paul Springer, Xipeng Li

cs.DCquant-phSubmitted Jun 1, 2026• Announced Jun 2, 2026• 0 citations
arXivHTMLPDF

Abstract

Exact tensor network contraction underpins quantum circuit simulation, quantum error correction, combinatorial optimization, and many-body dynamics. The dominant parallelization strategy, slicing, scales exponentially and incurs redundant computation. We present a multi-GPU framework that instead distributes intermediate tensors across devices with explicit communication, converting a fixed contraction path into a communication-efficient schedule via GEMM-oriented mode reordering and communication-aware mode distribution planning. Within a single DGX H100 node (8 GPUs, NVLink), distribution delivers $7$--$173\times$ extra speedup beyond embarrassingly parallel slicing, capturing nearly all of the available compute reduction (87--101%) because NVLink's high bandwidth keeps communication small relative to compute. Scaling the same four workloads to 1024 H100 GPUs over InfiniBand, the extra speedup beyond slicing ranges from $42\times$ to $67{,}869\times$, demonstrating that communication-aware distributed contraction far surpasses slicing-based scaling limits for frontier tensor networks.

Comments
12 pages, 6 figures, comments are welcome

Related papers

Efficient tensor network contraction with GPU acceleration and parallel strategies

Wei Min Xu, Weiqin Tong, Xiao-Yang Liu

Tensor network contraction is a fundamental operation for dealing with high-dimensional data, which is now widely used in classical simulations of quantum circuits. To improve the performance of simulations, existing methods are implemented by optimizing contraction methods. However, with the increasing size of the tensor network, the contraction efficiency of these methods is difficult to satisfy the performance requirements and also suffers from the challenge of contracting tensor tasks that exceed GPU memory. In this paper, we propose a high-performance chunking method for tensor contraction. By chunking the given tensor, we obtain small tensors or vectors assign the corresponding threads, and contract them in parallel through the GPU. Solving high-order tensor contractions using distributed tensor contraction methods. Compared to the Einsum, our method achieves an average speedup of 22.28×. We also have experiments for large-scale tensor network contractions and our method achieves an average speedup of 13.56× compared to Tensordot.

International Conference on Electronic Information Engineering, Big Data and Computer Technology (EIBDCT)•Jul 19, 2024•0 citations

Massive Parallel Simulation for Tensor Network Contraction Ordering Solving

Wei Xu, Weiqin Tong, Xiao-Yang Liu

Tensor network contraction is a powerful mathematical tool for dealing with complex systems and high-dimensional data, which is widely used in classical simulations of molecules and quantum circuits. To improve the performance of simulations, existing methods are implemented by searching the optimal contraction orderings for tensor networks. In particular, some reinforcement learning (RL) methods show a greater potential for solving. However, with the increasing scale of simulation, the number of tensors in the corresponding tensor network is huge, the existing RL methods suffer from sampling bottlenecks and slow solving speed, which make it difficult to efficiently solve for high-quality contraction orderings. In this paper, we design a massive parallel simulation environment for tensor network contraction ordering solving on a single GPU. We experimented with multiple numbers of parallel environments on the tensor-train network, and achieved the highest speedup when the number of environments was 26. We also have experimented on large-scale tensor-train networks and tree tensor networks, and compared to Kahypar and RL-Baseline, our method achieves average speedup of 8.96× and 7.48× on tensor-train networks, and 10.03× and 7.48× on tree tensor networks.

International Conference on Neural Networks, Information and Communication Engineering (NNICE)•Jan 19, 2024•0 citations

Implementation of Tensor Network Simulation TN-Sim under NWQ-Sim

Aaron C. Hoyt, Jonathan S. Bersson, Sean Garner +2 more

Large-scale tensor network simulations are crucial for developing robust complexity-theoretic bounds on classical quantum simulation, enabling circuit cutting approaches, and optimizing circuit compilation, all of which aid efficient quantum computation on limited quantum resources. Modern exascale high-performance computing platforms offer significant potential for advancing tensor network quantum circuit simulation capabilities. We implement TN-Sim, a tensor network simulator backend within the NWQ-Sim software package that utilizes the Tensor Algebra for Many-body Methods (TAMM) framework to support both distributed HPC-scale computations and local simulations with ITensor. To optimize the scale up in computation across multiple nodes we implement a task based parallelization scheme to demonstrate parallelized gate contraction for wide quantum circuits with many gates per layer. Through the integration of the TAMM framework with Matrix Product State (MPS) tensor network approaches, we deliver a simulation environment that can scale from local systems to HPC clusters. We demonstrate an MPS tensor network simulator running on the state-of-the-art Perlmutter (NVIDIA) supercomputer and discuss the potential portability of this software to HPC clusters such as Frontier (AMD) and Aurora (Intel). We also discuss future improvements including support for different tensor network topologies and enhanced computational efficiency.

quant-pharXiv:2601.04422•IEEE High Performance Extreme Computing Conference•Jan 9, 2026•0 citations

Distributed Quantum Simulation via Tensor Networks

Jincheng Zhang

This paper explores the potential of tensor networks for enabling distributed quantum simulation, a crucial step towards scaling up quantum computations. Traditional quantum simulation faces limitations due to the exponential growth of state vector sizes required for complex systems. Distributed quantum simulation aims to overcome this by partitioning the simulation across multiple quantum processors. However, efficient representation and manipulation of the resulting distributed quantum state remain a significant challenge. This work proposes utilizing tensor networks as a framework for representing and processing these distributed quantum states. Tensor networks offer an inherent structure that facilitates efficient parallel computation and minimizes communication overhead between quantum processors. We outline a theoretical framework, detailing the application of tensor network contractions and disentanglement techniques specifically tailored for distributed quantum simulation scenarios. The core claim of this work is that scaling up quantum simulations requires efficient methods for representing and manipulating quantum states, and tensor networks provide a promising avenue for achieving this goal. We present a method to reduce the computational complexity associated with simulating large quantum systems by leveraging the properties of tensor networks. The approach allows for a more scalable simulation strategy, moving beyond the limitations of traditional state vector representations. Further research will explore specific tensor network architectures and optimization strategies for different simulation problems.

•Sep 7, 2026•0 citations

Efficient Quantum Circuit Simulation by Tensor Network Methods on Modern GPUs

Feng Ping Pan, Hanfeng Gu, Lülin Kuang +2 more

Efficient simulation of quantum circuits has become indispensable with the rapid development of quantum hardware. The primary simulation methods are based on state vectors and tensor networks. As the number of qubits and quantum gates grows larger in current quantum devices, traditional state-vector-based quantum circuit simulation methods prove inadequate due to the overwhelming size of the Hilbert space and extensive entanglement. Consequently, brutal force tensor network simulation algorithms become the only viable solution in such scenarios. The two main challenges faced in tensor network simulation algorithms are optimal contraction path finding and efficient execution on modern computing devices, with the latter determining the actual efficiency. In this study, we investigate the optimization of such tensor network simulations on modern GPUs and propose general optimization strategies from two aspects: computational efficiency and accuracy. First, we propose to transform critical Einstein summation operations into GEMM operations, leveraging the specific features of tensor network simulations to amplify the efficiency of GPUs. Second, by analyzing the data characteristics of quantum circuits, we employ extended precision to ensure the accuracy of simulation results and mixed precision to fully exploit the potential of GPUs, resulting in faster and more precise simulations. Our numerical experiments demonstrate that our approach can achieve a 3.96× reduction in verification time for random quantum circuit samples in the 18-cycle case of Sycamore, with sustained performance exceeding 21 TFLOPS on one A100. This method can be easily extended to the 20-cycle case, maintaining the same performance, accelerating by 12.5× compared to the state-of-the-art CPU-based results and 4.48–6.78× compared to the state-of-the-art GPU-based results reported in the literature. Furthermore, the strategies presented in this article hold general applicability for accelerating problems that can be tackled by contracting tensor networks such as graphical models and combinatorial optimizations.

arXiv:2310.03978•ACM Transactions on Quantum Computing•Sep 23, 2024•10 citations

Efficient parallelization of tensor network contraction for simulating quantum computation

Cupjin Huang, Fang Zhang, Michael Newman +18 more

Abstract We develop an algorithmic framework for contracting tensor networks and demonstrate its power by classically simulating quantum computation of sizes previously deemed out of reach. Our main contribution, index slicing, is a method that efficiently parallelizes the contraction by breaking it down into much smaller and identically structured subtasks, which can then be executed in parallel without dependencies. We benchmark our algorithm on a class of random quantum circuits, achieving greater than 105 times acceleration over the original estimate of the simulation cost. We then demonstrate applications of the simulation framework for aiding the development of quantum algorithms and quantum error correction. As tensor networks are widely used in computational science, our simulation framework may find further applications.

Nature Computational Science•Sep 13, 2021•74 citations
Exact tensor network contraction underpins quantum circuit simulation, quantum error correction, combinatorial optimization, and many-body dynamics. The dominant parallelization strategy, slicing, scales exponentially and incurs redundant computation. We present a multi-GPU framework that instead distributes intermediate tensors across devices with explicit communication, converting a fixed contraction path into a communication-efficient schedule via GEMM-oriented mode reordering and communication-aware mode distribution planning. Within a single DGX H100 node (8 GPUs, NVLink), distribution delivers 777--173×173\times173× extra speedup beyond embarrassingly parallel slicing, capturing nearly all of the available compute reduction (87--101%) because NVLink's high bandwidth keeps communication small relative to compute. Scaling the same four workloads to 1024 H100 GPUs over InfiniBand, the extra speedup beyond slicing ranges from 42×42\times42× to 67,869×67{,}869\times67,869×, demonstrating that communication-aware distributed contraction far surpasses slicing-based scaling limits for frontier tensor networks.
Tensor network contraction is a fundamental operation for dealing with high-dimensional data, which is now widely used in classical simulations of quantum circuits. To improve the performance of simulations, existing methods are implemented by optimizing contraction methods. However, with the increasing size of the tensor network, the contraction efficiency of these methods is difficult to satisfy the performance requirements and also suffers from the challenge of contracting tensor tasks that exceed GPU memory. In this paper, we propose a high-performance chunking method for tensor contraction. By chunking the given tensor, we obtain small tensors or vectors assign the corresponding threads, and contract them in parallel through the GPU. Solving high-order tensor contractions using distributed tensor contraction methods. Compared to the Einsum, our method achieves an average speedup of 22.28×. We also have experiments for large-scale tensor network contractions and our method achieves an average speedup of 13.56× compared to Tensordot.
Tensor network contraction is a powerful mathematical tool for dealing with complex systems and high-dimensional data, which is widely used in classical simulations of molecules and quantum circuits. To improve the performance of simulations, existing methods are implemented by searching the optimal contraction orderings for tensor networks. In particular, some reinforcement learning (RL) methods show a greater potential for solving. However, with the increasing scale of simulation, the number of tensors in the corresponding tensor network is huge, the existing RL methods suffer from sampling bottlenecks and slow solving speed, which make it difficult to efficiently solve for high-quality contraction orderings. In this paper, we design a massive parallel simulation environment for tensor network contraction ordering solving on a single GPU. We experimented with multiple numbers of parallel environments on the tensor-train network, and achieved the highest speedup when the number of environments was 26. We also have experimented on large-scale tensor-train networks and tree tensor networks, and compared to Kahypar and RL-Baseline, our method achieves average speedup of 8.96× and 7.48× on tensor-train networks, and 10.03× and 7.48× on tree tensor networks.
Large-scale tensor network simulations are crucial for developing robust complexity-theoretic bounds on classical quantum simulation, enabling circuit cutting approaches, and optimizing circuit compilation, all of which aid efficient quantum computation on limited quantum resources. Modern exascale high-performance computing platforms offer significant potential for advancing tensor network quantum circuit simulation capabilities. We implement TN-Sim, a tensor network simulator backend within the NWQ-Sim software package that utilizes the Tensor Algebra for Many-body Methods (TAMM) framework to support both distributed HPC-scale computations and local simulations with ITensor. To optimize the scale up in computation across multiple nodes we implement a task based parallelization scheme to demonstrate parallelized gate contraction for wide quantum circuits with many gates per layer. Through the integration of the TAMM framework with Matrix Product State (MPS) tensor network approaches, we deliver a simulation environment that can scale from local systems to HPC clusters. We demonstrate an MPS tensor network simulator running on the state-of-the-art Perlmutter (NVIDIA) supercomputer and discuss the potential portability of this software to HPC clusters such as Frontier (AMD) and Aurora (Intel). We also discuss future improvements including support for different tensor network topologies and enhanced computational efficiency.
This paper explores the potential of tensor networks for enabling distributed quantum simulation, a crucial step towards scaling up quantum computations. Traditional quantum simulation faces limitations due to the exponential growth of state vector sizes required for complex systems. Distributed quantum simulation aims to overcome this by partitioning the simulation across multiple quantum processors. However, efficient representation and manipulation of the resulting distributed quantum state remain a significant challenge. This work proposes utilizing tensor networks as a framework for representing and processing these distributed quantum states. Tensor networks offer an inherent structure that facilitates efficient parallel computation and minimizes communication overhead between quantum processors. We outline a theoretical framework, detailing the application of tensor network contractions and disentanglement techniques specifically tailored for distributed quantum simulation scenarios. The core claim of this work is that scaling up quantum simulations requires efficient methods for representing and manipulating quantum states, and tensor networks provide a promising avenue for achieving this goal. We present a method to reduce the computational complexity associated with simulating large quantum systems by leveraging the properties of tensor networks. The approach allows for a more scalable simulation strategy, moving beyond the limitations of traditional state vector representations. Further research will explore specific tensor network architectures and optimization strategies for different simulation problems.
Efficient simulation of quantum circuits has become indispensable with the rapid development of quantum hardware. The primary simulation methods are based on state vectors and tensor networks. As the number of qubits and quantum gates grows larger in current quantum devices, traditional state-vector-based quantum circuit simulation methods prove inadequate due to the overwhelming size of the Hilbert space and extensive entanglement. Consequently, brutal force tensor network simulation algorithms become the only viable solution in such scenarios. The two main challenges faced in tensor network simulation algorithms are optimal contraction path finding and efficient execution on modern computing devices, with the latter determining the actual efficiency. In this study, we investigate the optimization of such tensor network simulations on modern GPUs and propose general optimization strategies from two aspects: computational efficiency and accuracy. First, we propose to transform critical Einstein summation operations into GEMM operations, leveraging the specific features of tensor network simulations to amplify the efficiency of GPUs. Second, by analyzing the data characteristics of quantum circuits, we employ extended precision to ensure the accuracy of simulation results and mixed precision to fully exploit the potential of GPUs, resulting in faster and more precise simulations. Our numerical experiments demonstrate that our approach can achieve a 3.96× reduction in verification time for random quantum circuit samples in the 18-cycle case of Sycamore, with sustained performance exceeding 21 TFLOPS on one A100. This method can be easily extended to the 20-cycle case, maintaining the same performance, accelerating by 12.5× compared to the state-of-the-art CPU-based results and 4.48–6.78× compared to the state-of-the-art GPU-based results reported in the literature. Furthermore, the strategies presented in this article hold general applicability for accelerating problems that can be tackled by contracting tensor networks such as graphical models and combinatorial optimizations.
Abstract We develop an algorithmic framework for contracting tensor networks and demonstrate its power by classically simulating quantum computation of sizes previously deemed out of reach. Our main contribution, index slicing, is a method that efficiently parallelizes the contraction by breaking it down into much smaller and identically structured subtasks, which can then be executed in parallel without dependencies. We benchmark our algorithm on a class of random quantum circuits, achieving greater than 105 times acceleration over the original estimate of the simulation cost. We then demonstrate applications of the simulation framework for aiding the development of quantum algorithms and quantum error correction. As tensor networks are widely used in computational science, our simulation framework may find further applications.