Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2309.16724

Boosting the effective performance of massively parallel tensor network state algorithms on hybrid CPU-GPU based architectures via non-Abelian symmetries

Andor Menczer, Örs Legeza

Published Sep 23, 2023• 2 citations
arXivHTMLPDF

Abstract

We present novel algorithmic solutions together with implementation details utilizing non-Abelian symmetries in order to boost the current limits of tensor network state algorithms on high performance computing infrastructure. In our in-house developed hybrid CPU-multiGPU solution scheduling is decentralized, threads are autonomous and inter-thread communications are solely limited to interactions with globally visible lock-free constructs. Our custom tailored virtual memory management ensures data is produced with high spatial locality, which together with the use of specific sequences of strided batched matrix operations translates to significantly higher overall throughput. In order to lower IO overhead, an adaptive buffering technique is used to dynamically match the level of data abstraction, at which cache repositories are built and reused, to system resources. The non-Abelian symmetry related tensor algebra based on Wigner-Eckhart theorem is fully detached from the conventional tensor network layer, thus massively parallel matrix and tensor operations can be performed without additional overheads. Altogether, we have achieved an order of magnitude increase in performance with respect to results reported in arXiv:2305.05581 in terms of computational complexity and at the same time a factor of three to six in the actual performance measured in TFLOPS. Benchmark results are presented on Hilbert space dimensions up to $2.88\times10^{36}$ obtained via large-scale SU(2) spin adapted density matrix renormalization group simulations on selected strongly correlated molecular systems. These demonstrate the utilization of NVIDIA's highly specialized tensor cores, leading to performance around 110 TFLOPS on a single node supplied with eight NVIDIA A100 devices. In comparison to U(1) implementations with matching accuracy, our solution has an estimated effective performance of 250-500 TFLOPS.

DOI
10.48550/arxiv.2309.16724

Related papers

Massively Parallel Tensor Network State Algorithms on Hybrid CPU-GPU Based Architectures

Andor Menczer, Örs Legeza

High Resolution Image Download MS PowerPoint Slide The interplay of quantum and classical simulation and the delicate divide between them is in the focus of massively parallelized tensor network state (TNS) algorithms designed for high performance computing (HPC). In this contribution, we present novel algorithmic solutions together with implementation details to extend current limits of TNS algorithms on HPC infrastructure building on state-of-the-art hardware and software technologies. Benchmark results obtained via large-scale density matrix renormalization group (DMRG) simulations on single node multiGPU NVIDIA A100 system are presented for selected strongly correlated molecular systems addressing problems on Hilbert space dimensions up to 4.17 × 10 35 .

arXiv:2305.05581•Journal of Chemical Theory and Computation•Feb 4, 2025•10 citations

Tensor Network State Algorithms on AI Accelerators

Andor Menczer, Örs Legeza

High Resolution Image Download MS PowerPoint Slide We introduce novel algorithmic solutions for hybrid CPU-multiGPU tensor network state algorithms utilizing non-Abelian symmetries building on AI-motivated state-of-the-art hardware and software technologies. The presented numerical simulations on the FeMo cofactor, which plays a crucial role in converting atmospheric nitrogen to ammonia, are far beyond the scope of traditional approaches. Our large-scale SU (2) spin adapted density matrix renormalization group calculations up to bond dimension D = 2 16 on complete active space (CAS) size of 18 electrons in 18 orbitals [CAS(18, 18)] demonstrate that the current limit of exact solution, i.e. full-CI limit, can be achieved in fraction of time. Furthermore, benchmarks up to CAS(113, 76) demonstrate the utilization of NVIDIA’s highly specialized AI accelerators via NVIDIA Tensor Cores, leading to performance around 115 TFLOPS on a single node supplied with eight NVIDIA A100 devices. As a consequence of reaching 71% of the full capacity of the hardware, the cubic scaling of computational time with bond dimension can be reduced to a linear form for a broad range of D values; thus, breaking the current computational limits of small CAS spaces in ab initio quantum chemistry and material science is becoming a reality. In comparison to strict U (1) implementations with matching accuracy, our solution has an estimated effective performance of 300–500 TFLOPS, which emphasizes the mutual need for both algorithmic and technological developments to push current frontiers on classical computation.

Journal of Chemical Theory and Computation•Oct 14, 2024•10 citations

Vectorized Symmetric and Fermionic Tensor Network Implementations for GPU-Accelerated Variational Monte Carlo

Si-Jing Du, Johnnie Gray, Ao Chen +1 more

Variational Monte Carlo (VMC) calculations based on tensor networks (TN) have recently achieved competitive accuracy in ground-state calculations of strongly correlated spin and fermionic systems. However, existing tensor network VMC (TN-VMC) algorithms have not been formulated in a manner that can fully utilize GPU acceleration. We tackle the key missing ingredient, namely, the vectorized evaluation of tensor network amplitudes and tensor network operations. This ensures high GPU utilization by batching over computations with identical structure. In particular, we show how to achieve vectorization for the practically relevant case of abelian symmetric tensor networks (which includes fermionic tensor networks) by developing a ``flat'' tensor network formalism for block-sparse tensor representation and contraction. Using this, we construct a GPU-adapted symmetric TN-VMC workflow with batched tensor network computation. In the two-dimensional Fermi--Hubbard model, we demonstrate a GPU speedup of up to $300 \times$ over single core CPU implementations, for fermionic TN variational wavefunctions.

physics.comp-phCross-listedarXiv:2608.27861•Aug 31, 2026•0 citations

Two-dimensional quantum lattice models via mode optimized hybrid CPU-GPU density matrix renormalization group method

Andor Menczer, Kornél Kapás, Miklós Antal Werner +1 more

We present a hybrid numerical approach to simulate quantum many-body problems on two spatial dimensional quantum lattice models via the non-Abelian ab initio version of the density matrix renormalization group method on state-of-the-art high-performance computing infrastructures. We demonstrate that for the two-dimensional spinless fermion model and for the Hubbard model on torus geometry, altogether several orders of magnitude in computational time can be saved by performing calculations on an optimized basis and by utilizing hybrid CPU-multiGPU parallelization. At least an order of magnitude reduction in computational complexity results from mode optimization, while a further order of reduction in wall time is achieved by massive parallelization. Our results are measured directly in the number of floating point operations and seconds. A detailed scaling analysis of the obtained performance as a function of matrix ranks and as a function of system size up to $12\ifmmode\times\else\texttimes\fi{}12$ lattice topology is discussed. Our CPU-multiGPU model also tremendously accelerates the calculation of the one- and two-particle reduced density matrices, which can be used to construct various order parameters and trace quantum phase transitions with high fidelity.

arXiv:2311.14106•Physical review. B./Physical review. B•May 14, 2024•11 citations

Cost optimized ab initio tensor network state methods: industrial perspectives

Andor Menczer, Örs Legeza

We introduce efficient solutions to optimize the cost of tree-like tensor network state method calculations when an expensive GPU-accelerated hardware is utilized. By supporting a main powerful compute node with additional auxiliary, but much cheaper nodes to store intermediate, precontracted tensor network scratch data, the IO time can be hidden behind the computation almost entirely without increasing memory peak. Our solution is based on the different bandwidths of the different communication channels, like NVLink, PCIe, InfiniBand and available storage media, which are utilized on different layers of the algorithm. This simple heterogeneous multiNode solution via asynchronous IO operation has the potential to minimize IO overhead, resulting in maximum performance rate for the main compute unit. In addition, we introduce an in-house developed massively parallel protocol to serialize and deserialize block sparse matrices and tensors, reducing data communication time tremendously. Performance profiles are presented for the spin adapted ab initio density matrix renormalization group method for corresponding U(1) bond dimension values up to 15400 on the active compounds of the FeMoco with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)].

arXiv:2412.04676•Dec 6, 2024•1 citation

A Fully GPU-Accelerated Framework for High-Performance Configuration Interaction Selection with Neural Network Quantum States

Daran Sun, Bowen Kan, Haoquan Long +13 more

AI-driven methods have demonstrated considerable success in tackling the central challenge of accurately solving the Schrödinger equation for complex many-body systems. Among neural network quantum state (NNQS) approaches, the NNQS-SCI (Selected Configuration Interaction) method stands out as a state-of-the-art technique, recognized for its high accuracy and scalability. However, its application to larger systems is severely constrained by a hybrid CPU-GPU architecture. Specifically, centralized CPU-based global de-duplication creates a severe scalability barrier due to communication bottlenecks, while host-resident coupled-configuration generation induces prohibitive computational overheads. We introduce QiankunNet-cuSCI, a fully GPU-accelerated SCI framework designed to overcome these bottlenecks. It first integrates a distributed, load-balanced global de-duplication algorithm to minimize redundancy and communication overhead at scale. To address compute limitations, it employs specialized, fine-grained CUDA kernels for exact coupled configuration generation. Finally, to break the single-GPU memory barrier exposed by this full acceleration, it incorporates a GPU memory-centric runtime featuring GPU-side pooling, streaming mini-batches, and overlapped offloading. This design enables much larger configuration spaces and shifts the bottleneck from host-side limitations back to on-device inference. Our evaluation demonstrates that our work fundamentally expands the scale of solvable problems. On an NVIDIA A100 cluster with 64 GPUs, our work achieves up to 2.32 × end-to-end speedup over the highly-optimized NNQS-SCI baseline while preserving the same chemical accuracy. Furthermore, it demonstrates excellent distributed performance, maintaining over 90% parallel efficiency in strong scaling tests.

ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC)•Jul 11, 2026•0 citations
We present novel algorithmic solutions together with implementation details utilizing non-Abelian symmetries in order to boost the current limits of tensor network state algorithms on high performance computing infrastructure. In our in-house developed hybrid CPU-multiGPU solution scheduling is decentralized, threads are autonomous and inter-thread communications are solely limited to interactions with globally visible lock-free constructs. Our custom tailored virtual memory management ensures data is produced with high spatial locality, which together with the use of specific sequences of strided batched matrix operations translates to significantly higher overall throughput. In order to lower IO overhead, an adaptive buffering technique is used to dynamically match the level of data abstraction, at which cache repositories are built and reused, to system resources. The non-Abelian symmetry related tensor algebra based on Wigner-Eckhart theorem is fully detached from the conventional tensor network layer, thus massively parallel matrix and tensor operations can be performed without additional overheads. Altogether, we have achieved an order of magnitude increase in performance with respect to results reported in arXiv:2305.05581 in terms of computational complexity and at the same time a factor of three to six in the actual performance measured in TFLOPS. Benchmark results are presented on Hilbert space dimensions up to 2.88×10362.88\times10^{36}2.88×1036 obtained via large-scale SU(2) spin adapted density matrix renormalization group simulations on selected strongly correlated molecular systems. These demonstrate the utilization of NVIDIA's highly specialized tensor cores, leading to performance around 110 TFLOPS on a single node supplied with eight NVIDIA A100 devices. In comparison to U(1) implementations with matching accuracy, our solution has an estimated effective performance of 250-500 TFLOPS.
High Resolution Image Download MS PowerPoint Slide The interplay of quantum and classical simulation and the delicate divide between them is in the focus of massively parallelized tensor network state (TNS) algorithms designed for high performance computing (HPC). In this contribution, we present novel algorithmic solutions together with implementation details to extend current limits of TNS algorithms on HPC infrastructure building on state-of-the-art hardware and software technologies. Benchmark results obtained via large-scale density matrix renormalization group (DMRG) simulations on single node multiGPU NVIDIA A100 system are presented for selected strongly correlated molecular systems addressing problems on Hilbert space dimensions up to 4.17 × 10 35 .
High Resolution Image Download MS PowerPoint Slide We introduce novel algorithmic solutions for hybrid CPU-multiGPU tensor network state algorithms utilizing non-Abelian symmetries building on AI-motivated state-of-the-art hardware and software technologies. The presented numerical simulations on the FeMo cofactor, which plays a crucial role in converting atmospheric nitrogen to ammonia, are far beyond the scope of traditional approaches. Our large-scale SU (2) spin adapted density matrix renormalization group calculations up to bond dimension D = 2 16 on complete active space (CAS) size of 18 electrons in 18 orbitals [CAS(18, 18)] demonstrate that the current limit of exact solution, i.e. full-CI limit, can be achieved in fraction of time. Furthermore, benchmarks up to CAS(113, 76) demonstrate the utilization of NVIDIA’s highly specialized AI accelerators via NVIDIA Tensor Cores, leading to performance around 115 TFLOPS on a single node supplied with eight NVIDIA A100 devices. As a consequence of reaching 71% of the full capacity of the hardware, the cubic scaling of computational time with bond dimension can be reduced to a linear form for a broad range of D values; thus, breaking the current computational limits of small CAS spaces in ab initio quantum chemistry and material science is becoming a reality. In comparison to strict U (1) implementations with matching accuracy, our solution has an estimated effective performance of 300–500 TFLOPS, which emphasizes the mutual need for both algorithmic and technological developments to push current frontiers on classical computation.
Variational Monte Carlo (VMC) calculations based on tensor networks (TN) have recently achieved competitive accuracy in ground-state calculations of strongly correlated spin and fermionic systems. However, existing tensor network VMC (TN-VMC) algorithms have not been formulated in a manner that can fully utilize GPU acceleration. We tackle the key missing ingredient, namely, the vectorized evaluation of tensor network amplitudes and tensor network operations. This ensures high GPU utilization by batching over computations with identical structure. In particular, we show how to achieve vectorization for the practically relevant case of abelian symmetric tensor networks (which includes fermionic tensor networks) by developing a ``flat'' tensor network formalism for block-sparse tensor representation and contraction. Using this, we construct a GPU-adapted symmetric TN-VMC workflow with batched tensor network computation. In the two-dimensional Fermi--Hubbard model, we demonstrate a GPU speedup of up to 300×300 \times300× over single core CPU implementations, for fermionic TN variational wavefunctions.
We present a hybrid numerical approach to simulate quantum many-body problems on two spatial dimensional quantum lattice models via the non-Abelian ab initio version of the density matrix renormalization group method on state-of-the-art high-performance computing infrastructures. We demonstrate that for the two-dimensional spinless fermion model and for the Hubbard model on torus geometry, altogether several orders of magnitude in computational time can be saved by performing calculations on an optimized basis and by utilizing hybrid CPU-multiGPU parallelization. At least an order of magnitude reduction in computational complexity results from mode optimization, while a further order of reduction in wall time is achieved by massive parallelization. Our results are measured directly in the number of floating point operations and seconds. A detailed scaling analysis of the obtained performance as a function of matrix ranks and as a function of system size up to $12\ifmmode\times\else\texttimes\fi{}12$ lattice topology is discussed. Our CPU-multiGPU model also tremendously accelerates the calculation of the one- and two-particle reduced density matrices, which can be used to construct various order parameters and trace quantum phase transitions with high fidelity.
We introduce efficient solutions to optimize the cost of tree-like tensor network state method calculations when an expensive GPU-accelerated hardware is utilized. By supporting a main powerful compute node with additional auxiliary, but much cheaper nodes to store intermediate, precontracted tensor network scratch data, the IO time can be hidden behind the computation almost entirely without increasing memory peak. Our solution is based on the different bandwidths of the different communication channels, like NVLink, PCIe, InfiniBand and available storage media, which are utilized on different layers of the algorithm. This simple heterogeneous multiNode solution via asynchronous IO operation has the potential to minimize IO overhead, resulting in maximum performance rate for the main compute unit. In addition, we introduce an in-house developed massively parallel protocol to serialize and deserialize block sparse matrices and tensors, reducing data communication time tremendously. Performance profiles are presented for the spin adapted ab initio density matrix renormalization group method for corresponding U(1) bond dimension values up to 15400 on the active compounds of the FeMoco with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)].
AI-driven methods have demonstrated considerable success in tackling the central challenge of accurately solving the Schrödinger equation for complex many-body systems. Among neural network quantum state (NNQS) approaches, the NNQS-SCI (Selected Configuration Interaction) method stands out as a state-of-the-art technique, recognized for its high accuracy and scalability. However, its application to larger systems is severely constrained by a hybrid CPU-GPU architecture. Specifically, centralized CPU-based global de-duplication creates a severe scalability barrier due to communication bottlenecks, while host-resident coupled-configuration generation induces prohibitive computational overheads. We introduce QiankunNet-cuSCI, a fully GPU-accelerated SCI framework designed to overcome these bottlenecks. It first integrates a distributed, load-balanced global de-duplication algorithm to minimize redundancy and communication overhead at scale. To address compute limitations, it employs specialized, fine-grained CUDA kernels for exact coupled configuration generation. Finally, to break the single-GPU memory barrier exposed by this full acceleration, it incorporates a GPU memory-centric runtime featuring GPU-side pooling, streaming mini-batches, and overlapped offloading. This design enables much larger configuration spaces and shifts the bottleneck from host-side limitations back to on-device inference. Our evaluation demonstrates that our work fundamentally expands the scale of solvable problems. On an NVIDIA A100 cluster with 64 GPUs, our work achieves up to 2.32 × end-to-end speedup over the highly-optimized NNQS-SCI baseline while preserving the same chemical accuracy. Furthermore, it demonstrates excellent distributed performance, maintaining over 90% parallel efficiency in strong scaling tests.