Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Journal of Chemical Theory and Computation

Tensor Network State Algorithms on AI Accelerators

Andor Menczer, Örs Legeza

Published Oct 14, 2024• 10 citations
Publisher

Abstract

High Resolution Image Download MS PowerPoint Slide We introduce novel algorithmic solutions for hybrid CPU-multiGPU tensor network state algorithms utilizing non-Abelian symmetries building on AI-motivated state-of-the-art hardware and software technologies. The presented numerical simulations on the FeMo cofactor, which plays a crucial role in converting atmospheric nitrogen to ammonia, are far beyond the scope of traditional approaches. Our large-scale SU (2) spin adapted density matrix renormalization group calculations up to bond dimension D = 2 16 on complete active space (CAS) size of 18 electrons in 18 orbitals [CAS(18, 18)] demonstrate that the current limit of exact solution, i.e. full-CI limit, can be achieved in fraction of time. Furthermore, benchmarks up to CAS(113, 76) demonstrate the utilization of NVIDIA’s highly specialized AI accelerators via NVIDIA Tensor Cores, leading to performance around 115 TFLOPS on a single node supplied with eight NVIDIA A100 devices. As a consequence of reaching 71% of the full capacity of the hardware, the cubic scaling of computational time with bond dimension can be reduced to a linear form for a broad range of D values; thus, breaking the current computational limits of small CAS spaces in ab initio quantum chemistry and material science is becoming a reality. In comparison to strict U (1) implementations with matching accuracy, our solution has an estimated effective performance of 300–500 TFLOPS, which emphasizes the mutual need for both algorithmic and technological developments to push current frontiers on classical computation.

DOI
10.1021/acs.jctc.4c00800

Related papers

Boosting the effective performance of massively parallel tensor network state algorithms on hybrid CPU-GPU based architectures via non-Abelian symmetries

Andor Menczer, Örs Legeza

We present novel algorithmic solutions together with implementation details utilizing non-Abelian symmetries in order to boost the current limits of tensor network state algorithms on high performance computing infrastructure. In our in-house developed hybrid CPU-multiGPU solution scheduling is decentralized, threads are autonomous and inter-thread communications are solely limited to interactions with globally visible lock-free constructs. Our custom tailored virtual memory management ensures data is produced with high spatial locality, which together with the use of specific sequences of strided batched matrix operations translates to significantly higher overall throughput. In order to lower IO overhead, an adaptive buffering technique is used to dynamically match the level of data abstraction, at which cache repositories are built and reused, to system resources. The non-Abelian symmetry related tensor algebra based on Wigner-Eckhart theorem is fully detached from the conventional tensor network layer, thus massively parallel matrix and tensor operations can be performed without additional overheads. Altogether, we have achieved an order of magnitude increase in performance with respect to results reported in arXiv:2305.05581 in terms of computational complexity and at the same time a factor of three to six in the actual performance measured in TFLOPS. Benchmark results are presented on Hilbert space dimensions up to $2.88\times10^{36}$ obtained via large-scale SU(2) spin adapted density matrix renormalization group simulations on selected strongly correlated molecular systems. These demonstrate the utilization of NVIDIA's highly specialized tensor cores, leading to performance around 110 TFLOPS on a single node supplied with eight NVIDIA A100 devices. In comparison to U(1) implementations with matching accuracy, our solution has an estimated effective performance of 250-500 TFLOPS.

arXiv:2309.16724•Sep 23, 2023•2 citations

Massively Parallel Tensor Network State Algorithms on Hybrid CPU-GPU Based Architectures

Andor Menczer, Örs Legeza

High Resolution Image Download MS PowerPoint Slide The interplay of quantum and classical simulation and the delicate divide between them is in the focus of massively parallelized tensor network state (TNS) algorithms designed for high performance computing (HPC). In this contribution, we present novel algorithmic solutions together with implementation details to extend current limits of TNS algorithms on HPC infrastructure building on state-of-the-art hardware and software technologies. Benchmark results obtained via large-scale density matrix renormalization group (DMRG) simulations on single node multiGPU NVIDIA A100 system are presented for selected strongly correlated molecular systems addressing problems on Hilbert space dimensions up to 4.17 × 10 35 .

arXiv:2305.05581•Journal of Chemical Theory and Computation•Feb 4, 2025•10 citations

Parallel Implementation of the Density Matrix Renormalization Group Method Achieving a Quarter petaFLOPS Performance on a Single DGX-H100 GPU Node

Andor Menczer, Maarten Van Damme, Alan E. Rask +5 more

Density Matrix Renormalization Group (DMRG) method on current state-of-the-art NVIDIA DGX-H100 architectures. We evaluate the performance of the DMRG electronic structure calculations for the active compounds of the FeMoco, the primary cofactor of nitrogenase, and cytochrome P450 (CYP) enzymes with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)] and 63 electrons in 58 orbitals [CAS(63, 58)], respectively. We achieve 246 teraFLOPS of sustained performance, an improvement of more than 2.5× compared to the performance achieved on the DGX-A100 architectures and an 80× acceleration compared to an OpenMP parallelized implementation on a 128-core CPU architecture. Our work highlights the ability of tensor network algorithms to efficiently utilize high-performance multi-GPU hardware and shows that the combination of tensor networks with modern large-scale GPU accelerators can pave the way toward solving some of the most challenging problems in quantum chemistry and beyond.

arXiv:2407.07411•Journal of Chemical Theory and Computation•Sep 19, 2024•34 citations

Cost optimized ab initio tensor network state methods: industrial perspectives

Andor Menczer, Örs Legeza

We introduce efficient solutions to optimize the cost of tree-like tensor network state method calculations when an expensive GPU-accelerated hardware is utilized. By supporting a main powerful compute node with additional auxiliary, but much cheaper nodes to store intermediate, precontracted tensor network scratch data, the IO time can be hidden behind the computation almost entirely without increasing memory peak. Our solution is based on the different bandwidths of the different communication channels, like NVLink, PCIe, InfiniBand and available storage media, which are utilized on different layers of the algorithm. This simple heterogeneous multiNode solution via asynchronous IO operation has the potential to minimize IO overhead, resulting in maximum performance rate for the main compute unit. In addition, we introduce an in-house developed massively parallel protocol to serialize and deserialize block sparse matrices and tensors, reducing data communication time tremendously. Performance profiles are presented for the spin adapted ab initio density matrix renormalization group method for corresponding U(1) bond dimension values up to 15400 on the active compounds of the FeMoco with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)].

arXiv:2412.04676•Dec 6, 2024•1 citation

Distributed Multi-GPU Ab Initio Density Matrix Renormalization Group Algorithm with Applications to the P-Cluster of Nitrogenase

Chunyang Xiang, Weile Jia, Wei‐Hai Fang +1 more

The presence of many degenerate d/f orbitals makes polynuclear transition-metal compounds, such as iron–sulfur clusters in nitrogenase, challenging for state-of-the-art quantum chemistry methods. To address this challenge, we present the first distributed multi-graphics processing unit (GPU) ab initio density matrix renormalization group (DMRG) algorithm suitable for modern high-performance computing (HPC) infrastructures. The central idea is to parallelize the most computationally intensive part─the multiplication of O ( K 2 ) operators with a trial wave function, where K is the number of spatial orbitals, by combining operator parallelism for distributing the workload with a batched algorithm for performing contractions on GPU. With this new implementation, we are able to reach an unprecedentedly large bond dimension D = 14,000 on 48 GPUs (NVIDIA A100 80 GB SXM) for an active space model (114 electrons in 73 active orbitals) of the P-cluster, which is nearly 3 times larger than the bond dimensions reported in previous DMRG calculations for the same system using only central processing units (CPUs).

Journal of Chemical Theory and Computation•Jan 10, 2024•32 citations

A distributed multi-GPU ab initio density matrix renormalization group algorithm with applications to the P-cluster of nitrogenase

Xiang, Chunyang, Weile Jia, Wei‐Hai Fang +1 more

The presence of many degenerate $d/f$ orbitals makes polynuclear transition metal compounds such as iron-sulfur clusters in nitrogenase challenging for state-of-the-art quantum chemistry methods. To address this challenge, we present the first distributed multi-GPU (Graphics Processing Unit) \emph{ab initio} density matrix renormalization (DMRG) algorithm, suitable for modern high-performance computing (HPC) infrastructures. The central idea is to parallelize the most computationally intensive part - the multiplication of $O(K^2)$ operators with a trial wavefunction, where $K$ is the number of spatial orbitals, by combining operator parallelism for distributing the workload with a batched algorithm for performing contractions on GPU. With this new implementation, we are able to reach an unprecedentedly large bond dimension $D=14000$ on 48 GPUs (NVIDIA A100 80 GB SXM) for an active space model (114 electrons in 73 active orbitals) of the P-cluster, which is nearly three times larger than the bond dimensions reported in previous DMRG calculations for the same system using only CPUs.

arXiv:2311.02854•Nov 6, 2023•2 citations
High Resolution Image Download MS PowerPoint Slide We introduce novel algorithmic solutions for hybrid CPU-multiGPU tensor network state algorithms utilizing non-Abelian symmetries building on AI-motivated state-of-the-art hardware and software technologies. The presented numerical simulations on the FeMo cofactor, which plays a crucial role in converting atmospheric nitrogen to ammonia, are far beyond the scope of traditional approaches. Our large-scale SU (2) spin adapted density matrix renormalization group calculations up to bond dimension D = 2 16 on complete active space (CAS) size of 18 electrons in 18 orbitals [CAS(18, 18)] demonstrate that the current limit of exact solution, i.e. full-CI limit, can be achieved in fraction of time. Furthermore, benchmarks up to CAS(113, 76) demonstrate the utilization of NVIDIA’s highly specialized AI accelerators via NVIDIA Tensor Cores, leading to performance around 115 TFLOPS on a single node supplied with eight NVIDIA A100 devices. As a consequence of reaching 71% of the full capacity of the hardware, the cubic scaling of computational time with bond dimension can be reduced to a linear form for a broad range of D values; thus, breaking the current computational limits of small CAS spaces in ab initio quantum chemistry and material science is becoming a reality. In comparison to strict U (1) implementations with matching accuracy, our solution has an estimated effective performance of 300–500 TFLOPS, which emphasizes the mutual need for both algorithmic and technological developments to push current frontiers on classical computation.
We present novel algorithmic solutions together with implementation details utilizing non-Abelian symmetries in order to boost the current limits of tensor network state algorithms on high performance computing infrastructure. In our in-house developed hybrid CPU-multiGPU solution scheduling is decentralized, threads are autonomous and inter-thread communications are solely limited to interactions with globally visible lock-free constructs. Our custom tailored virtual memory management ensures data is produced with high spatial locality, which together with the use of specific sequences of strided batched matrix operations translates to significantly higher overall throughput. In order to lower IO overhead, an adaptive buffering technique is used to dynamically match the level of data abstraction, at which cache repositories are built and reused, to system resources. The non-Abelian symmetry related tensor algebra based on Wigner-Eckhart theorem is fully detached from the conventional tensor network layer, thus massively parallel matrix and tensor operations can be performed without additional overheads. Altogether, we have achieved an order of magnitude increase in performance with respect to results reported in arXiv:2305.05581 in terms of computational complexity and at the same time a factor of three to six in the actual performance measured in TFLOPS. Benchmark results are presented on Hilbert space dimensions up to 2.88×10362.88\times10^{36}2.88×1036 obtained via large-scale SU(2) spin adapted density matrix renormalization group simulations on selected strongly correlated molecular systems. These demonstrate the utilization of NVIDIA's highly specialized tensor cores, leading to performance around 110 TFLOPS on a single node supplied with eight NVIDIA A100 devices. In comparison to U(1) implementations with matching accuracy, our solution has an estimated effective performance of 250-500 TFLOPS.
High Resolution Image Download MS PowerPoint Slide The interplay of quantum and classical simulation and the delicate divide between them is in the focus of massively parallelized tensor network state (TNS) algorithms designed for high performance computing (HPC). In this contribution, we present novel algorithmic solutions together with implementation details to extend current limits of TNS algorithms on HPC infrastructure building on state-of-the-art hardware and software technologies. Benchmark results obtained via large-scale density matrix renormalization group (DMRG) simulations on single node multiGPU NVIDIA A100 system are presented for selected strongly correlated molecular systems addressing problems on Hilbert space dimensions up to 4.17 × 10 35 .
Density Matrix Renormalization Group (DMRG) method on current state-of-the-art NVIDIA DGX-H100 architectures. We evaluate the performance of the DMRG electronic structure calculations for the active compounds of the FeMoco, the primary cofactor of nitrogenase, and cytochrome P450 (CYP) enzymes with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)] and 63 electrons in 58 orbitals [CAS(63, 58)], respectively. We achieve 246 teraFLOPS of sustained performance, an improvement of more than 2.5× compared to the performance achieved on the DGX-A100 architectures and an 80× acceleration compared to an OpenMP parallelized implementation on a 128-core CPU architecture. Our work highlights the ability of tensor network algorithms to efficiently utilize high-performance multi-GPU hardware and shows that the combination of tensor networks with modern large-scale GPU accelerators can pave the way toward solving some of the most challenging problems in quantum chemistry and beyond.
We introduce efficient solutions to optimize the cost of tree-like tensor network state method calculations when an expensive GPU-accelerated hardware is utilized. By supporting a main powerful compute node with additional auxiliary, but much cheaper nodes to store intermediate, precontracted tensor network scratch data, the IO time can be hidden behind the computation almost entirely without increasing memory peak. Our solution is based on the different bandwidths of the different communication channels, like NVLink, PCIe, InfiniBand and available storage media, which are utilized on different layers of the algorithm. This simple heterogeneous multiNode solution via asynchronous IO operation has the potential to minimize IO overhead, resulting in maximum performance rate for the main compute unit. In addition, we introduce an in-house developed massively parallel protocol to serialize and deserialize block sparse matrices and tensors, reducing data communication time tremendously. Performance profiles are presented for the spin adapted ab initio density matrix renormalization group method for corresponding U(1) bond dimension values up to 15400 on the active compounds of the FeMoco with complete active space (CAS) sizes of up to 113 electrons in 76 orbitals [CAS(113, 76)].
The presence of many degenerate d/f orbitals makes polynuclear transition-metal compounds, such as iron–sulfur clusters in nitrogenase, challenging for state-of-the-art quantum chemistry methods. To address this challenge, we present the first distributed multi-graphics processing unit (GPU) ab initio density matrix renormalization group (DMRG) algorithm suitable for modern high-performance computing (HPC) infrastructures. The central idea is to parallelize the most computationally intensive part─the multiplication of O ( K 2 ) operators with a trial wave function, where K is the number of spatial orbitals, by combining operator parallelism for distributing the workload with a batched algorithm for performing contractions on GPU. With this new implementation, we are able to reach an unprecedentedly large bond dimension D = 14,000 on 48 GPUs (NVIDIA A100 80 GB SXM) for an active space model (114 electrons in 73 active orbitals) of the P-cluster, which is nearly 3 times larger than the bond dimensions reported in previous DMRG calculations for the same system using only central processing units (CPUs).
The presence of many degenerate d/fd/fd/f orbitals makes polynuclear transition metal compounds such as iron-sulfur clusters in nitrogenase challenging for state-of-the-art quantum chemistry methods. To address this challenge, we present the first distributed multi-GPU (Graphics Processing Unit) \emph{ab initio} density matrix renormalization (DMRG) algorithm, suitable for modern high-performance computing (HPC) infrastructures. The central idea is to parallelize the most computationally intensive part - the multiplication of O(K2)O(K^2)O(K2) operators with a trial wavefunction, where KKK is the number of spatial orbitals, by combining operator parallelism for distributing the workload with a batched algorithm for performing contractions on GPU. With this new implementation, we are able to reach an unprecedentedly large bond dimension D=14000D=14000D=14000 on 48 GPUs (NVIDIA A100 80 GB SXM) for an active space model (114 electrons in 73 active orbitals) of the P-cluster, which is nearly three times larger than the bond dimensions reported in previous DMRG calculations for the same system using only CPUs.