Many-body quantum chemistry on graphics processing units

A. Eugene DePrince, Jeff R. Hammond, Stephen K. Gray

Published Jan 1, 2011• 1 citation

Abstract

Heterogeneous nodes composed of a multicore CPU and at least one graphics processing unit (GPU) are increasingly common in high-performance scientific computing, and significant programming effort is currently being undertaken to port existing scientific algorithms to these unique architectures. We present implementations for two many-body quantum chemistry methods on heterogeneous nodes: the coupled-cluster with single and double excitations (CCSD) and time-dependent configuration interaction with single and double excitations (TD-CISD) methods. Both methods can be implemented on a computer as a series of dense matrix-matrix multiplications, operations that GPUs are particularly adept at performing. The GPU-accelerated CCSD algorithm is as much as 4.3 times faster than the corresponding CPU algorithm and 9.7 times faster than the algorithm in the Molpro package. The TD-CISD algorithm is accelerated by as much as a factor of 3.9 by GPUs. Enhanced performance is achieved by overlapping CPU and GPU computations.

Related papers

Coupled Cluster Theory on Graphics Processing Units I. The Coupled Cluster Doubles Method

A. Eugene DePrince, Jeff R. Hammond

The coupled cluster (CC) ansatz is generally recognized as providing one of the best wave function-based descriptions of electronic correlation in small- and medium-sized molecules. The fact that the CC equations with double excitations (CCD) may be expressed as a handful of dense matrix-matrix multiplications makes it an ideal method to be ported to graphics processing units (GPUs). We present our implementation of the spin-free CCD equations in which the entire iterative procedure is evaluated on the GPU. The GPU-accelerated algorithm readily achieves a factor of 4-5 speedup relative to the multithreaded CPU algorithm on same-generation hardware. The GPU-accelerated algorithm is approximately 8-12 times faster than Molpro, 17-22 times faster than NWChem, and 21-29 times faster than GAMESS for each CC iteration. Single-precision GPU-accelerated computations are also performed, leading to an additional doubling of performance. Single-precision errors in the energy are typically on the order of 10(-6) hartrees and can be improved by about an order of magnitude by performing one additional iteration in double precision.

Journal of Chemical Theory and ComputationApr 15, 2011118 citations

Quantum Chemical Many-Body Theory on Heterogeneous Nodes

A. Eugene DePrince, Jeff R. Hammond

The iterative solution of the coupled-cluster with single and double excitations (CCSD) equations is a very time-consuming component of the ``gold standard'' in quantum chemistry, the CCSD(T) method. In an effort to accelerate accurate quantum mechanical calculations, we explore two implementation strategies for the iterative solution of the CC equations on graphics procesing units (GPUs). We consider a communication-avoiding algorithm for the spin-free coupled cluster doubles (CCD) equations followed by a low-storage algorithm for the spin-free CCSD equations. In the communication-avoiding algorithm, the entire iterative procedure for the CCD method is performed on the GPU, resulting in accelerations of a factor of 4-5 relative to the pure CPU algorithm. The low-storage CCSD algorithm requires that a minimum of 4o2v2+2ov elements be stored on the device, where o and v represent the number of orbitals occupied and unoccupied in the reference configuration, respectively. The algorithm masks the transfer time for copying large amounts of data to the GPU by overlapping GPU and CPU computations. The per-iteration costs of this hybrid GPU/CPU algorithm are up to 4.06 times less than those of the pure CPU algorithm and up to 10.63 times less than those of the CCSD implementation found in the Molpro electronic structure package. These results provide insight into how to organize communication and computation as to maximize utilization of a GPU and multicore CPU at the same time.

Symposium on Application Accelerators in High-Performance Computing (SAAHPC)Jul 1, 20115 citations

Accelerating Coupled-Cluster Calculations with GPUs: An Implementation of the Density-Fitted CCSD(T) Approach for Heterogeneous Computing Architectures Using OpenMP Directives

Dipayan Datta, Mark S. Gordon

An algorithm is presented for the coupled-cluster singles, doubles, and perturbative triples correction [CCSD(T)] method based on the density fitting or the resolution-of-the-identity (RI) approximation for performing calculations on heterogeneous computing platforms composed of multicore CPUs and graphics processing units (GPUs). The directive-based approach to GPU offloading offered by the OpenMP application programming interface has been employed to adapt the most compute-intensive terms in the RI-CCSD amplitude equations with computational costs scaling as O ( N O 2 N V 4 ), O ( N O 3 N V 3 ), and O ( N O 4 N V 2 ) (where N O and N V denote the numbers of correlated occupied and virtual orbitals, respectively) and the perturbative triples correction to execute on GPU architectures. The pertinent tensor contractions are performed using an accelerated math library such as cuBLAS or hipBLAS. Optimal strategies are discussed for splitting large data arrays into tiles to fit them into the relatively small memory space of the GPUs, while also minimizing the low-bandwidth CPU–GPU data transfers. The performance of the hybrid CPU–GPU RI-CCSD(T) code is demonstrated on pre-exascale supercomputers composed of heterogeneous nodes equipped with NVIDIA Tesla V100 and A100 GPUs and on the world’s first exascale supercomputer named “Frontier”, the nodes of which consist of AMD MI250X GPUs. Speedups within the range 4–8× relative to the recently reported CPU-only algorithm are obtained for the GPU-offloaded terms in the RI-CCSD amplitude equations. Applications to polycyclic aromatic hydrocarbons containing 16–66 carbon atoms demonstrate that the acceleration of the hybrid CPU–GPU code for the perturbative triples correction relative to the CPU-only code increases with the molecule size, attaining a speedup of 5.7× for the largest circumovalene molecule (C 66 H 20 ). The GPU-offloaded code enables the computation of the perturbative triples correction for the C 60 molecule using the cc-pVDZ/aug-cc-pVTZ-RI basis sets in 7 min on Frontier when using 12,288 AMD GPUs with a parallel efficiency of 83.1%.

Journal of Chemical Theory and ComputationOct 25, 202319 citations

Breaking bonds on graphical processing units: Scalable implementations of the density-fitted CR-CC(2,3) and ΛCCSD(T) approaches

Dipayan Datta, Mark S. Gordon

Efficient scalable implementations are presented for two single-reference coupled-cluster (CC) methods with non-iterative corrections for triple excitations, which accurately describe single bond dissociations in molecules in contrast to the standard CCSD(T) approach. These methods are (a) the completely renormalized coupled-cluster theory, CR-CC(2,3), and (b) the ΛCCSD(T) approach, the triples energy corrections in both of which are formulated using the left eigenvector of the CC similarity transformed Hamiltonian. The density fitting approximation is employed for factorizing the two-electron repulsion integrals (ERIs), which eliminates memory bottlenecks associated with the storage of the four-index ERIs by allowing an integral-direct algorithm. Massively parallel algorithms have been developed for the above methods within the GAMESS quantum chemistry program using a hybrid MPI and OpenMP based parallelization model. All compute-intensive steps in the CCSD amplitude and the left eigenvector equations, as well as the triples corrections in both methods, have been adapted to execute on graphical processing units (GPUs) using the OpenMP target directives. Performance of these codes is demonstrated on petascale supercomputers equipped with NVIDIA A100 GPUs. While the CR-CC(2,3) and ΛCCSD(T) computations take roughly twice as much time as the standard CCSD(T) method, the GPU-offloaded codes are shown to impart 3.5-5× accelerations relative to the CPU-only parallel codes, thereby significantly reducing the time-to-solution for the CR-CC(2,3) and ΛCCSD(T) methods.

The Journal of Chemical PhysicsSep 22, 20260 citations

Accelerating CCSD(T) on Graphical Processing Units (GPUs)

O. Jonathan Fajen, Joseph Kelly, Edward G. Hohenstein +1 more

Coupled cluster with singles, doubles, and perturbative triples (CCSD(T)) often provides ground state correlation energies within ″chemical accuracy", but it suffers from high computational cost and steep scaling with system size. We present a GPU-accelerated implementation of CCSD(T) in the TeraChem software package. The new implementation achieves state-of-the-art performance, enabling the calculation of the (T) correction for a system with 63 atoms and more than 1000 basis functions in a little under 8 h on a single node. Additionally, we demonstrate the utility of our optimized implementation for the rapid calculation of full CCSD(T)/CBS stacking energies for all ten unique DNA base pair stacked tetramers. We expect that the TeraChem CCSD(T) implementation will enable the rapid calculation of high-level data that was not previously accessible in a reasonable time frame.

arXiv:2512.01055The Journal of Physical Chemistry AFeb 26, 20260 citations

One programming model to rule them all: High-performance computational chemistry and applications via OpenMP offloading mode

Melisa Alkan

This dissertation is focused around enabling large-scale quantum chemistry calculations by developing novel algorithms on new heterogeneous hardware architectures. Section 2 contains a study on assessing the importance of many-body dispersion phenomena in molecular clusters and understanding the need to incorporate the phenomena into existing quantum chemistry methods to accurately capture long-range interactions. It is known that highly accurate quantum chemistry methods are computationally expensive; therefore, there is a need for new computer algorithms that will be able to vastly improve the time to solution and enable correlated quantum chemistry calculations on large molecular systems. Sections 3 and 4 focus on the development and implementation of various algorithms that enable quantum chemistry calculations on molecules of unprecedent sizes. Specifically, Section 3 describes a novel CUDA-based algorithm that enables GPU-acceleration of the main parts of the Self-Consistent Field (SCF) cycle, resulting in the fastest SCF code to date. Section 4 introduces a novel OpenMP-based implementation of the Fock build formation, which enables GPU-accelerated calculations on all existing GPU and CPU hardware. Section 5 expands upon the OpenMP work and describes a new GPU-accelerated library named LibERI for evaluating molecular integrals – often the main computational bottleneck in many quantum chemistry methods. LibERI is intended to serve as a library for fast quantum chemistry calculations that can be interfaced with any existing quantum chemistry software packages through the MolSSI Driver Interface (MDI). MDI is a standard interface between scientific computational codes. One of LibERI’s objectives is to seamlessly integrate with a Molecular Mechanics (MM) package using MDI to enable fast QM/MM calculations. This integration would allow for efficient GPU-accelerated treatment of large QM regions via LibERI. Lastly, Section 6 concludes this dissertation with a study that applies existing quantum chemistry methods to unravel the mechanism of a Pd-catalyzed carboacylation reaction involving the activation of ester C-O bonds.

Jan 1, 20230 citations