The Journal of Chemical Physics

Sparse tensor based nuclear gradients for periodic Hartree–Fock and low-scaling correlated wave function methods in the CP2K software package: A massively parallel and GPU accelerated implementation

Augustin Bussy, Ole Schütt, Jürg Hutter

Published Apr 27, 2023• 17 citations

Abstract

The development of novel double-hybrid density functionals offers new levels of accuracy and is leading to fresh insights into the fundamental properties of matter. Hartree-Fock exact exchange and correlated wave function methods, such as second-order Møller-Plesset (MP2) and direct random phase approximation (dRPA), are usually required to build such functionals. Their high computational cost is a concern, and their application to large and periodic systems is, therefore, limited. In this work, low-scaling methods for Hartree-Fock exchange (HFX), SOS-MP2, and direct RPA energy gradients are developed and implemented in the CP2K software package. The use of the resolution-of-the-identity approximation with a short range metric and atom-centered basis functions leads to sparsity, allowing for sparse tensor contractions to take place. These operations are efficiently performed with the newly developed Distributed Block-sparse Tensors (DBT) and Distributed Block-sparse Matrices (DBM) libraries, which scale to hundreds of graphics processing unit (GPU) nodes. The resulting methods, resolution-of-the-identity (RI)-HFX, SOS-MP2, and dRPA, were benchmarked on large supercomputers. They exhibit favorable sub-cubic scaling with system size, good strong scaling performance, and GPU acceleration up to a factor of 3. These developments will allow for double-hybrid level calculations of large and periodic condensed phase systems to take place on a more regular basis.

DOI: 10.1063/5.0144493

Related papers

An Efficient RI-MP2 Algorithm for Distributed Many-GPU Architectures

Calum Snowdon, Giuseppe M. J. Barca

Second-order Møller-Plesset perturbation theory (MP2) using the Resolution of the Identity approximation (RI-MP2) is a widely used method for computing molecular energies beyond the Hartree-Fock mean-field approximation. However, its high computational cost and lack of efficient algorithms for modern supercomputing architectures limit its applicability to large molecules. In this paper, we present the first distributed-memory many-GPU RI-MP2 algorithm explicitly designed to utilize hundreds of GPU accelerators for every step of the computation. Our novel algorithm achieves near-peak performance on GPU-based supercomputers through the development of a distributed memory algorithm for forming RI-MP2 intermediate tensors with zero inter-node communication, except for a single O(N^2) asynchronous broadcast, and a distributed memory algorithm for the O(N^5) energy reduction step, capable of sustaining near-peak performance on clusters with several hundred GPUs. Comparative analysis shows our implementation outperforms state-of-the-art quantum chemistry software by over 3.5 times in speed while achieving an eightfold reduction in computational power consumption. Benchmarking on the Perlmutter supercomputer, our algorithm achieves 11.8 PFLOP/s (83% of peak performance) performing and the RI-MP2 energy calculation on a 314-water cluster with 7,850 primary and 30,144 auxiliary basis functions in 4 minutes on 180 nodes and 720 A100 GPUs. This performance represents a substantial improvement over traditional CPU-based methods, demonstrating significant time-to-solution and power consumption benefits of leveraging modern GPU-accelerated computing environments for quantum chemistry calculations.

Journal of Chemical Theory and ComputationOct 18, 20247 citations

Lower-scaling advanced correlation methods with local Resolution of Identity strategies

Arvid Conrad Ihrig

First-principles methods are a valuable tool to study the electronic structure of systems and explore the quantum-mechanical mechanisms driving them on the microscopic scale. Semi-local Density-Functional Approximations (DFAs) are still very popular due to the good balance between accuracy and computational efficiency. However, the instantaneous electronic interactions are described with an averaged potential in these methods, which is often insufficient to describe charge transfers or subtle many-body correlations. A promising way to improve the description of such interactions is the usage of advanced correlation methods which explicitly treat the electronic interactions in a many-body picture. Well known methods form this class are e.g. the Random- Phase Approximation (RPA), 2nd Order Møller-Plesset Perturbation Theory (MP2) and Coupled-Cluster Theory (CC). Unfortunately, the time and memory requirements for all these methods suffer from a steep scaling with system size in their canonical formulations. Several lower-scaling reformulations exist for these methods, but the at least cubic memory requirements and the increased algorithmic complexity prevented any efficient parallelization of these methods so far. In my thesis I developed a new lower-scaling technique for advanced correlation methods to address this challenge. The key components of this technique are the Laplace-Transformation (LT) and a new local Resolution of Identity (RI) strategy, named RI-LVL. Together, they significantly reduce the memory requirement without sacrificing accuracy and allow for an efficient parallelization by means of à priori known sparsity patterns. In combination with established integral screening techniques, the computational scaling can also be reduced considerably. The potential of this new approach is demonstrated at the example of an in-memory LT-MP2 with a MPI/OpenMP hybrid parallelization. It features an at worst cubic computational scaling, a quadratic memory scaling and a high parallel efficiency. Based on the Numeric Atom-Centered Orbital (NAO) framework used in the Fritz Haber Institute ab initio molecular simulations (FHI-aims) code, this implementation can handle both molecular and periodic systems with the same numerical footing. I will also discuss in detail how the combination of the LT and RI- LVL can be applied to RPA calculations. The accuracy and performance of my LT- MP2 is demonstrated in detail for both molecular and periodic systems. In addition, the convergence of the MP2 level adsorption energy of water on a rutile titanium dioxide (TiO2) surface in the low coverage limit is explored. This new LT-MP2 recovers more than 99.9% of the canonical MP2 correlation energy and can handle TiO2 surfaces with up to 270 atoms in the unit cell.

Jan 1, 20170 citations

A GPU Implementation of the Second- and Third-Order Møller-Plesset Perturbation Theory

Satoki Tsuji, Yasuaki Ito, Nobuya Yokogawa +4 more

Post-Hartree-Fock methods are wavefunction-based quantum chemistry approaches that explicitly account for the electron correlation beyond the mean-field approximation, allowing accurate determination of electron orbitals and energies in a given system. However, their computational cost scales as the fourth power or higher with respect to the number of atoms, severely limiting their practical applications. This paper proposes an efficient GPU implementation of the Møller-Plesset method, a post-Hartree-Fock approach based on perturbation theory. We focus on the computation of second- and thirdorder perturbation energies (MP2 and MP3) for closed-shell systems, which incorporate electron correlation on top of the Hartree-Fock energy. To accelerate the extensive contraction operations arising in the Møller-Plesset method, our implementation employs a formulation that maximizes the use of highly optimized matrix multiplication routines. Furthermore, we effectively utilize computational capabilities of GPU architecture through a hierarchical reduction using warp-level primitives and shared memory, as well as contiguous coalesced access to matrix elements. Performance evaluation on an NVIDIA A100 GPU shows up to 255.5 and 71.1 times speedups for the MP2 correlation energy computation, compared with the widely used quantum chemistry software package PySCF and its GPU plugin GPU4PySCF, respectively. In the MP3 calculations, our DGEMM-based implementation achieves a maximum speedup of 96.2 times over PySCF.

International Symposium on Computing and Networking Workshops (CANDARW)Nov 25, 20251 citation

Communication: A reduced scaling J-engine based reformulation of SOS-MP2 using graphics processing units

Simon A. Maurer, Jörg Kußmann, Christian Ochsenfeld

We present a low-prefactor, cubically scaling scaled-opposite-spin second-order Møller-Plesset perturbation theory (SOS-MP2) method which is highly suitable for massively parallel architectures like graphics processing units (GPU). The scaling is reduced from O(N⁵) to O(N³) by a reformulation of the MP2-expression in the atomic orbital basis via Laplace transformation and the resolution-of-the-identity (RI) approximation of the integrals in combination with efficient sparse algebra for the 3-center integral transformation. In contrast to previous works that employ GPUs for post Hartree-Fock calculations, we do not simply employ GPU-based linear algebra libraries to accelerate the conventional algorithm. Instead, our reformulation allows to replace the rate-determining contraction step with a modified J-engine algorithm, that has been proven to be highly efficient on GPUs. Thus, our SOS-MP2 scheme enables us to treat large molecular systems in an accurate and efficient manner on a single GPU-server.

The Journal of Chemical PhysicsAug 6, 201444 citations

GPU-Accelerated Analytic Coulomb- and Exchange Gradients for Hartree Fock and Density Functional Theory

Vladimir Mironov, Alexander C. Paul, Dmitry I. Morozov +3 more

We present a GPU-accelerated software package for the evaluation of analytic two-electron energy and gradient contributions in Hartree-Fock (HF) and Density Functional theory (DFT) calculations. The implementation is provided as a Python library with a C++ backend, enabling straightforward integration into modern computational chemistry and drug-discovery workflows. The code supports single-point energy and nuclear gradient evaluations on both single- and multi-GPU systems, and employs MPI-based parallelization with dynamic load balancing in multi-node environments. We report comprehensive benchmarks demonstrating favorable scaling with respect to system size, as well as high throughput for batched evaluations relevant to molecular dynamics, geometry optimization, and large-scale virtual screening. Parallel execution of a single system was carried out on up to 24 A100 GPUs. The implementation builds on optimized GPU-enabled variants of the LibintX and GauXC libraries to efficiently compute density-fitted Coulomb, semi-numerical exchange, and exchange--correlation contributions.

Mar 23, 20260 citations

Formulation and implementation of accurate and efficient low-scaling methods for explicitly correlated F12 theory

Lars Urban

One of the major obstacles in modern quantum chemistry is the need to employ exceedingly large one-electron basis sets in electron correlation methods to obtain reliably accurate results. The family of explicitly correlated wave function methods circumvents this issue by introducing terms that explicitly depend on the interelectronic distance, thereby enforcing the correct electron–electron cusp behavior, albeit at the cost of additional complicated many-electron integrals when evaluated without further approximation. The main focus of the present work is the efficient, low-scaling evaluation of explicitly correlated approaches, examined here through the explicitly correlated correction to second-order Møller–Plesset perturbation theory (MP2-F12) as a representative example. While these methods are highly effective in reducing the basis set incompleteness error, the cost associated with the corresponding correction often far exceeds that of the underlying correlation method. To overcome these limitations, several techniques and strategies are introduced to significantly reduce computational demand and resolve the main bottlenecks that arise. A particularly straightforward approach is provided by the scaled MP2-F12 ansatz, which introduces a single empirically determined factor that scales the direct-type geminal–geminal terms to closely reproduce the MP2-F12 correction, thereby omitting the most expensive exchange-type contributions. Furthermore, the computation of Fock matrix elements spanning multiple orbital spaces is drastically accelerated by extending techniques originally developed for Hartree–Fock and density functional theory. The direct contributions are evaluated using a modified resolution-of-the-identity Coulomb (RI-J) approach employing the J-engine ansatz for integral construction, while the exchange contributions are treated via three-dimensional numerical quadrature in the form of the seminumerical sn-LinK method. These approaches reduce the computational cost by more than three orders of magnitude compared to the approximation-free evaluation while maintaining high accuracy, with additional speedups achieved by exploiting the massive parallelism of graphics processing units. The central focus of this work is the efficient decomposition of complicated exchange-type multi-electron integrals, where numerical quadrature plays a key role in improving performance. The complementary auxiliary basis set resolution of the identity (CABS-RI) reduces the most expensive exchange-type integral to a three-electron integral, which is then efficiently decomposed via numerical quadrature. In this context, a highly efficient batch-wise, distance-dependent integral screening exploiting the short-range behavior of F12 operators is introduced, significantly improving performance and enabling near-linear-scaling evaluations while retaining high accuracy. These strategies are extended to all exchange-type contributions in F12 theory, leading to an alternative formulation of integrals better suited for efficient factorization. A novel combination of numerical quadrature with density fitting is presented for the efficient computation of products of four-center two-electron integrals, alongside algorithms covering all exchange-type contributions, including advantageous batching strategies. In general, the formal computational cost with system size M is reduced from O(M⁵) to O(M⁴), with further improvements appearing feasible through the use of localized molecular orbitals combined with advanced integral screening techniques.

Jan 1, 20260 citations