Tensor contractions are ubiquitous in computational chemistry and physics, where tensors generally represent states or operators and contractions express the algebra of these quantities. In this context, the states and operators often preserve physical conservation laws, which are manifested as group symmetries in the tensors. These group symmetries imply that each tensor has block sparsity and can be stored in a reduced form. For nontrivial contractions, the memory footprint and cost are lowered, respectively, by a linear and a quadratic factor in the number of symmetry sectors. State-of-the-art tensor contraction software libraries exploit this opportunity by iterating over blocks or using general block-sparse tensor representations. Both approaches entail overhead in performance and code complexity. With intuition aided by tensor diagrams, we present a technique, irreducible representation alignment, which enables efficient handling of Abelian group symmetries via only dense tensors, by using contraction-specific reduced forms. This technique yields a general algorithm for arbitrary group symmetric contractions, which we implement in Python and apply to a variety of representative contractions from quantum chemistry and tensor network methods. As a consequence of relying on only dense tensor contractions, we can easily make use of efficient batched matrix multiplication via Intel’s MKL and distributed tensor contraction via the Cyclops library, achieving good efficiency and parallel scalability on up to 4096 Knights Landing cores of a supercomputer.
Tensor contractions are ubiquitous in computational chemistry and physics, where tensors generally represent states or operators and contractions express the algebra of these quantities. In this context, the states and operators often preserve physical conservation laws, which are manifested as group symmetries in the tensors. These group symmetries imply that each tensor has block sparsity and can be stored in a reduced form. For nontrivial contractions, the memory footprint and cost are lowered, respectively, by a linear and a quadratic factor in the number of symmetry sectors. State-of-the-art tensor contraction software libraries exploit this opportunity by iterating over blocks or using general block-sparse tensor representations. Both approaches entail overhead in performance and code complexity. With intuition aided by tensor diagrams, we present a technique, irreducible representation alignment, which enables efficient handling of Abelian group symmetries via only dense tensors, by using contraction-specific reduced forms. This technique yields a general algorithm for arbitrary group symmetric contractions, which we implement in Python and apply to a variety of representative contractions from quantum chemistry and tensor network methods. As a consequence of relying on only dense tensor contractions, we can easily make use of efficient batched matrix multiplication via Intel’s MKL and distributed tensor contraction via the Cyclops library, achieving good efficiency and parallel scalability on up to 4096 Knights Landing cores of a supercomputer.
The density matrix renormalization group (DMRG) algorithm is a powerful tool for solving eigenvalue problems to model quantum systems. DMRG relies on tensor contractions and dense linear algebra to compute properties of condensed matter physics systems. However, its efficient parallel implementation is challenging due to limited concurrency, large memory footprint, and tensor sparsity. We mitigate these problems by implementing two new parallel approaches that handle block sparsity arising in DMRG, via Cyclops, a distributed memory tensor contraction library. We benchmark their performance on two physical systems using the Blue Waters and Stampede2 supercomputers. Our DMRG performance is improved by up to 5.9X in runtime and 99X in processing rate over ITensor, at roughly comparable computational resource use. This enables higher accuracy calculations via larger tensors for quantum state approximation. We demonstrate that despite having limited concurrency, DMRG is weakly scalable with the use of efficient parallel tensor contraction mechanisms.
The Density Matrix Renormalization Group (DMRG) algorithm is a powerful tool for solving eigenvalue problems to model quantum systems. DMRG relies on tensor contractions and dense linear algebra to compute properties of condensed matter physics systems. However, its efficient parallel implementation is challenging due to limited concurrency, large memory footprint, and tensor sparsity. We mitigate these problems by implementing two new parallel approaches that handle block sparsity arising in DMRG, via Cyclops, a distributed memory tensor contraction library. We benchmark their performance on two physical systems using the Blue Waters and Stampede2 supercomputers. Our DMRG performance is improved by up to 5.9X in runtime and 99X in processing rate over ITensor, at roughly comparable computational resource use. This enables higher accuracy calculations via larger tensors for quantum state approximation. We demonstrate that despite having limited concurrency, DMRG is weakly scalable with the use of efficient parallel tensor contraction mechanisms.
We have developed a symbolic manipulation program and program generator (tensor contraction engine or TCE) that abstracts and automates the time-consuming, error-prone processes of deriving the working equations of a well-defined model of second-quantized many-electron theories and synthesizing efficient parallel computer programs on the basis of these equations. Provided an ansatz of a many-electron theory model, TCE performs valid contractions of creation and annihilation operators according to Wick's theorem, consolidates identical terms, and reduces the expressions into the form of multiple tensor contractions acted upon by permutation operators. It subsequently determines the binary contraction order for each multiple tensor contraction with the minimal operation and memory cost, factorizes common binary contractions (defines intermediate tensors), and identifies reusable intermediates. The resulting ordered list of binary tensor contractions, additions, and index permutations is translated into an optimized program that is combined with the NWCHEM and UTCHEM computational chemistry software packages. The programs synthesized by TCE take advantage of spin symmetry (within the spin−orbital formalisms), real Abelian point-group symmetry, and index permutation symmetry at every stage of the calculations to minimize the number of arithmetic operations and storage requirement, adjust the peak local memory usage by index-range tiling, and support parallel I/O interfaces and dynamic load balancing for parallel executions. We demonstrate the utility of TCE through automatic derivation and implementation of parallel programs for a range of predictive computational methods configuration-interaction theory (CISD, CISDT, CISDTQ), generalized many-body perturbation theory [MBPT(2), MBPT(3), MBPT(4)], and coupled-cluster theory (LCCD, CCD, LCCSD, CCSD, QCISD, CCSDT, and CCSDTQ), some for the first time and discuss the performance of the implemented programs.
The benefits of exploiting the presence of symmetries in tensor network algorithms have been extensively demonstrated in the context of matrix product states (MPSs). These include the ability to select a specific symmetry sector (e.g., with a given particle number or spin), to ensure the exact preservation of total charge, and to significantly reduce computational costs. Compared to the case of a generic tensor network, the practical implementation of symmetries in the MPS is simplified by the fact that tensors only have three indices (they are trivalent, just as the Clebsch-Gordan coefficients of the symmetry group) and are organized as a one-dimensional array of tensors, without closed loops. Instead, a more complex tensor network, one where tensors have a larger number of indices and/or a more elaborate network structure, requires a more general treatment. In two recent papers, namely, (i) [Singh, Pfeifer, and Vidal, Phys. Rev. A 82, 050301 (2010)] and (ii) [Singh, Pfeifer, and Vidal, Phys. Rev. B 83, 115125 (2011)], we described how to incorporate a global internal symmetry into a generic tensor network algorithm based on decomposing and manipulating tensors that are invariant under the symmetry. In (i) we considered a generic symmetry group G that is compact, completely reducible, and multiplicity free, acting as a global internal symmetry. Then, in (ii) we described the implementation of Abelian group symmetries in much more detail, considering a U(1) symmetry (e.g., conservation of global particle number) as a concrete example. In this paper, we describe the implementation of non-Abelian group symmetries in great detail. For concreteness, we consider an SU(2) symmetry (e.g., conservation of global quantum spin). Our formalism can be readily extended to more exotic symmetries associated with conservation of total fermionic or anyonic charge. As a practical demonstration, we describe the SU(2)-invariant version of the multiscale entanglement renormalization ansatz and apply it to study the low-energy spectrum of a quantum spin chain with a global SU(2) symmetry.
The optimization of tensor expressions with hundreds of terms is required for the development of accurate quantum chemistry models such as the coupled cluster method. In this paper, we address the effective exploitation of symmetry properties of tensors in performing algebraic transformations for minimizing operation count of tensor expressions. We develop rules to detect symmetries in intermediate tensors, cost models for tensor contractions with symmetries, and a canonical representation to facilitate effective common subexpression elimination. We demonstrate significant improvements to the operation counts for the coupled cluster method when compared to several state-of-the-art im-plementations. Furthermore, we show that tensor expressions optimized for a few input parameter combinations can be used to achieve operation counts within 3% of the optimal, for the entire parameter space of interest.