We present a brief survey on the modern tensor numerical methods for multidimensional stationary and time-dependent partial differential equations (PDEs). The guiding principle of the tensor approach is the rank-structured separable approximation of multivariate functions and operators represented on a grid. Recently, the traditional Tucker, canonical, and matrix product states (tensor train) tensor models have been applied to the grid-based electronic structure calculations, to parametric PDEs, and to dynamical equations arising in scientific computing. The essential progress is based on the quantics tensor approximation method proved to be capable to represent (approximate) function related d-dimensional data arrays of size Nd with log-volume complexity, O(dlog N). Combined with the traditional numerical schemes, these novel tools establish a new promising approach for solving multidimensional integral and differential equations using low-parametric rank-structured tensor formats. As the main example, we describe the grid-based tensor numerical approach for solving the 3D nonlinear Hartree-Fock eigenvalue problem, that was the starting point for the developments of tensor-structured numerical methods for large-scale computations in solving real-life multidimensional problems. We also discuss a new method for the fast 3D lattice summation of electrostatic potentials by assembled low-rank tensor approximation capable to treat the potential sum over millions of atoms in few seconds. We address new results on tensor approximation of the dynamical Fokker-Planck and master equations in many dimensions up to d = 20. Numerical tests demonstrate the benefits of the rank-structured tensor approximation on the aforementioned examples of multidimensional PDEs. In particular, the use of grid-based tensor representations in the reduced basis of atomics orbitals yields an accurate solution of the Hartree-Fock equation on large N × N × N grids with a grid size of up to N = 105.
We present a brief survey on the modern tensor numerical methods for multidimensional stationary and time-dependent partial differential equations (PDEs). The guiding principle of the tensor approach is the rank-structured separable approximation of multivariate functions and operators represented on a grid. Recently, the traditional Tucker, canonical, and matrix product states (tensor train) tensor models have been applied to the grid-based electronic structure calculations, to parametric PDEs, and to dynamical equations arising in scientific computing. The essential progress is based on the quantics tensor approximation method proved to be capable to represent (approximate) function related d-dimensional data arrays of size Nd with log-volume complexity, O(dlog N). Combined with the traditional numerical schemes, these novel tools establish a new promising approach for solving multidimensional integral and differential equations using low-parametric rank-structured tensor formats. As the main example, we describe the grid-based tensor numerical approach for solving the 3D nonlinear Hartree-Fock eigenvalue problem, that was the starting point for the developments of tensor-structured numerical methods for large-scale computations in solving real-life multidimensional problems. We also discuss a new method for the fast 3D lattice summation of electrostatic potentials by assembled low-rank tensor approximation capable to treat the potential sum over millions of atoms in few seconds. We address new results on tensor approximation of the dynamical Fokker-Planck and master equations in many dimensions up to d = 20. Numerical tests demonstrate the benefits of the rank-structured tensor approximation on the aforementioned examples of multidimensional PDEs. In particular, the use of grid-based tensor representations in the reduced basis of atomics orbitals yields an accurate solution of the Hartree-Fock equation on large N × N × N grids with a grid size of up to N = 105.
The conventional numerical methods when applied to multidimensional problems suffer from the so-called "curse of dimensionality", that cannot be eliminated by using parallel architectures and high performance computing. The novel tensor numerical methods are based on a "smart" rank-structured tensor representation of the multivariate functions and operators discretized on Cartesian grids thus reducing solution of the multidimensional integral-differential equations to 1D calculations. We explain basic tensor formats and algorithms and show how the orthogonal Tucker tensor decomposition originating from chemometrics made a revolution in numerical analysis, relying on rigorous results from approximation theory. Benefits of tensor approach are demonstrated in ab-initio electronic structure calculations. Computation of the 3D convolution integrals for functions with multiple singularities is replaced by a sequence of 1D operations, thus enabling accurate MATLAB calculations on a laptop using 3D uniform tensor grids of the size up to 10 15 . Fast tensor-based Hartree-Fock solver, incorporating the grid-based low-rank factorization of the two-electron integrals, serves as a prerequisite for economical calculation of the excitation energies of molecules. Tensor approach suggests efficient grid-based numerical treatment of the long-range electrostatic potentials on large 3D finite lattices with defects.The novel range-separated tensor format applies to interaction potentials of multi-particle systems of general type opening the new prospects for tensor methods in scientific computing. This research monograph presenting the modern tensor techniques applied to problems in quantum chemistry may be interesting for a wide audience of students and scientists working in computational chemistry, material science and scientific computing.
We resume the recent successes of the grid-based tensor numerical methods and discuss their prospects in real-space electronic structure calculations. These methods, based on the low-rank representation of the multidimensional functions and integral operators, first appeared as an accurate tensor calculus for the 3D Hartree potential using 1D complexity operations, and have evolved to entirely grid-based tensor-structured 3D Hartree-Fock eigenvalue solver. It benefits from tensor calculation of the core Hamiltonian and two-electron integrals (TEI) in O(n log n) complexity using the rank-structured approximation of basis functions, electron densities and convolution integral operators all represented on 3D n × n × n Cartesian grids. The algorithm for calculating TEI tensor in a form of the Cholesky decomposition is based on multiple factorizations using algebraic 1D "density fitting" scheme, which yield an almost irreducible number of product basis functions involved in the 3D convolution integrals, depending on a threshold ε > 0. The basis functions are not restricted to separable Gaussians, since the analytical integration is substituted by high-precision tensor-structured numerical quadratures. The tensor approaches to post-Hartree-Fock calculations for the MP2 energy correction and for the Bethe-Salpeter excitation energies, based on using low-rank factorizations and the reduced basis method, were recently introduced. Another direction is towards the tensor-based Hartree-Fock numerical scheme for finite lattices, where one of the numerical challenges is the summation of electrostatic potentials of a large number of nuclei. The 3D grid-based tensor method for calculation of a potential sum on a L × L × L lattice manifests the linear in L computational work, O(L), instead of the usual O(L(3) log L) scaling by the Ewald-type approaches.
Quantification of stochastic or quantum systems by a joint probability density or wave function is a notoriously difficult computational problem, since the solution depends on all possible states (or realizations) of the system. Due to this combinatorial flavor, even a system containing as few as ten particles may yield as many as 1010 discretized states. None of even modern supercomputers are capable to cope with this curse of dimensionality straightforwardly, when the amount of quantum particles, for example, grows up to more or less interesting order of hundreds. A traditional approach for a long time was to avoid models formulated in terms of probabilistic functions, and simulate particular system realizations in a randomized process. Since different times in different communities, data-sparse methods came into play. Generally, they aim to define all data points indirectly, by a map from a low amount of representers, and recast all operations (e.g. linear system solution) from the initial data to the effective parameters. The most advanced techniques can be applied (at least, tried) to any given array, and do not rely explicitly on its origin. The current work contributes further progress to this area in the particular direction: tensor product methods for separation of variables. The separation of variables has a long history, and is based on the following elementary concept: a function of many variables may be expanded as a product of univariate functions. On the discrete level, a function is encoded by an array of its values, or a tensor. Therefore, instead of a huge initial array, the separation of variables allows to work with univariate factors with much less efforts. The dissertation contains a short overview of existing tensor representations: canonical PARAFAC, Hierarchical Tucker, Tensor Train (TT) formats, as well as the artificial tensorisation, resulting in the Quantized Tensor Train (QTT) approximation method. The contribution of the dissertation consists in both theoretical constructions and practical numerical algorithms for high-dimensional models, illustrated on the examples of the Fokker-Planck and the chemical master equations. Both arise from stochastic dynamical processes in multiconfigurational systems, and govern the evolution of the probability function in time. A special focus is put on time propagation schemes and their properties related to tensor product methods. We show that these applications yield large-scale systems of linear equations, and prove analytical separable representations of the involved functions and operators. We propose a new combined tensor format (QTT-Tucker), which descends from the TT format (hence TT algorithms may be generalized smoothly), but provides complexity reduction by an order of magnitude. We develop a robust iterative solution algorithm, constituting most advantageous properties of the classical iterative methods from numerical analysis and alternating density matrix renormalization group (DMRG) techniques from quantum physics. Numerical experiments confirm that the new method is preferable to DMRG algorithms. It is as fast as the simplest alternating schemes, but as reliable and accurate as the Krylov methods in linear algebra.
Quantum-mechanical calculations based on Kohn–Sham density functional theory (DFT) played a significant role in accurately predicting various aspects of materials behavior over the past decade. The Kohn–Sham approach to DFT reduces the many-body Schrodinger (eigen value) problem of interacting electrons into an equivalent problem of noninteracting electrons in an effective mean field that is governed by electron-density. Despite the reduced computational complexity of Kohn–Sham DFT, large-scale DFT calculations are still computationally very demanding with the resulting computational complexity scaling cubically with number of atoms in a given materials system. Numerical algorithms with reduced scaling behavior which are robust, computationally efficient and scalable on parallel computing architectures are always desirable to enable simulations at larger scales and on more complex systems. Following this line of thought, this study explores the use of tensor structured methods for ab-initio numerical solution of Kohn–Sham equations arising in DFT calculations. Earlier studies on tensor-structured methods have been quite successful in the accurate calculation of Hartree and the nonlocal exchange operators arising in the Hartree–Fock equations. A recent investigation of low-rank Tucker-type decomposition of the electron-density of large aluminum clusters (obtained from the finite-element discretization of orbital free DFT) shows the exponential decay of approximation error with respect to Tucker rank (number of tensor-basis functions in Tucker type representation). The results also indicate a smaller Tucker rank for the accurate representation of the electron density and is only weakly dependent on the system sizes studied. The promising success of tensor-structured techniques in resolving the electronic structure of material systems has enabled us to take a step further. In this study, we propose a systematic way of computing a globally adapted Tucker-type basis for solving the Kohn–Sham DFT problem by using a separable approximation of the Kohn–Sham Hamiltonian. Further, the resulting Kohn–Sham eigenvalue problem is projected into the aforementioned Tucker basis and is solved for ground-state energy using a self-consistent field iteration. The rank of the resulting Tucker representation and the computational complexity of the calculation are examined on representative benchmark examples involving metallic and insulating systems.
Electronic structure calculations have been one of the most successful scientific fields in the past 50 years. Density functional theory (DFT) with its nice balance between accuracy and efficiency has now become the standard technique for many materials research. On the other hand, the size of systems that wavefunction-based methods can handle has also been improved thanks to the recent developments in both computation power and more advanced methodologies. However, these methods are still suffering from the curse of dimensionality. The cubic scaling of a typical DFT calculation restricts the handleable system size of DFT to only a few thousand electrons. To this end, tensor-structured techniques provide a route to constructing a reduced-order algorithm and hence are promising to improve the computational efficiency. In this dissertation work, we aim at exploring the applications of the tensor-structured techniques on different aspects of electronic structure calculations. In the first part of this work, we present a tensor-structured algorithm for efficient large-scale DFT calculations by constructing a Tucker tensor basis that is adapted to the Kohn–Sham Hamiltonian and localized in real-space. The proposed approach uses an additive separable approximation to the Kohn–Sham Hamiltonian and an L1 localization technique to generate the 1-D localized functions that constitute the Tucker tensor basis. Numerical results show that the resulting Tucker tensor basis exhibits exponential convergence in the ground-state energy with increasing Tucker rank. Further, the proposed tensor-structured algorithm demonstrated sub-quadratic scaling with system-size for both systems with and without a gap, and involving many thousands of atoms. This reduced-order scaling has also resulted in the proposed approach outperforming plane-wave DFT implementation for systems beyond 2000 electrons. In the second part of this work, we present an acceleration scheme for the proposed tensor-structured algorithm. In the proposed algorithm, the discrete Kohn-Sham problem is solved using the Chebyshev filtering subspace iteration method that relies on matrix-matrix multiplications of a sparse symmetric Hamiltonian matrix and a dense wavefunction matrix, expressed in the localized Tucker tensor basis. These matrix-matrix multiplication operations, which constitute the most computationally intensive step of the solution procedure, are GPU accelerated providing ~8-fold GPU-CPU speedup for these operations on the largest systems studied. The computational performance of the GPU accelerated code is presented using benchmark studies on aluminum nano-particles and silicon quantum dots with system sizes ranging up to ~7,000 atoms. Finally, we present a computation kernel using tensor-structured techniques to evaluate the one-electron and two-electron integrals, which are the central quantities to construct the Hamiltonian matrix of a full configuration interaction (FCI) calculation, for any molecular orbitals projected on a tensor-structured finite-element mesh. The proposed computation kernel is used to compute the integrals using Hartree-Fock molecular orbitals, which is subsequently used to perform an FCI calculation. In our numerical study, the FCI energy using integrals from the proposed computation kernel is confirmed to be consistent with a standard FCI calculation using the same Hartree-Fock orbitals. The proposed computation kernel provides a useful tool to investigate the behavior of Kohn-Sham orbitals as a basis for FCI calculations in the future, which could be a route to construct a reduced-order basis for an FCI calculation to reduce the computational cost and enable FCI calculation for the previously computationally inaccessible systems.
The simulation and analysis of high-dimensional problems is often infeasible due to the curse of dimensionality. In this thesis, we investigate the potential of tensor decompositions for mitigating this curse when considering systems from several application areas. Using tensor-based solvers, we directly compute numerical solutions of master equations associated with Markov processes on extremely large state spaces. Furthermore, we exploit the tensor-train format to approximate eigenvalues and corresponding eigentensors of linear tensor operators. In order to analyze the dominant dynamics of high- dimensional stochastic processes, we propose several decomposition techniques for highly diverse problems. These include tensor representations for operators based on nearest-neighbor interactions, construction of pseudoinverses for tensor-based reformulations of dimensionality reduction methods, and the approximation of transfer operators of dynamical systems. The results show that the tensor-train format enables us to compute low-rank approximations for various numerical problems as well as to reduce the memory consumption and the computational costs compared to classical approaches significantly. We demonstrate that tensor decompositions are a powerful tool for solving high-dimensional problems from various application areas.