© 2026 Aqora Quantum S.A.S.

Terms Privacy Legal Notice

arXiv:2411.03900 · Machine Learning Science and Technology

Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry

Oliver Knitter, Dan Zhao, JAMES M. STOKES, Martin Ganahl, Stefan Leichenauer, Shravan Veerapaneni

Published Apr 10, 2025• 3 citations

Abstract

Abstract Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansätze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in ab initio quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet’s time complexity outperforms that of the transformer. Though this efficiency comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model—namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS.

DOI: 10.1088/2632-2153/adcb88

Related papers

Exploration of Quantum-Inspired Deep Learning Architectures for Fundamental Applications in Scientific Computing

Oliver Knitter

Neural-network quantum states (NQS) is an exciting area of quantum-inspired machine learning, wherein a fully classical neural network is used to efficiently learn the optimal state vector for a quantum unsupervised learning problem, bypassing the need to model the full Hilbert space, which is exponentially large. In theory, NQS architectures are broadly applicable, and have been demonstrated successfully on several fundamental applications. We seek to further explore the practical utility of NQS as a black box solver for various applications in the scientific computing domain. We first present NQS as an explicit avenue for de-quantizing variational quantum eigensolvers, demonstrating with the variational quantum linear solver (VQLS). We then explore the utility of VQLS and its de-quantization, VNLS, as black box matrix inversion tools for a Newton-based linear complementarity solver, used to model the time evolution of a rudimentary granular medium. Turning our attention toward applications of NQS in quantum chemistry, we introduce a new retentive network (RetNet) autoregressive NQS ansatz to solve electronic ground state problems, and explore the incorporation of neural annealing techniques to these problems as a means of improving accuracy and ease of training. We finish with a rudimentary scaling law analysis of the RetNet architecture and its immediate autoregressive predecessors as applied to electronic structure calculations.

•Jan 1, 2024•0 citations

Parallel Scan Recurrent Neural Quantum States for Scalable Variational Monte Carlo

Ejaaz Merali, Mohamed Hibat-Allah, Mohammad Kohandel +2 more

Neural-network quantum states have emerged as a powerful variational framework for quantum many-body systems, with recent progress often driven by massively parallel architectures such as transformers. Recurrent neural network quantum states, however, are frequently regarded as intrinsically sequential and therefore less scalable. Here we revisit this view by showing that modern recurrent architectures can support fast, accurate, and computationally accessible neural quantum state simulations. Using autoregressive recurrent wave functions together with recent advances in parallelizable recurrence, we develop variational ansätze, called parallel scan recurrent neural quantum states (PSR-NQS), which can be trained efficiently within variational Monte Carlo in one and two spatial dimensions. We demonstrate accurate benchmark results and show that, with iterative retraining, our approach reaches two-dimensional spin lattices as large as $52\times52$ while remaining in agreement with available quantum Monte Carlo data. Our results establish recurrent architectures as a practical and promising route toward scalable neural quantum state simulations with modest computational resources.

cond-mat.str-elCross-listedarXiv:2605.13807•May 14, 2026•0 citations

NNQS-Transformer: an Efficient and Scalable Neural Network Quantum States Approach for Ab initio Quantum Chemistry

Yangjun Wu, Chu Guo, Fan Yi +2 more

Neural network quantum state (NNQS) has emerged as a promising candidate for quantum many-body problems, but its practical applications are often hindered by the high cost of sampling and local energy calculation. We develop a high-performance NNQS method for \textit{ab initio} electronic structure calculations. The major innovations include: (1) A transformer based architecture as the quantum wave function ansatz; (2) A data-centric parallelization scheme for the variational Monte Carlo (VMC) algorithm which preserves data locality and well adapts for different computing architectures; (3) A parallel batch sampling strategy which reduces the sampling cost and achieves good load balance; (4) A parallel local energy evaluation scheme which is both memory and computationally efficient; (5) Study of real chemical systems demonstrates both the superior accuracy of our method compared to state-of-the-art and the strong and weak scalability for large molecular systems with up to $120$ spin orbitals.

arXiv:2306.16705•Jun 29, 2023•0 citations

NNQS-Transformer: an Efficient and Scalable Neural Network Quantum States Approach for Ab initio Quantum Chemistry

Yangjun Wu, Chu Guo, Yi Fan +2 more

Neural network quantum state (NNQS) has emerged as a promising candidate for quantum many-body problems, but its practical applications are often hindered by the high cost of sampling and local energy calculation. We develop a high-performance NNQS method for ab initio electronic structure calculations. The major innovations include: (1) A transformer based architecture as the quantum wave function ansatz; (2) A data-centric parallelization scheme for the variational Monte Carlo (VMC) algorithm which preserves data locality and well adapts for different computing architectures; (3) A parallel batch sampling strategy which reduces the sampling cost and achieves good load balance; (4) A parallel local energy evaluation scheme which is both memory and computationally efficient; (5) Study of real chemical systems demonstrates both the superior accuracy of our method compared to state-of-the-art and the strong and weak scalability for large molecular systems with up to 120 spin orbitals.

International Conference for High Performance Computing, Networking, Storage and Analysis (SC)•Oct 30, 2023•34 citations

Autoregressive neural-network wavefunctions for ab initio quantum chemistry

Thomas D. Barrett, Aleksei Malyshev, Alexander I. Lvovsky

In recent years, neural network quantum states (NNQS) have emerged as powerful tools for the study of quantum many-body systems. Electronic structure calculations are one such canonical many-body problem that have attracted significant research efforts spanning multiple decades, whilst only recently being attempted with NNQS. However, the complex non-local interactions and high sample complexity are significant challenges that call for bespoke solutions. Here, we parameterise the electronic wavefunction with a novel autoregressive neural network (ARN) that permits highly efficient and scalable sampling, whilst also embedding physical priors reflecting the structure of molecular systems without sacrificing expressibility. This allows us to perform electronic structure calculations on molecules with up to 30 spin-orbitals -- at least an order of magnitude more Slater determinants than previous applications of conventional NNQS -- and we find that our ansatz can outperform the de-facto gold-standard coupled cluster methods even in the presence of strong quantum correlations. With a highly expressive neural network for which sampling is no longer a computational bottleneck, we conclude that the barriers to further scaling are not associated with the wavefunction ansatz itself, but rather are inherent to any variational Monte Carlo approach.

arXiv:2109.12606•Sep 26, 2021•1 citation

Efficient optimization of neural network backflow for ab-initio quantum chemistry

An-Jun Liu, Bryan K. Clark

The ground state of second-quantized quantum chemistry Hamiltonians is key to determining molecular properties. Neural quantum states (NQS) offer flexible and expressive wavefunction ansatze for this task but face two main challenges: highly peaked ground-state wavefunctions hinder efficient sampling, and local energy evaluations scale quartically with system size, incurring significant computational costs. In this work, we overcome these challenges by introducing a suite of algorithmic enhancements, which includes efficient periodic compact subspace construction, truncated local energy evaluations, improved stochastic sampling, and physics-informed modifications. Applying these techniques to the neural network backflow (NNBF) ansatz, we demonstrate significant gains in both accuracy and scalability. Our enhanced method surpasses traditional quantum chemistry methods like CCSD and CCSD(T), outperforms other NQS approaches, and achieves competitive energies with state-of-the-art ab initio techniques such as HCI, ASCI, FCIQMC, and DMRG. A series of ablation and comparative studies quantifies the contribution of each enhancement to the observed improvements in accuracy and efficiency. Furthermore, we investigate the representational capacity of the ansatz, finding that its performance correlates with the inverse participation ratio (IPR), with more delocalized states being more challenging to approximate.

arXiv:2502.18843•Feb 26, 2025•0 citations

Abstract Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansätze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in ab initio quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet’s time complexity outperforms that of the transformer. Though this efficiency comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model—namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS.

Neural-network quantum states (NQS) is an exciting area of quantum-inspired machine learning, wherein a fully classical neural network is used to efficiently learn the optimal state vector for a quantum unsupervised learning problem, bypassing the need to model the full Hilbert space, which is exponentially large. In theory, NQS architectures are broadly applicable, and have been demonstrated successfully on several fundamental applications. We seek to further explore the practical utility of NQS as a black box solver for various applications in the scientific computing domain. We first present NQS as an explicit avenue for de-quantizing variational quantum eigensolvers, demonstrating with the variational quantum linear solver (VQLS). We then explore the utility of VQLS and its de-quantization, VNLS, as black box matrix inversion tools for a Newton-based linear complementarity solver, used to model the time evolution of a rudimentary granular medium. Turning our attention toward applications of NQS in quantum chemistry, we introduce a new retentive network (RetNet) autoregressive NQS ansatz to solve electronic ground state problems, and explore the incorporation of neural annealing techniques to these problems as a means of improving accuracy and ease of training. We finish with a rudimentary scaling law analysis of the RetNet architecture and its immediate autoregressive predecessors as applied to electronic structure calculations.

Neural-network quantum states have emerged as a powerful variational framework for quantum many-body systems, with recent progress often driven by massively parallel architectures such as transformers. Recurrent neural network quantum states, however, are frequently regarded as intrinsically sequential and therefore less scalable. Here we revisit this view by showing that modern recurrent architectures can support fast, accurate, and computationally accessible neural quantum state simulations. Using autoregressive recurrent wave functions together with recent advances in parallelizable recurrence, we develop variational ansätze, called parallel scan recurrent neural quantum states (PSR-NQS), which can be trained efficiently within variational Monte Carlo in one and two spatial dimensions. We demonstrate accurate benchmark results and show that, with iterative retraining, our approach reaches two-dimensional spin lattices as large as

52\times52

while remaining in agreement with available quantum Monte Carlo data. Our results establish recurrent architectures as a practical and promising route toward scalable neural quantum state simulations with modest computational resources.

Neural network quantum state (NNQS) has emerged as a promising candidate for quantum many-body problems, but its practical applications are often hindered by the high cost of sampling and local energy calculation. We develop a high-performance NNQS method for \textit{ab initio} electronic structure calculations. The major innovations include: (1) A transformer based architecture as the quantum wave function ansatz; (2) A data-centric parallelization scheme for the variational Monte Carlo (VMC) algorithm which preserves data locality and well adapts for different computing architectures; (3) A parallel batch sampling strategy which reduces the sampling cost and achieves good load balance; (4) A parallel local energy evaluation scheme which is both memory and computationally efficient; (5) Study of real chemical systems demonstrates both the superior accuracy of our method compared to state-of-the-art and the strong and weak scalability for large molecular systems with up to

120

spin orbitals.

Neural network quantum state (NNQS) has emerged as a promising candidate for quantum many-body problems, but its practical applications are often hindered by the high cost of sampling and local energy calculation. We develop a high-performance NNQS method for ab initio electronic structure calculations. The major innovations include: (1) A transformer based architecture as the quantum wave function ansatz; (2) A data-centric parallelization scheme for the variational Monte Carlo (VMC) algorithm which preserves data locality and well adapts for different computing architectures; (3) A parallel batch sampling strategy which reduces the sampling cost and achieves good load balance; (4) A parallel local energy evaluation scheme which is both memory and computationally efficient; (5) Study of real chemical systems demonstrates both the superior accuracy of our method compared to state-of-the-art and the strong and weak scalability for large molecular systems with up to 120 spin orbitals.

In recent years, neural network quantum states (NNQS) have emerged as powerful tools for the study of quantum many-body systems. Electronic structure calculations are one such canonical many-body problem that have attracted significant research efforts spanning multiple decades, whilst only recently being attempted with NNQS. However, the complex non-local interactions and high sample complexity are significant challenges that call for bespoke solutions. Here, we parameterise the electronic wavefunction with a novel autoregressive neural network (ARN) that permits highly efficient and scalable sampling, whilst also embedding physical priors reflecting the structure of molecular systems without sacrificing expressibility. This allows us to perform electronic structure calculations on molecules with up to 30 spin-orbitals -- at least an order of magnitude more Slater determinants than previous applications of conventional NNQS -- and we find that our ansatz can outperform the de-facto gold-standard coupled cluster methods even in the presence of strong quantum correlations. With a highly expressive neural network for which sampling is no longer a computational bottleneck, we conclude that the barriers to further scaling are not associated with the wavefunction ansatz itself, but rather are inherent to any variational Monte Carlo approach.

The ground state of second-quantized quantum chemistry Hamiltonians is key to determining molecular properties. Neural quantum states (NQS) offer flexible and expressive wavefunction ansatze for this task but face two main challenges: highly peaked ground-state wavefunctions hinder efficient sampling, and local energy evaluations scale quartically with system size, incurring significant computational costs. In this work, we overcome these challenges by introducing a suite of algorithmic enhancements, which includes efficient periodic compact subspace construction, truncated local energy evaluations, improved stochastic sampling, and physics-informed modifications. Applying these techniques to the neural network backflow (NNBF) ansatz, we demonstrate significant gains in both accuracy and scalability. Our enhanced method surpasses traditional quantum chemistry methods like CCSD and CCSD(T), outperforms other NQS approaches, and achieves competitive energies with state-of-the-art ab initio techniques such as HCI, ASCI, FCIQMC, and DMRG. A series of ablation and comparative studies quantifies the contribution of each enhancement to the observed improvements in accuracy and efficiency. Furthermore, we investigate the representational capacity of the ansatz, finding that its performance correlates with the inverse participation ratio (IPR), with more delocalized states being more challenging to approximate.