arXiv:2508.00068 · Physical review. E

Double descent: When do neural quantum states generalize?

M. Schuyler Moss, Alev Orfi, Christopher Roth, Anirvan M. Sengupta, Antoine Georges, Dries Sels, Anna Dawid, Agnes Valenti

Published Mar 13, 2026• 1 citation

arXiv HTML PDF

Abstract

Neural quantum states (NQS) provide flexible and compact wave-function parametrizations for numerical studies of quantum many-body physics. In particular, NQS aim to circumvent the exponential scaling of the Hilbert space by compressing quantum many-body wave functions with a tractable amount of parameters. While inspired by deep learning, it remains unclear to what extent NQS share characteristics with neural networks used for standard machine learning tasks. We demonstrate that, in a simplified supervised setting, NQS exhibit the double descent phenomenon, a key feature of modern deep learning, where generalization worsens as network size increases before improving again in an overparameterized regime. Notably, we find the second descent to occur only for network sizes much larger than the Hilbert space dimension, i.e., network sizes that are out of reach for problems of practical interest. Within our setting, this observation places typical NQS in the underparameterized regime. We also observe that the optimal network size in the underparameterized regime depends on the number of unique training samples. While the double descent phenomenon does indeed translate to the NQS setting, potential practical consequences of our findings point more toward the need for symmetry-aware, physics-informed architecture design, rather than directly adopting machine learning heuristics.

DOI: 10.1103/cwmj-fxr4

Related papers

Double Descent in Quantum Kernel Methods

Marie Kempkes, Aroosa Ijaz, Elies Gil-Fuster +4 more

The double descent phenomenon challenges traditional statistical learning theory by revealing scenarios where larger models do not necessarily lead to reduced performance on unseen data. While this counterintuitive behavior has been observed in a variety of classical machine learning models, particularly modern neural network architectures, it remains elusive within the context of quantum machine learning. In this work, we analytically demonstrate that linear regression models in quantum feature spaces can exhibit double descent behavior by drawing on insights from classical linear regression and random matrix theory. Additionally, our numerical experiments on quantum kernel methods across different real-world datasets and system sizes further confirm the existence of a test error peak, a characteristic feature of double descent. Our findings provide evidence that quantum models can operate in the modern, overparameterized regime without experiencing overfitting, potentially opening pathways to improved learning performance beyond traditional statistical learning theory.

arXiv:2501.10077PRX QuantumSep 15, 20252 citations

Grokking and epoch-wise double descent in quantum neural networks

Daniel Pranjić, Marco Roth, Christian Tutschku

Grokking, the delayed transition from memorization to generalization, is a fundamental phenomenon in gradient-based learning, yet its dynamics within variational quantum machine learning (QML) remain largely unexamined. In this work, we report the empirical observation of both the grokking transition and epoch-wise double descent in a two-qubit quantum neural network (QNN) under a complete parameterization of the SU(4) manifold. We demonstrate that overparameterization via increased circuit depth improves the probability of successful generalization. Notably, these architectures frequently exhibit an epoch-wise double descent in test error, degrading at a critical epoch before recovering into a generalizing state. Crucially, we identify a generalization decay in late-stage training, where the test error increases significantly despite a stagnant training loss. Bridging this behavior with algorithmic stability theory, our analysis reveals that this decay correlates with an unconstrained increase of the weight-norm, drifting away from sparse, phase-aligned harmonic solutions toward overfitted solutions in the Hilbert space. We analyze the underlying temporal dynamics of this transition, demonstrating how the onset of generalization is linked to optimization hyperparameters such as learning rate and weight decay. Finally, to mitigate late-stage decay, we introduce a weak explicit weight-norm regularization into the loss function. We demonstrate that this structural anchor stabilizes the post-grokking phase and permanently preserves generalization gains, providing a robust framework for training overparameterized quantum circuits.

quant-pharXiv:2607.08350Jul 10, 20260 citations

Generalization properties of neural network approximations to frustrated magnet ground states

Tom Westerhout, Nikita Astrakhantsev, Konstantin S. Tikhonov +2 more

Neural quantum states (NQS) attract a lot of attention due to their potential to serve as a very expressive variational ansatz for quantum many-body systems. Here we study the main factors governing the applicability of NQS to frustrated magnets by training neural networks to approximate ground states of several moderately-sized Hamiltonians using the corresponding wave function structure on a small subset of the Hilbert space basis as training dataset. We notice that generalization quality, i.e. the ability to learn from a limited number of samples and correctly approximate the target state on the rest of the space, drops abruptly when frustration is increased. We also show that learning the sign structure is considerably more difficult than learning amplitudes. Finally, we conclude that the main issue to be addressed at this stage, in order to use the method of NQS for simulating realistic models, is that of generalization rather than expressibility.

arXiv:1907.08186Nature CommunicationsMar 27, 2020134 citations

Cautious optimism for deep parameterized quantum circuits

Marie Kempkes, Elies Gil-Fuster, Carlos Bravo-Prieto +5 more

A central challenge in quantum machine learning is understanding the scaling behavior of parameterized quantum circuits (PQCs). In particular, it remains unclear how their performance on unseen data changes as the number of trainable parameters increases. Prior works have derived formal generalization guarantees for quantum models, but it is well-known that many such results do not fully characterize generalization behavior in practice. In this work, we show that gradient-based PQCs can exhibit improved performance on unseen data as model size increases, displaying the phenomenon of double descent. This contrasts with the traditional view that larger models lead to degraded generalization. We provide analytical results rigorously underpinning this behavior by leveraging add-one-in perturbation techniques and spectral properties of random matrices. We support these results with numerical experiments on re-uploading PQCs across several data sets and training set sizes, consistently observing the predicted double descent behavior. While other obstacles on the path toward practical quantum machine learning remain, our finding that deeper parameterized quantum circuits do not necessarily exhibit degraded performance provides reasons for cautious optimism.

quant-pharXiv:2607.21409Jul 24, 20260 citations

Exploration of Quantum-Inspired Deep Learning Architectures for Fundamental Applications in Scientific Computing

Oliver Knitter

Neural-network quantum states (NQS) is an exciting area of quantum-inspired machine learning, wherein a fully classical neural network is used to efficiently learn the optimal state vector for a quantum unsupervised learning problem, bypassing the need to model the full Hilbert space, which is exponentially large. In theory, NQS architectures are broadly applicable, and have been demonstrated successfully on several fundamental applications. We seek to further explore the practical utility of NQS as a black box solver for various applications in the scientific computing domain. We first present NQS as an explicit avenue for de-quantizing variational quantum eigensolvers, demonstrating with the variational quantum linear solver (VQLS). We then explore the utility of VQLS and its de-quantization, VNLS, as black box matrix inversion tools for a Newton-based linear complementarity solver, used to model the time evolution of a rudimentary granular medium. Turning our attention toward applications of NQS in quantum chemistry, we introduce a new retentive network (RetNet) autoregressive NQS ansatz to solve electronic ground state problems, and explore the incorporation of neural annealing techniques to these problems as a means of improving accuracy and ease of training. We finish with a rudimentary scaling law analysis of the RetNet architecture and its immediate autoregressive predecessors as applied to electronic structure calculations.

Jan 1, 20240 citations

From Architectures to Applications: A Review of Neural Quantum States

Hannah Lange, Van de Walle, Anka, Atiye Abedinnia +1 more

Due to the exponential growth of the Hilbert space dimension with system size, the simulation of quantum many-body systems has remained a persistent challenge until today. Here, we review a relatively new class of variational states for the simulation of such systems, namely neural quantum states (NQS), which overcome the exponential scaling by compressing the state in terms of the network parameters rather than storing all exponentially many coefficients needed for an exact parameterization of the state. We introduce the commonly used NQS architectures and their various applications for the simulation of ground and excited states, finite temperature and open system states as well as NQS approaches to simulate the dynamics of quantum states. Furthermore, we discuss NQS in the context of quantum state tomography.

arXiv:2402.09402Feb 14, 20241 citation