We introduce a generative pretained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encoder-decoder architecture takes as input the interacting Hamiltonian, and outputs an autoregressive sequence of qubit measurement probabilities. Its performance is studied in the vicinity of a quantum phase transition in Rydberg atoms in a square lattice array. We explore the ability of the architecture to generalize, by producing groundstate measurements for Hamiltonian parameters not seen in the training set. We focus on examples of physical observables obtained from inference on three different models, trained in fixed compute time on a single NVIDIA A100 GPU. These can act as benchmarks for the scaling of larger RydbergGPT models in the future. Finally, we provide RydbergGPT open source, to aid in the development of foundation models based off of a wide variety of quantum computer interactions and data sets in the future.
We introduce a generative pretained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encoder-decoder architecture takes as input the interacting Hamiltonian, and outputs an autoregressive sequence of qubit measurement probabilities. Its performance is studied in the vicinity of a quantum phase transition in Rydberg atoms in a square lattice array. We explore the ability of the architecture to generalize, by producing groundstate measurements for Hamiltonian parameters not seen in the training set. We focus on examples of physical observables obtained from inference on three different models, trained in fixed compute time on a single NVIDIA A100 GPU. These can act as benchmarks for the scaling of larger RydbergGPT models in the future. Finally, we provide RydbergGPT open source, to aid in the development of foundation models based off of a wide variety of quantum computer interactions and data sets in the future.
Abstract We introduce a generative pretrained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encoder–decoder architecture takes as input the interacting Hamiltonian, and outputs an autoregressive sequence of qubit measurement probabilities. Its performance is studied in the vicinity of a quantum phase transition in Rydberg atoms in a square lattice array. We explore the model’s generalization capabilities by demonstrating that it can accurately predict ground-state measurement outcomes for Hamiltonian parameter values that were not included in the training data. We evaluate three model variants, each trained for a fixed duration on a single NVIDIA A100 GPU, by examining their predictions of key physical observables. These results establish performance benchmarks for scaling to larger RydbergGPT models. These can act as benchmarks for the scaling of larger RydbergGPT models in the future. Finally, we release RydbergGPT as open-source software to facilitate the development of foundation models for diverse quantum computing platforms and datasets.
In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.
Programmable quantum simulators based on Rydberg atom arrays are a fast-emerging quantum platform, bringing together long coherence times, high-fidelity operations, and large numbers of interacting qubits deterministically arranged in flexible geometries. Today's Rydberg array devices are demonstrating their utility as quantum simulators for studying phases and phase transitions in quantum matter. In this paper, we show that unprocessed and imperfect experimental projective measurement data can be used to enhance in silico simulations of quantum matter, by improving the performance of variational Monte Carlo simulations. As an example, we focus on data spanning the disordered-to-checkerboard transition in a $16\ifmmode\times\else\texttimes\fi{}16$ square-lattice array [S. Ebadi et al., Nature (London) 595, 227 (2021)] and employ the data-enhanced variational Monte Carlo algorithm to train powerful autoregressive wave-function ans\"atze based on recurrent neural networks (RNNs). We observe universal improvements in the convergence times of our simulations with this hybrid training scheme. Notably, we also find that pretraining with experimental data enables relatively simple RNN ans\"atze to accurately capture phases of matter that are not learned with a purely variational training approach. Our work highlights the promise of hybrid quantum-classical approaches for large-scale simulation of quantum many-body systems, combining autoregressive language models with experimental data from existing quantum devices.
Configurable arrays of optically trapped Rydberg atoms are a versatile platform for quantum computation and quantum simulation, also allowing controllable decoherence. We demonstrate theoretically that they also enable proof-of-principle demonstrations for a technique to build models for open quantum dynamics using machine learning with artificial neural networks, recently proposed by Mukherjee et al. [Quantum Sci. Technol. 10, 02LT01 (2025)]. Using the outcome of quantum transport through a network of sites that correspond to excited Rydberg atoms, the multistage neural-network algorithm successfully identifies the number of atoms (or nodes in the network) and, subsequently, their location. It further extracts an effective interaction Hamiltonian and decoherence operators induced by the environment. To probe the Rydberg array, one initiates dynamics repeatedly from the same initial state and then measures the transport probability to an output atom. Large datasets are generated by varying the position of the latter. Measurements are required in only one single basis, making the approach complementary to, e.g., quantum process tomography. Due to steeply increasing requirements for training data size, the present implementation appears best suited for networks of just a few sites. The cold-atom platform discussed in this article can be used to explore the performance of the proposed protocol when training the neural network with simulation data but then applying it to construct models based on experimental data.
Machine learning has emerged recently as a powerful tool for predicting properties of quantum many-body systems. For many ground states of gapped Hamiltonians, generative models can learn from measurements of a single quantum state to reconstruct the state accurately enough to predict local observables. Alternatively, classification and regression models can predict local observables by learning from measurements on different but related states. In this work, we combine the benefits of both approaches and propose the use of conditional generative models to simultaneously represent a family of states, learning shared structures of different quantum states from measurements. The trained model enables us to predict arbitrary local properties of ground states, even for states not included in the training data, without necessitating further training for new observables. We first numerically validate our approach on 2D random Heisenberg models using simulations of up to 45 qubits. Furthermore, we conduct quantum simulations on a neutral-atom quantum computer and demonstrate that our method can accurately predict the quantum phases of square lattices of 13$\times$13 Rydberg atoms.
Benchmarking of quantum machine learning (QML) algorithms is challenging due to the complexity and variability of QML systems, e. g., regarding model ansatzes, data sets, training techniques, and hyper-parameter selection. The QUantum computing Application benchmaRK (QUARK) framework simplifies and standardizes benchmarking studies for quantum computing applications. Here, we propose several extensions of QUARK to include the ability to evaluate the training and deployment of quantum generative models. We describe the updated software architecture and illustrate its flexibility through several example applications: (1) We trained different quantum generative models using several circuit ansatzes, data sets, and data transformations. (2) We evaluated our models on GPU and real quantum hardware. (3) We assessed the generalization capabilities of our generative models using a broad set of metrics that capture, e. g., the novelty and validity of the generated data.