This document presents a rigorous experimental framework for investigating automatic cal-ibration of superconducting qubits using reinforcement learning (RL). The calibration problemis formulated initially as a partially observable Markov decision process (POMDP), becausethe physical state of the device is not directly observable. A reduced MDP representationmay be used only when the measured state vector is demonstrably sufficient for the selectedcontrol task.The controller receives experimentally accessible observables, selects bounded calibrationactions, and receives a reward constructed from measurable calibration-quality metrics, actioncost, and safety constraints. A digital twin is used for offline training, while the final evaluationis performed on hardware under a frozen evaluation protocol.Version 3.1 introduces explicit separation between training, validation, and evaluation data;a formal sim-to-real analysis; a safety supervisor and deterministic fallback controller; latencyand drift-timescale requirements; preregistered primary and secondary endpoints; paired ex-perimental blocks; cooldown-level clustering; bootstrap and hierarchical statistical analysis;placebo and ablation tests; and reproducibility requirements.The primary endpoint is the difference in time-above-threshold between the RL controllerand a matched baseline controller. The analysis is designed to distinguish genuine adaptivecalibration from improvements caused by resource imbalance, initial-condition differences,reward leakage, hardware drift, or statistical artifacts.No experimental improvement is claimed in this document.
This document presents a rigorous experimental framework for investigating automatic cal-ibration of superconducting qubits using reinforcement learning (RL). The calibration problemis formulated initially as a partially observable Markov decision process (POMDP), becausethe physical state of the device is not directly observable. A reduced MDP representationmay be used only when the measured state vector is demonstrably sufficient for the selectedcontrol task.The controller receives experimentally accessible observables, selects bounded calibrationactions, and receives a reward constructed from measurable calibration-quality metrics, actioncost, and safety constraints. A digital twin is used for offline training, while the final evaluationis performed on hardware under a frozen evaluation protocol.Version 3.1 introduces explicit separation between training, validation, and evaluation data;a formal sim-to-real analysis; a safety supervisor and deterministic fallback controller; latencyand drift-timescale requirements; preregistered primary and secondary endpoints; paired ex-perimental blocks; cooldown-level clustering; bootstrap and hierarchical statistical analysis;placebo and ablation tests; and reproducibility requirements.The primary endpoint is the difference in time-above-threshold between the RL controllerand a matched baseline controller. The analysis is designed to distinguish genuine adaptivecalibration from improvements caused by resource imbalance, initial-condition differences,reward leakage, hardware drift, or statistical artifacts.No experimental improvement is claimed in this document.
This work proposes an experimentally falsifiable framework for incorporating real-timethermal information into quantum error correction for superconducting qubits. The centralidea is not to assume a universal thermal error law, but to determine experimentally whetherlocal temperature, thermal gradients, and related thermal observables contain predictive in-formation about physical error probabilities.Version 3.1 replaces the previous additive phenomenological error expression with a boundedstatistical model based on conditional error probabilities. For each physical error channel a,the model has the generic formp(a)i(t) = logit−1[︂β(a)0 + β(a)TδTi(t) + β(a)∇T Gi(t) + β(a)τδτ (t) + β(a)·xi(t)]︂,where Gi represents a normalized local temperature-gradient observable and xi containsoptional control variables. A physically motivated thermal occupation term may be includedas a covariate, but it is not imposed as the complete error model.Three operating modes are compared:1. Mode A: fixed-rate baseline decoder;2. Mode B: thermal-information-aware decoder with fixed cycle duration;3. Mode C: thermal-information-aware decoder with bounded adaptive cycle duration.The primary endpoint is the logical error-rate ratio between Mode B and Mode A undermatched experimental conditions. The secondary endpoint compares Mode C with Mode B.The protocol incorporates calibration, cross-validation, pre-registration of the primaryendpoint, paired experimental blocks, bootstrap confidence intervals, multiple-comparisoncontrol, systematic-error analysis, placebo tests, latency characterization, and independentcooldown reproducibility.No experimental result is claimed.
Quantum error correction (QEC) decoders are usually trained and evaluated assuming stationary physical noise.Real superconducting and trapped-ion processors, however, exhibit noise that drifts over calibration cycles, shifts abruptly because of twolevel-system defects and cosmic-ray events, and varies across qubits.Static decoders, whether algorithmic (minimum-weight perfect matching, union-find) or learned (such as AlphaQubit), degrade when deployed noise diverges from the training distribution, and full retraining is costly and can cause catastrophic forgetting.This paper proposes Adaptive-RL-QEC, a research framework that formulates decoding and noise adaptation as a reinforcement learning problem over a non-stationary Markov decision process with an unobserved noise-regime variable.It couples a lightweight online noise-regime estimator with a continual/meta-reinforcement-learning decoding policy that adapts within a bounded number of syndrome rounds without catastrophic P.
Version 6.1 develops an integral framework for an analog quantum simulator of spin modelsbased on programmable superconducting-qubit arrays.The principal objective is not merely to construct a quantum device, but to establish atraceable mappingcontrol parameters −→ Hreal −→ quantum evolution −→ observables −→ classical reference.The framework supports transverse-field Ising, XY, XXZ, Heisenberg, long-range, anddisordered spin Hamiltonians.Version 6.1 introduces an explicit Hamiltonian- identification layer between hardwarecalibration and scientific validation.The realized Hamiltonian is decomposed into target terms, static calibration errors, dynamiccontrol errors, crosstalk, leakage, decoherence, and residual unwanted interactions.Validation is organized hierarchically:1. hardware calibration;2. Hamiltonian identification;3. state-preparation validation;4. readout calibration;5. static observable validation;6. dynamical observable validation;7. finite-size benchmarking;8. statistical equivalence testing;9. cross-device reproducibility;10. independent analytical reproduction. The statistical framework replaces the inadequate criterion of “statistical significance ofagreement” with an equivalence framework based on preregistered tolerances.The primary hypothesis is therefore formulated using two one-sided tests (TOST) or anequivalent confidence interval criterion:|∆O| < ϵO.Version 6.1 additionally specifies:• a supplementary Bayesian posterior probability of equivalence as a robustness check;• an explicit distinction between engineering no-go and physical falsification.The framework also specifies negative controls, positive controls, blind analysis, lockedanalysis code, exclusion rules, uncertainty propagation, data provenance, and reproducibilityrequirements.No experimental result or quantum advantage is claimed.
Frequent recalibration of multi-qubit systems is essential to maintain hardware fidelity, but calibration itself is costly and subject to rapid parameter drift. This work presents a novel framework for time-aware active calibration that combines Bayesian Optimization (BO) and Reinforcement Learning (RL) to adaptively schedule and prioritize calibration experiments under uncertainty. We model the calibration landscape as a directed acyclic graph (DAG), where nodes represent hardware parameters (e.g., frequency, amplitude ratio, DRAG, or phase) and edges capture inter-parameter dependencies extracted automatically from Qiskit Experiments metadata. Each parameter's temporal drift is modeled as an Ornstein-Uhlenbeck (OU) or exponential decay process with uncertainty bounds. The BO agent prioritizes the next calibration target using a time-aware acquisition score that balances expected improvement, predictive uncertainty, and experiment duration under system-level constraints. A parallel RL scheduler coordinates cross-qubit calibration and readout tasks to minimize idle time and avoid conflicts. Our system further incorporates self-triggered recalibration based on online RB/XEB health metrics, enabling autonomous recovery from drift without human intervention. Experimental evaluation on multi-qubit testbeds demonstrates improved calibration efficiency, longer mean time between recalibrations (MTBC), and reduced overall calibration overhead compared to fixed or purely heuristic scheduling baselines.
Quantum networks make use of the quantum entanglement as building block. When two qubits are entangled, their state changes exhibit non-classical correlations used to design new applications not possible with classical communication, such us quantum key distribution or distributed quantum computation. Unfortunately, quantum entanglement is a probabilistic process strongly dependent on the features of involved devices (optical fibers, lasers, quantum memories, …). The management decisions (i.e., the control policy) to set up and keep the entanglement as long as possible with the highest quality constitutes a stochastic control problem. This process can be modelled as Markov Decision Process (MDP) and solved via the Reinforcement Learning (RL) framework (a form of Machine Learning). In this work, we apply this RL framework to learn an entanglement management policy outperforming the State-of-the-Art policy when models characterising precisely the involved quantum devices are not known.
Superconducting-qubit control is fundamentally constrained by decoherence, finite bandwidth, and hardware-limited drive amplitudes, making high-fidelity state preparation sensitive to optimizer initialization under non-convex open-system dynamics. We propose a hybrid reinforcement learning (RL)–quantum optimal control (QOC) pipeline in which a lightweight, tabular, model-free RL agent is trained offline in simulation to generate feasible, bounded seed pulses, which are subsequently refined via GRAPE under Lindblad dynamics. Hard amplitude constraints are enforced consistently across both stages, ensuring strict feasibility throughout optimization. Performance is evaluated using a budget-matched protocol based on fidelity evaluations (F-evals), enabling controlled comparison with random-start multi-start GRAPE. On a transmon-like qubit benchmark with relaxation and dephasing, RL warm-starting reduces the median online refinement effort in the adopted finite-difference GRAPE implementation from 7568 to 3543 F-evals (2.14× reduction) while achieving terminal state fidelity ≥0.995 under identical constraints and evaluation budgets. We provide a theoretical interpretation of the improvement in terms of basin-of-attraction probability shaping in constrained control landscapes and an amortized cost analysis showing that the offline RL cost is recovered after a small number of reuse cycles. The results support the view that learning-based initialization can improve warm-start quality relative to uninformed feasible multi-start in constrained open-system quantum-control benchmarks, while broader practical comparison against stronger physics-guided seeds remains for future work.