Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Electronics

Reinforcement Learning Based Warm Initialization for Constrained Open-System Quantum Optimal Control: A Controlled Budget-Matched RL-GRAPE Benchmark

Daniele Gabriele, Lorenzo Ricciardi Celsi

Published Mar 17, 2026• 0 citations
Publisher

Abstract

Superconducting-qubit control is fundamentally constrained by decoherence, finite bandwidth, and hardware-limited drive amplitudes, making high-fidelity state preparation sensitive to optimizer initialization under non-convex open-system dynamics. We propose a hybrid reinforcement learning (RL)–quantum optimal control (QOC) pipeline in which a lightweight, tabular, model-free RL agent is trained offline in simulation to generate feasible, bounded seed pulses, which are subsequently refined via GRAPE under Lindblad dynamics. Hard amplitude constraints are enforced consistently across both stages, ensuring strict feasibility throughout optimization. Performance is evaluated using a budget-matched protocol based on fidelity evaluations (F-evals), enabling controlled comparison with random-start multi-start GRAPE. On a transmon-like qubit benchmark with relaxation and dephasing, RL warm-starting reduces the median online refinement effort in the adopted finite-difference GRAPE implementation from 7568 to 3543 F-evals (2.14× reduction) while achieving terminal state fidelity ≥0.995 under identical constraints and evaluation budgets. We provide a theoretical interpretation of the improvement in terms of basin-of-attraction probability shaping in constrained control landscapes and an amortized cost analysis showing that the offline RL cost is recovered after a small number of reuse cycles. The results support the view that learning-based initialization can improve warm-start quality relative to uninformed feasible multi-start in constrained open-system quantum-control benchmarks, while broader practical comparison against stronger physics-guided seeds remains for future work.

DOI
10.3390/electronics15061251

Related papers

Optimal control with flag qubits

Liang-Xu Xie, Lui Zuccherelli de Paula, Weizhou Cai +6 more

High-fidelity quantum operations are the cornerstone of fault-tolerant quantum computation. In open quantum systems, traditional optimal control only passively resists decoherence, leaving environment-induced uncertainty as a fundamental performance bottleneck. To overcome this, we propose a new optimal control framework with flag ancillas and the Flag-GRAPE algorithm, which can actively tailor the system's noise structure. Through embedding post-selection directly into the objective function, Flag-GRAPE correlates decoherence errors with the ancilla's unexpected state. Subsequent measurement and post-selection effectively expel this uncertainty, circumventing the fidelity bounds of traditional control. Numerical simulations in a superconducting quantum circuit demonstrate a $51\%$ reduction in infidelity compared to traditional closed-system pulses and also show that such enhancement is robust across broad noise regimes. Furthermore, by actively converting unstructured decoherence into heralded erasure errors, Flag-GRAPE is inherently compatible with quantum error correction. We demonstrate this by initializing a logical cat-code state, showing that the combination between Flag-GRAPE and QEC yields immediate state preparation enhancements. This new framework can reduce hardware overhead for fault-tolerant architectures and open up a practical path toward logical state preparation gain in near-term experiments.

quant-pharXiv:2603.12162•Mar 13, 2026•0 citations

Dissipative Feedback and Hybrid Lyapunov–Reinforcement Learning Control for NV-Center Quantum Sensing

Yuxuan Zhao, Xiaofeng Jiang, Tianxiang Xiong +2 more

Practical Nitrogen-vacancy (NV)-center magnetometry requires both accurate modeling of open quantum dynamics and robust control under environmental drift. However, commonly used perturbative descriptions inadequately capture dissipative processes, while existing control strategies remain individually limited: open-loop protocols cannot adapt to unknown frequency variations, Lyapunov feedback typically converges only to a neighborhood of the optimal sensing state, and reinforcement learning alone lacks stability guarantees and requires extensive training. Here, we develop a Lindblad-based framework for one effective NV center that explicitly incorporates relaxation and dephasing; the analytical model does not describe a collective many-NV ensemble. We introduce a dissipation-triggered feedback mechanism that restores high-sensitivity states following spontaneous emission events. Building on this model, we propose a hybrid control architecture that combines Lyapunov stabilization with reinforcement-learning-based adaptive correction to compensate unknown Hamiltonian drift. Robustness tests over detuning and microwave-amplitude variations show that hybrid attains the highest quantum Fisher information (QFI), substantially lower cross-realization dispersion than pure reinforcement learning (RL) and robust gradient-ascent pulse engineering (GRAPE), faster fixed-budget learning than pure RL, and substantially lower offline dynamical-propagation cost with smoother control modulation than robust GRAPE. This physics-informed and data-driven strategy provides a robust and resource-efficient approach for maintaining high QFI in realistic NV sensing environments.

IEEE Transactions on Quantum Engineering•Jan 1, 2026•0 citations

Reinforcement learning for optimal control of spin magnetometers

Logan Cooke, Stefanie Czischek

Quantum optimal control in the presence of decoherence is difficult, particularly when not all Hamiltonian parameters are known precisely, as in quantum sensing applications. In this context, maximizing the sensitivity of the system is the objective, for which the optimal target state or unitary transformations are unknown, especially in the case of multiparameter estimation. Here we investigate the use of reinforcement learning (RL), specifically the soft actor-critic (SAC) algorithm, for problems in quantum optimal control. We adopt a spin-based magnetometer as a benchmarking system for the efficacy of the SAC algorithm. In such systems, the magnitude of a background magnetic field acting on a spin can be determined via projective measurements. The precision of the determined magnitude can be optimized by applying pulses of transverse fields with different strengths. We train an RL agent on numerical simulations of the spin system to determine a transverse-field pulse sequence that optimizes the precision and compare it to existing sensing strategies. We evaluate the agent's performance against various Hamiltonian parameters, including values not seen in training, to investigate the agent's ability to generalize to different situations. We find that the RL agents are sensitive to certain parameters of the system, such as the pulse duration and the purity of the initial state, but overall are able to generalize well, supporting the use of RL in quantum optimal control settings.

arXiv:2506.21475•Physical Review A•Nov 3, 2025•3 citations

Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization

Haftu W. Fentaw, Steve Campbell, Simon Caton

We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.

quant-pharXiv:2605.26925•May 27, 2026•0 citations

EO-GRAPE and EO-DRLPE: Open and Closed Loop Approaches for Energy Efficient Quantum Optimal Control

Sebastiaan Fauquenot, Aritra Sarkar, Sebastian Feld

This research investigates the possibility of using quantum optimal control techniques to co-optimize the energetic cost and the process fidelity of a quantum unitary gate. The energetic cost is theoretically defined, and thereby, the gradient of the energetic cost for pulse engineering is derived. We empirically demonstrate the Pareto optimality in the trade-off between process fidelity and energetic cost. Thereafter, two novel numerical quantum optimal control approaches are proposed: (i) energy-optimized gradient ascent pulse engineering (EO-GRAPE) as an open-loop gradient-based method, and (ii) energy-optimized deep reinforcement learning for pulse engineering (EO-DRLPE) as a closed-loop method. The performance of both methods is probed in the presence of increasing noise. We find that the EO-GRAPE method performs better than the EO-DRLPE methods with and without a warm start for most experimental settings. Additionally, for one qubit unitary gate, we illustrate the correlation between the Bloch sphere path length and the energetic cost.

arXiv:2411.06556•Nov 10, 2024•0 citations

Uncovering Latent Structures in Robust Pulse Sequences: A Model-Based Reinforcement Learning Approach for Adaptable Quantum Control

Tobias Kiermeyer, Thomas Heydenreich, Léo Van Damme +3 more

Real-time adaptive control of quantum systems requires rapid generation of robust, high-fidelity pulses across a continuous range of operating conditions. Standard optimization algorithms such as gradient-ascent pulse engineering (GRAPE) solve each instance independently, discarding information between runs and requiring costly reinitialization when parameters change. We present an approach to robust optimal quantum control based on model-based reinforcement learning, in which a single neural network -- embedding the Hamiltonian directly into the training pipeline -- generates robust gates across an entire family of gate configurations, without pre-computed training data. Demonstrated on a single-spin (two-level) system, the trained networks produce pulses for arbitrary rotation angles over a range of pulse durations, detunings, and field inhomogeneities in milliseconds, at fidelities comparable to multi-seed GRAPE. The framework is inherently adaptable: any parameter entering the Hamiltonian can serve as a network input, extending the approach to different systems and control settings. Beyond speed, the network reveals structure in the control landscape: it discovers the same structured phase profiles that appear in GRAPE solutions -- made identifiable through fidelity-invariant symmetry transformations -- but more consistently than independent optimization. This consistency enables smooth interpolation across the entire trained parameter space.

quant-pharXiv:2606.24507•Jun 24, 2026•0 citations
Superconducting-qubit control is fundamentally constrained by decoherence, finite bandwidth, and hardware-limited drive amplitudes, making high-fidelity state preparation sensitive to optimizer initialization under non-convex open-system dynamics. We propose a hybrid reinforcement learning (RL)–quantum optimal control (QOC) pipeline in which a lightweight, tabular, model-free RL agent is trained offline in simulation to generate feasible, bounded seed pulses, which are subsequently refined via GRAPE under Lindblad dynamics. Hard amplitude constraints are enforced consistently across both stages, ensuring strict feasibility throughout optimization. Performance is evaluated using a budget-matched protocol based on fidelity evaluations (F-evals), enabling controlled comparison with random-start multi-start GRAPE. On a transmon-like qubit benchmark with relaxation and dephasing, RL warm-starting reduces the median online refinement effort in the adopted finite-difference GRAPE implementation from 7568 to 3543 F-evals (2.14× reduction) while achieving terminal state fidelity ≥0.995 under identical constraints and evaluation budgets. We provide a theoretical interpretation of the improvement in terms of basin-of-attraction probability shaping in constrained control landscapes and an amortized cost analysis showing that the offline RL cost is recovered after a small number of reuse cycles. The results support the view that learning-based initialization can improve warm-start quality relative to uninformed feasible multi-start in constrained open-system quantum-control benchmarks, while broader practical comparison against stronger physics-guided seeds remains for future work.
High-fidelity quantum operations are the cornerstone of fault-tolerant quantum computation. In open quantum systems, traditional optimal control only passively resists decoherence, leaving environment-induced uncertainty as a fundamental performance bottleneck. To overcome this, we propose a new optimal control framework with flag ancillas and the Flag-GRAPE algorithm, which can actively tailor the system's noise structure. Through embedding post-selection directly into the objective function, Flag-GRAPE correlates decoherence errors with the ancilla's unexpected state. Subsequent measurement and post-selection effectively expel this uncertainty, circumventing the fidelity bounds of traditional control. Numerical simulations in a superconducting quantum circuit demonstrate a 51%51\%51% reduction in infidelity compared to traditional closed-system pulses and also show that such enhancement is robust across broad noise regimes. Furthermore, by actively converting unstructured decoherence into heralded erasure errors, Flag-GRAPE is inherently compatible with quantum error correction. We demonstrate this by initializing a logical cat-code state, showing that the combination between Flag-GRAPE and QEC yields immediate state preparation enhancements. This new framework can reduce hardware overhead for fault-tolerant architectures and open up a practical path toward logical state preparation gain in near-term experiments.
Practical Nitrogen-vacancy (NV)-center magnetometry requires both accurate modeling of open quantum dynamics and robust control under environmental drift. However, commonly used perturbative descriptions inadequately capture dissipative processes, while existing control strategies remain individually limited: open-loop protocols cannot adapt to unknown frequency variations, Lyapunov feedback typically converges only to a neighborhood of the optimal sensing state, and reinforcement learning alone lacks stability guarantees and requires extensive training. Here, we develop a Lindblad-based framework for one effective NV center that explicitly incorporates relaxation and dephasing; the analytical model does not describe a collective many-NV ensemble. We introduce a dissipation-triggered feedback mechanism that restores high-sensitivity states following spontaneous emission events. Building on this model, we propose a hybrid control architecture that combines Lyapunov stabilization with reinforcement-learning-based adaptive correction to compensate unknown Hamiltonian drift. Robustness tests over detuning and microwave-amplitude variations show that hybrid attains the highest quantum Fisher information (QFI), substantially lower cross-realization dispersion than pure reinforcement learning (RL) and robust gradient-ascent pulse engineering (GRAPE), faster fixed-budget learning than pure RL, and substantially lower offline dynamical-propagation cost with smoother control modulation than robust GRAPE. This physics-informed and data-driven strategy provides a robust and resource-efficient approach for maintaining high QFI in realistic NV sensing environments.
Quantum optimal control in the presence of decoherence is difficult, particularly when not all Hamiltonian parameters are known precisely, as in quantum sensing applications. In this context, maximizing the sensitivity of the system is the objective, for which the optimal target state or unitary transformations are unknown, especially in the case of multiparameter estimation. Here we investigate the use of reinforcement learning (RL), specifically the soft actor-critic (SAC) algorithm, for problems in quantum optimal control. We adopt a spin-based magnetometer as a benchmarking system for the efficacy of the SAC algorithm. In such systems, the magnitude of a background magnetic field acting on a spin can be determined via projective measurements. The precision of the determined magnitude can be optimized by applying pulses of transverse fields with different strengths. We train an RL agent on numerical simulations of the spin system to determine a transverse-field pulse sequence that optimizes the precision and compare it to existing sensing strategies. We evaluate the agent's performance against various Hamiltonian parameters, including values not seen in training, to investigate the agent's ability to generalize to different situations. We find that the RL agents are sensitive to certain parameters of the system, such as the pulse duration and the purity of the initial state, but overall are able to generalize well, supporting the use of RL in quantum optimal control settings.
We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.
This research investigates the possibility of using quantum optimal control techniques to co-optimize the energetic cost and the process fidelity of a quantum unitary gate. The energetic cost is theoretically defined, and thereby, the gradient of the energetic cost for pulse engineering is derived. We empirically demonstrate the Pareto optimality in the trade-off between process fidelity and energetic cost. Thereafter, two novel numerical quantum optimal control approaches are proposed: (i) energy-optimized gradient ascent pulse engineering (EO-GRAPE) as an open-loop gradient-based method, and (ii) energy-optimized deep reinforcement learning for pulse engineering (EO-DRLPE) as a closed-loop method. The performance of both methods is probed in the presence of increasing noise. We find that the EO-GRAPE method performs better than the EO-DRLPE methods with and without a warm start for most experimental settings. Additionally, for one qubit unitary gate, we illustrate the correlation between the Bloch sphere path length and the energetic cost.
Real-time adaptive control of quantum systems requires rapid generation of robust, high-fidelity pulses across a continuous range of operating conditions. Standard optimization algorithms such as gradient-ascent pulse engineering (GRAPE) solve each instance independently, discarding information between runs and requiring costly reinitialization when parameters change. We present an approach to robust optimal quantum control based on model-based reinforcement learning, in which a single neural network -- embedding the Hamiltonian directly into the training pipeline -- generates robust gates across an entire family of gate configurations, without pre-computed training data. Demonstrated on a single-spin (two-level) system, the trained networks produce pulses for arbitrary rotation angles over a range of pulse durations, detunings, and field inhomogeneities in milliseconds, at fidelities comparable to multi-seed GRAPE. The framework is inherently adaptable: any parameter entering the Hamiltonian can serve as a network input, extending the approach to different systems and control settings. Beyond speed, the network reveals structure in the control landscape: it discovers the same structured phase profiles that appear in GRAPE solutions -- made identifiable through fidelity-invariant symmetry transformations -- but more consistently than independent optimization. This consistency enables smooth interpolation across the entire trained parameter space.