Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2604.21863

Replay-buffer engineering for noise-robust quantum circuit optimization

Akash Kundu, Sebastian Feld

quant-phcs.AIcs.ETcs.LGSubmitted Apr 23, 2026• Announced Apr 24, 2026• 0 citations
v1 · Apr 23, 2026v2 · Sep 29, 2026
arXivHTMLPDF

Abstract

Deep reinforcement learning (RL) for quantum circuit optimization faces three fundamental bottlenecks: replay buffers that ignore the reliability of temporal-difference (TD) targets, curriculum-based architecture search that triggers a full quantum-classical evaluation at every environment step, and the routine discard of noiseless trajectories when retraining under hardware noise. We address all three by treating the replay buffer as a primary algorithmic lever for quantum optimization. We introduce ReaPER$+$, an annealed replay rule that transitions from TD error-driven prioritization early in training to reliability-aware sampling as value estimates mature, achieving $4-32\times$ gains in sample efficiency over fixed PER, ReaPER, and uniform replay while consistently discovering more compact circuits across quantum compilation and QAS benchmarks; validation on LunarLander-v3 confirms the principle is domain-agnostic. Furthermore we eliminate the quantum-classical evaluation bottleneck in curriculum RL by introducing OptCRLQAS which amortizes expensive evaluations over multiple architectural edits, cutting wall-clock time per episode by up to $67.5\%$ on a 12-qubit optimization problem without degrading solution quality. Finally we introduce a lightweight replay-buffer transfer scheme that warm-starts noisy-setting learning by reusing noiseless trajectories, without network-weight transfer or $ε$-greedy pretraining. This reduces steps to chemical accuracy by up to $85-90\%$ and final energy error by up to $90\%$ over from-scratch baselines on 6-, 8-, and 12-qubit molecular tasks. Together, these results establish that experience storage, sampling, and transfer are decisive levers for scalable, noise-robust quantum circuit optimization.

Comments
Comments are warmly welcomed. 9 page main content, 17 page appendix

Related papers

Generative Replay Mitigates Sample Starvation in Quantum Architecture Search

Akash Kundu, Amit Kumar Jaiswal, Sebastian Feld +1 more

Reinforcement learning (RL) can automate quantum architecture search, but its scalability is limited when useful circuit trajectories become rare in the rapidly expanding search space. Existing replay mechanisms reuse observed transitions; the proposed learned model produces additional predicted one step transitions from real state-action seeds. Here we introduce GenQAS, a tensor network-guided RL framework that combines a fixed matrix product state warm-start with prioritized generative replay. A learned local transition model generates synthetic circuit transitions on demand and mixes them with real experience during Double Deep Q-Network updates. Under a random exploration analysis, near ground state circuits occupy a rapidly shrinking region of the accessible state space. We investigate whether real data anchored synthetic replay can improve the effective training signal in this regime. Across chemical Hamiltonian benchmarks from 6 to 12 qubits, GenQAS improves fixed-budget success probability and identifies compact circuits at competitive energy error. At 12 qubits, it improves final success probability by up to $7.0\times$ over passive replay. On a 15-qubit transverse field Ising model, GenQAS increases success probability from $12\%$ to $21\%$. In a noisy 6-qubit BeH$_2$ transfer experiment, generative replay reduces the steps to chemical accuracy by $92.7\%$. These results show that generative replay can mitigate sample starvation in quantum architecture search and support more resource efficient circuit discovery.

quant-pharXiv:2609.11248•Sep 11, 2026•0 citations

Curriculum reinforcement learning for quantum architecture search under hardware errors

Yash J. Patel, Akash Kundu, Mateusz Ostaszewski +3 more

The key challenge in the noisy intermediate-scale quantum era is finding useful circuits compatible with current device limitations. Variational quantum algorithms (VQAs) offer a potential solution by fixing the circuit architecture and optimizing individual gate parameters in an external loop. However, parameter optimization can become intractable, and the overall performance of the algorithm depends heavily on the initially chosen circuit architecture. Several quantum architecture search (QAS) algorithms have been developed to design useful circuit architectures automatically. In the case of parameter optimization alone, noise effects have been observed to dramatically influence the performance of the optimizer and final outcomes, which is a key line of study. However, the effects of noise on the architecture search, which could be just as critical, are poorly understood. This work addresses this gap by introducing a curriculum-based reinforcement learning QAS (CRLQAS) algorithm designed to tackle challenges in realistic VQA deployment. The algorithm incorporates (i) a 3D architecture encoding and restrictions on environment dynamics to explore the search space of possible circuits efficiently, (ii) an episode halting scheme to steer the agent to find shorter circuits, and (iii) a novel variant of simultaneous perturbation stochastic approximation as an optimizer for faster convergence. To facilitate studies, we developed an optimized simulator for our algorithm, significantly improving computational efficiency in simulating noisy quantum circuits by employing the Pauli-transfer matrix formalism in the Pauli-Liouville basis. Numerical experiments focusing on quantum chemistry tasks demonstrate that CRLQAS outperforms existing QAS algorithms across several metrics in both noiseless and noisy environments.

arXiv:2402.03500•Feb 5, 2024•11 citations

Quantum prioritized experience replay with MaDi-based priority and quantum circuit mechanisms for optimizing reinforcement learning

Reinforcement learning (RL) encounters significant challenges related to scalability and computational efficiency, particularly in complex decision-making environments. Traditional RL algorithms often struggle with large-scale tasks, necessitating innovative approaches to enhance adaptability and performance. The quantum circuit-based priority replay (QCPR) algorithm was introduced in this study, leveraging quantum-inspired techniques to enhance learning efficiency and decision-making quality through quantum computing (QC). The QCPR algorithm implements two key mechanisms for prioritizing experiences: the magnitude and direction (MaDi) based priority, which assesses the significance and directionality of experiences, and QCPR-specific methods that ensure efficient experience replay. These mechanisms are seamlessly integrated into the Q-learning (QL) framework to optimize the RL process for superior performance. QCPR was evaluated against four well-established RL algorithms-QL, deep Q-networks (DQN), prioritized experience replay (PER), and dueling DQN (DDQN)-using standard decision-making benchmarks. The algorithm was further tested in various simulation and real-world environments, including Atari games, Qiskit integration, and Raspberry Pi hardware and software setups. These evaluations demonstrated QCPR’s adaptability and robustness, showcasing its capability for dynamic, large-scale applications. The results revealed that QCPR significantly outperforms its counterparts across key performance metrics, including accuracy, precision, recall, F1-score, and cumulative rewards. Specifically, QCPR achieved a 15.29% improvement in accuracy over QL, a 9.98% increase over DQN, a 6.33% improvement over PER, and an 8.23% enhancement over DDQN. This study highlights the potential of quantum-inspired approaches to advance RL, offering scalable and efficient solutions for complex decision-making tasks.

International Journal of Advanced Technology and Engineering Exploration•Dec 31, 2024•0 citations

Transferable Agentic AI for Accelerating Quantum Algorithm Discovery: Meta-Learning and AutoML for Parameterized Quantum Circuits

Satyanarayana Burugupalli

Discovering effective parameterized quantum circuits (PQCs) remains a major obstacle in the noisy intermediatescale quantum (NISQ) era due to barren plateaus, noise sensitivity, and heavy trial-and-error in ansatz selection and initialization. We propose Q-Transfer-Search, a transferable agentic AI framework that unifies meta-learning and automated machine learning (AutoML) to accelerate quantum algorithm discovery. Our method learns reusable parameter priors via meta-learning, enabling fast adaptation to new tasks, while a noise-aware architecture controller proposes PQCs optimized for hardware constraints. A measurement-efficient evaluator based on classical shadows and parameter-shift rules provides low-cost feedback for both inner and outer optimization loops. Evaluations on VQE, QAOA, and quantum machine learning tasks show that Q-Transfer-Search achieves lower energy errors, higher approximation ratios, and improved classification accuracy while reducing circuit evaluations and measurement shots. Compared to state-of-the-art baselines such as QuantumNAS, Meta-VQA, Q-MAML, and MG-Net, our framework consistently improves convergence speed and robustness. These results highlight the promise of agentic AI in enabling scalable, transferable, and resource-efficient quantum algorithm design.

International Conference on Intelligent Systems and Embedded Design (ISED)•Dec 17, 2025•0 citations

Deep reinforcement learning for quantum circuit optimization

Helkkula, Christopher

Quantum computing promises significant computational advantages over classical methods in certain applications but is currently too unstable for executing complex tasks. High-fidelity quantum circuits are critical to overcoming this challenge, yet they are computationally challenging to design. On current noisy intermediate-scale quantum (NISQ) hardware, shallower circuits with fewer two-qubit gates generally run with higher fidelity. Increased fidelity can be sought by sequentially rewriting the circuit, which involves a vast search space. This costly search space exploration could be circumvented by using reinforcement learning (RL) to teach an agent to rewrite the circuit in a sequential decision-making process. This thesis aims to implement a Deep Q-Network (DQN) algorithm to optimize BernsteinVazirani (BVZ) circuits. The DQN algorithm interacts with a Cirq-based environment. The environment provides valid rewrite actions to the DQN algorithm, which gains a scalar reward emphasizing depth and gate reduction. In the thesis we compare two different state encodings as input for the neural network: a compact state summarizing vector representation with three features, and an image-like encoding that maps circuit state topology with tokens through a fixed embedding. Both input encodings use the same action rewrite set and reward design. The network is implemented in PyTorch and trained with Adam optimizer, and runs on GPU via CUDA. Empirically, the agent shows promising initial optimizing performance. On several BVZ circuits with varying qubit count and topology, the agent drives the depth toward the theoretical optimum of three. The simplistic vector encoding already reaches reasonable circuit depth plateaus, with the image-like encoding together with hyperparameter tuning improving reliability, showing avenues for continued improvement. We conclude that DQN is a viable foundation for automated circuit optimization on BVZ circuits. To achieve reliable circuit optimization at scale, future research should investigate even richer encoding methods and reward function shaping. Finally, the results support applying the DQN to be applied to circuit families beyond BVZ.

•Oct 1, 2025•0 citations

Thermal Noise Mitigation Algorithms for Quantum Enhanced Neural Network Training

P. Sagayaraj, Dr.G.H Kerinab Beenu, M Vinitha +1 more

Although quantum computers will not necessarily transform ML in general, they hold immense potential for specific uses, such as dealing with high-dimensional data and complex correlations that cannot be effectively modeled using classical systems. It is, however, difficult to design efficient quantum circuits, especially with the existing Noisy Intermediate-Scale Quantum (NISQ) hardware that has constraints including decoherence, a limited number of qubits, and gate fidelity limitations. The authors propose an Adaptive Quantum Circuit Architecture Search (AQCAS) framework whose structure and parameters are optimized dynamically to improve the performance of the task-specific ML. The framework integrates quantum circuit encoding, a hybrid search strategy that is based on reinforcement learning and evolutionary algorithms, and a task-specific adaptivity mechanism. Benchmark datasets were used for experimental evaluations of both simulation using Qiskit Aer and classical optimization using the Python frameworks TensorFlow and PennyLane. Fixed and standard variational quantum circuits were compared with performance measurements like the classification accuracy, circuit depth, convergence speed, and computational efficiency. The results show that the accuracy of AQCAS can be raised to 94.1%, and the circuit depth is lowered at the same time. It converged rapidly and is computationally efficient. Ablation studies show that both hybrid search and adaptivity mechanisms help to enhance generalization and stability under NISQ constraints. The results demonstrate the potential of AQCAS as a practical approach to using adaptive quantum circuits in specialized ML applications, and lay the groundwork for future development of scaling to larger devices, implementing noise-aware optimization, and integrating with hybrid quantum-classical ML pipelines.

International Journal of Artificial Intelligence and Machine Learning•May 24, 2026•0 citations
Deep reinforcement learning (RL) for quantum circuit optimization faces three fundamental bottlenecks: replay buffers that ignore the reliability of temporal-difference (TD) targets, curriculum-based architecture search that triggers a full quantum-classical evaluation at every environment step, and the routine discard of noiseless trajectories when retraining under hardware noise. We address all three by treating the replay buffer as a primary algorithmic lever for quantum optimization. We introduce ReaPER+++, an annealed replay rule that transitions from TD error-driven prioritization early in training to reliability-aware sampling as value estimates mature, achieving 4−32×4-32\times4−32× gains in sample efficiency over fixed PER, ReaPER, and uniform replay while consistently discovering more compact circuits across quantum compilation and QAS benchmarks; validation on LunarLander-v3 confirms the principle is domain-agnostic. Furthermore we eliminate the quantum-classical evaluation bottleneck in curriculum RL by introducing OptCRLQAS which amortizes expensive evaluations over multiple architectural edits, cutting wall-clock time per episode by up to 67.5%67.5\%67.5% on a 12-qubit optimization problem without degrading solution quality. Finally we introduce a lightweight replay-buffer transfer scheme that warm-starts noisy-setting learning by reusing noiseless trajectories, without network-weight transfer or εεε-greedy pretraining. This reduces steps to chemical accuracy by up to 85−90%85-90\%85−90% and final energy error by up to 90%90\%90% over from-scratch baselines on 6-, 8-, and 12-qubit molecular tasks. Together, these results establish that experience storage, sampling, and transfer are decisive levers for scalable, noise-robust quantum circuit optimization.
Reinforcement learning (RL) can automate quantum architecture search, but its scalability is limited when useful circuit trajectories become rare in the rapidly expanding search space. Existing replay mechanisms reuse observed transitions; the proposed learned model produces additional predicted one step transitions from real state-action seeds. Here we introduce GenQAS, a tensor network-guided RL framework that combines a fixed matrix product state warm-start with prioritized generative replay. A learned local transition model generates synthetic circuit transitions on demand and mixes them with real experience during Double Deep Q-Network updates. Under a random exploration analysis, near ground state circuits occupy a rapidly shrinking region of the accessible state space. We investigate whether real data anchored synthetic replay can improve the effective training signal in this regime. Across chemical Hamiltonian benchmarks from 6 to 12 qubits, GenQAS improves fixed-budget success probability and identifies compact circuits at competitive energy error. At 12 qubits, it improves final success probability by up to 7.0×7.0\times7.0× over passive replay. On a 15-qubit transverse field Ising model, GenQAS increases success probability from 12%12\%12% to 21%21\%21%. In a noisy 6-qubit BeH2_22​ transfer experiment, generative replay reduces the steps to chemical accuracy by 92.7%92.7\%92.7%. These results show that generative replay can mitigate sample starvation in quantum architecture search and support more resource efficient circuit discovery.
The key challenge in the noisy intermediate-scale quantum era is finding useful circuits compatible with current device limitations. Variational quantum algorithms (VQAs) offer a potential solution by fixing the circuit architecture and optimizing individual gate parameters in an external loop. However, parameter optimization can become intractable, and the overall performance of the algorithm depends heavily on the initially chosen circuit architecture. Several quantum architecture search (QAS) algorithms have been developed to design useful circuit architectures automatically. In the case of parameter optimization alone, noise effects have been observed to dramatically influence the performance of the optimizer and final outcomes, which is a key line of study. However, the effects of noise on the architecture search, which could be just as critical, are poorly understood. This work addresses this gap by introducing a curriculum-based reinforcement learning QAS (CRLQAS) algorithm designed to tackle challenges in realistic VQA deployment. The algorithm incorporates (i) a 3D architecture encoding and restrictions on environment dynamics to explore the search space of possible circuits efficiently, (ii) an episode halting scheme to steer the agent to find shorter circuits, and (iii) a novel variant of simultaneous perturbation stochastic approximation as an optimizer for faster convergence. To facilitate studies, we developed an optimized simulator for our algorithm, significantly improving computational efficiency in simulating noisy quantum circuits by employing the Pauli-transfer matrix formalism in the Pauli-Liouville basis. Numerical experiments focusing on quantum chemistry tasks demonstrate that CRLQAS outperforms existing QAS algorithms across several metrics in both noiseless and noisy environments.
Reinforcement learning (RL) encounters significant challenges related to scalability and computational efficiency, particularly in complex decision-making environments. Traditional RL algorithms often struggle with large-scale tasks, necessitating innovative approaches to enhance adaptability and performance. The quantum circuit-based priority replay (QCPR) algorithm was introduced in this study, leveraging quantum-inspired techniques to enhance learning efficiency and decision-making quality through quantum computing (QC). The QCPR algorithm implements two key mechanisms for prioritizing experiences: the magnitude and direction (MaDi) based priority, which assesses the significance and directionality of experiences, and QCPR-specific methods that ensure efficient experience replay. These mechanisms are seamlessly integrated into the Q-learning (QL) framework to optimize the RL process for superior performance. QCPR was evaluated against four well-established RL algorithms-QL, deep Q-networks (DQN), prioritized experience replay (PER), and dueling DQN (DDQN)-using standard decision-making benchmarks. The algorithm was further tested in various simulation and real-world environments, including Atari games, Qiskit integration, and Raspberry Pi hardware and software setups. These evaluations demonstrated QCPR’s adaptability and robustness, showcasing its capability for dynamic, large-scale applications. The results revealed that QCPR significantly outperforms its counterparts across key performance metrics, including accuracy, precision, recall, F1-score, and cumulative rewards. Specifically, QCPR achieved a 15.29% improvement in accuracy over QL, a 9.98% increase over DQN, a 6.33% improvement over PER, and an 8.23% enhancement over DDQN. This study highlights the potential of quantum-inspired approaches to advance RL, offering scalable and efficient solutions for complex decision-making tasks.
Discovering effective parameterized quantum circuits (PQCs) remains a major obstacle in the noisy intermediatescale quantum (NISQ) era due to barren plateaus, noise sensitivity, and heavy trial-and-error in ansatz selection and initialization. We propose Q-Transfer-Search, a transferable agentic AI framework that unifies meta-learning and automated machine learning (AutoML) to accelerate quantum algorithm discovery. Our method learns reusable parameter priors via meta-learning, enabling fast adaptation to new tasks, while a noise-aware architecture controller proposes PQCs optimized for hardware constraints. A measurement-efficient evaluator based on classical shadows and parameter-shift rules provides low-cost feedback for both inner and outer optimization loops. Evaluations on VQE, QAOA, and quantum machine learning tasks show that Q-Transfer-Search achieves lower energy errors, higher approximation ratios, and improved classification accuracy while reducing circuit evaluations and measurement shots. Compared to state-of-the-art baselines such as QuantumNAS, Meta-VQA, Q-MAML, and MG-Net, our framework consistently improves convergence speed and robustness. These results highlight the promise of agentic AI in enabling scalable, transferable, and resource-efficient quantum algorithm design.
Quantum computing promises significant computational advantages over classical methods in certain applications but is currently too unstable for executing complex tasks. High-fidelity quantum circuits are critical to overcoming this challenge, yet they are computationally challenging to design. On current noisy intermediate-scale quantum (NISQ) hardware, shallower circuits with fewer two-qubit gates generally run with higher fidelity. Increased fidelity can be sought by sequentially rewriting the circuit, which involves a vast search space. This costly search space exploration could be circumvented by using reinforcement learning (RL) to teach an agent to rewrite the circuit in a sequential decision-making process. This thesis aims to implement a Deep Q-Network (DQN) algorithm to optimize BernsteinVazirani (BVZ) circuits. The DQN algorithm interacts with a Cirq-based environment. The environment provides valid rewrite actions to the DQN algorithm, which gains a scalar reward emphasizing depth and gate reduction. In the thesis we compare two different state encodings as input for the neural network: a compact state summarizing vector representation with three features, and an image-like encoding that maps circuit state topology with tokens through a fixed embedding. Both input encodings use the same action rewrite set and reward design. The network is implemented in PyTorch and trained with Adam optimizer, and runs on GPU via CUDA. Empirically, the agent shows promising initial optimizing performance. On several BVZ circuits with varying qubit count and topology, the agent drives the depth toward the theoretical optimum of three. The simplistic vector encoding already reaches reasonable circuit depth plateaus, with the image-like encoding together with hyperparameter tuning improving reliability, showing avenues for continued improvement. We conclude that DQN is a viable foundation for automated circuit optimization on BVZ circuits. To achieve reliable circuit optimization at scale, future research should investigate even richer encoding methods and reward function shaping. Finally, the results support applying the DQN to be applied to circuit families beyond BVZ.
Although quantum computers will not necessarily transform ML in general, they hold immense potential for specific uses, such as dealing with high-dimensional data and complex correlations that cannot be effectively modeled using classical systems. It is, however, difficult to design efficient quantum circuits, especially with the existing Noisy Intermediate-Scale Quantum (NISQ) hardware that has constraints including decoherence, a limited number of qubits, and gate fidelity limitations. The authors propose an Adaptive Quantum Circuit Architecture Search (AQCAS) framework whose structure and parameters are optimized dynamically to improve the performance of the task-specific ML. The framework integrates quantum circuit encoding, a hybrid search strategy that is based on reinforcement learning and evolutionary algorithms, and a task-specific adaptivity mechanism. Benchmark datasets were used for experimental evaluations of both simulation using Qiskit Aer and classical optimization using the Python frameworks TensorFlow and PennyLane. Fixed and standard variational quantum circuits were compared with performance measurements like the classification accuracy, circuit depth, convergence speed, and computational efficiency. The results show that the accuracy of AQCAS can be raised to 94.1%, and the circuit depth is lowered at the same time. It converged rapidly and is computationally efficient. Ablation studies show that both hybrid search and adaptivity mechanisms help to enhance generalization and stability under NISQ constraints. The results demonstrate the potential of AQCAS as a practical approach to using adaptive quantum circuits in specialized ML applications, and lay the groundwork for future development of scaling to larger devices, implementing noise-aware optimization, and integrating with hybrid quantum-classical ML pipelines.