Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2002.01068

Policy Gradient based Quantum Approximate Optimization Algorithm

Jiahao Yao, Marin Bukov, Lin Lin

Published Feb 4, 2020• 34 citations
arXivHTMLPDF

Abstract

The quantum approximate optimization algorithm (QAOA), as a hybrid quantum/classical algorithm, has received much interest recently. QAOA can also be viewed as a variational ansatz for quantum control. However, its direct application to emergent quantum technology encounters additional physical constraints: (i) the states of the quantum system are not observable; (ii) obtaining the derivatives of the objective function can be computationally expensive or even inaccessible in experiments, and (iii) the values of the objective function may be sensitive to various sources of uncertainty, as is the case for noisy intermediate-scale quantum (NISQ) devices. Taking such constraints into account, we show that policy-gradient-based reinforcement learning (RL) algorithms are well suited for optimizing the variational parameters of QAOA in a noise-robust fashion, opening up the way for developing RL techniques for continuous quantum control. This is advantageous to help mitigate and monitor the potentially unknown sources of errors in modern quantum simulators. We analyze the performance of the algorithm for quantum state transfer problems in single- and multi-qubit systems, subject to various sources of noise such as error terms in the Hamiltonian, or quantum uncertainty in the measurement process. We show that, in noisy setups, it is capable of outperforming state-of-the-art existing optimization algorithms.

DOI
10.48550/arxiv.2002.01068

Related papers

Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems

Sami Khairy, Ruslan Shaydulin, Łukasz Cincio +2 more

Quantum computing exploits basic quantum phenomena such as state superposition and entanglement to perform computations. The Quantum Approximate Optimization Algorithm (QAOA) is arguably one of the leading quantum algorithms that can outperform classical state-of-the-art methods in the near term. QAOA is a hybrid quantum-classical algorithm that combines a parameterized quantum state evolution with a classical optimization routine to approximately solve combinatorial problems. The quality of the solution obtained by QAOA within a fixed budget of calls to the quantum computer depends on the performance of the classical optimization routine used to optimize the variational parameters. In this work, we propose an approach based on reinforcement learning (RL) to train a policy network that can be used to quickly find high-quality variational parameters for unseen combinatorial problem instances. The RL agent is trained on small problem instances which can be simulated on a classical computer, yet the learned RL policy is generalizable and can be used to efficiently solve larger instances. Extensive simulations using the IBM Qiskit Aer quantum circuit simulator demonstrate that our trained RL policy can reduce the optimality gap by a factor up to 8.61 compared with other off-the-shelf optimizers tested.

arXiv:1911.04574•Nov 11, 2019•21 citations

Adaptive and Robust Feedback-Based Quantum Optimization using Reinforcement Learning

Sanjay Sharma, Shyam Akashe, Govind Murari Upadhyay +1 more

Abstract The paper introduces a novel hybrid Feedback-Based Quantum Optimization (FBQO) framework that integrates reinforcement learning (RL) for adaptive parameter control and Kalman filters for noise mitigation to enhance quantum optimization on noisy intermediate-scale quantum (NISQ) devices. Unlike traditional quantum Approximate optimization algorithm (QAOA), the proposed method dynamically tunes parameters through learned policies and statistically filtered feedback, enabling faster convergence and greater noise resilience. Key contributions include the formulation of quantum optimization as a Markov Decision Process, integration of deep quantum networks (DQNs) for adaptive control, and the use of Kalman filtering for robust state estimation. Experimental results on the Max-Cut problem demonstrate superior performance in convergence rate, stability, and optimization accuracy over existing techniques. The main finding is that RL-FBQO achieves convergence in 10–20 iterations, compared to 20–30 for FBQO and 40–50 for QAOA, with superior stability.

Journal of Transformative Technologies and Sustainable Development•Oct 21, 2025•0 citations

Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks

Jiahao Yao, Paul Köttering, Hans Gundlach +2 more

Variational quantum eigensolvers have recently received increased attention, as they enable the use of quantum computing devices to find solutions to complex problems, such as the ground energy and ground state of strongly-correlated quantum many-body systems. In many applications, it is the optimization of both continuous and discrete parameters that poses a formidable challenge. Using reinforcement learning (RL), we present a hybrid policy gradient algorithm capable of simultaneously optimizing continuous and discrete degrees of freedom in an uncertainty-resilient way. The hybrid policy is modeled by a deep autoregressive neural network to capture causality. We employ the algorithm to prepare the ground state of the nonintegrable quantum Ising model in a unitary process, parametrized by a generalized quantum approximate optimization ansatz: the RL agent solves the discrete combinatorial problem of constructing the optimal sequences of unitaries out of a predefined set and, at the same time, it optimizes the continuous durations for which these unitaries are applied. We demonstrate the noise-robust features of the agent by considering three sources of uncertainty: classical and quantum measurement noise, and errors in the control unitary durations. Our work exhibits the beneficial synergy between reinforcement learning and quantum control.

arXiv:2012.06701•Dec 12, 2020•8 citations

Robust Control Optimization for Quantum Approximate Optimization Algorithm

Yulong Dong, Xiang Meng, Lin Lin +2 more

Quantum variational algorithms have garnered significant interest recently, due to their feasibility of being implemented and tested on noisy intermediate scale quantum (NISQ) devices. We examine the robustness of the quantum approximate optimization algorithm (QAOA), which can be used to solve certain quantum control problems, state preparation problems, and combinatorial optimization problems. We demonstrate that the error of QAOA simulation can be significantly reduced by robust control optimization techniques, specifically, by sequential convex programming (SCP), to ensure error suppression in situations where the source of the error is known but not necessarily its magnitude. We show that robust optimization improves both the objective landscape of QAOA as well as overall circuit fidelity in the presence of coherent errors and errors in initial state preparation.

arXiv:1911.00789•Nov 2, 2019•7 citations

Robust Control Optimization for Quantum Approximate Optimization Algorithms

Yulong Dong, Xiang Meng, Lin Lin +2 more

Quantum variational algorithms have garnered significant interest recently, due to their feasibility of being implemented and tested on noisy intermediate scale quantum (NISQ) devices. We examine the robustness of the quantum approximate optimization algorithm (QAOA), which can be used to solve certain quantum control problems, state preparation problems, and combinatorial optimization problems. We demonstrate that the error of QAOA simulation can be significantly reduced by robust control optimization techniques, specifically, by sequential convex programming (SCP), to ensure error suppression in situations where the source of the error is known but not necessarily its magnitude. We show that robust optimization improves both the objective landscape of QAOA as well as overall circuit fidelity in the presence of coherent errors and errors in initial state preparation.

IFAC-PapersOnLine•Jan 1, 2020•19 citations

Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems

Sami Khairy, Ruslan Shaydulin, Lukasz Cincio +2 more

Quantum computing is a computational paradigm with the potential to outperform classical methods for a variety of problems. Proposed recently, the Quantum Approximate Optimization Algorithm (QAOA) is considered as one of the leading candidates for demonstrating quantum advantage in the near term. QAOA is a variational hybrid quantum-classical algorithm for approximately solving combinatorial optimization problems. The quality of the solution obtained by QAOA for a given problem instance depends on the performance of the classical optimizer used to optimize the variational parameters. In this paper, we formulate the problem of finding optimal QAOA parameters as a learning task in which the knowledge gained from solving training instances can be leveraged to find high-quality solutions for unseen test instances. To this end, we develop two machine-learning-based approaches. Our first approach adopts a reinforcement learning (RL) framework to learn a policy network to optimize QAOA circuits. Our second approach adopts a kernel density estimation (KDE) technique to learn a generative model of optimal QAOA parameters. In both approaches, the training procedure is performed on small-sized problem instances that can be simulated on a classical computer; yet the learned RL policy and the generative model can be used to efficiently solve larger problems. Extensive simulations using the IBM Qiskit Aer quantum circuit simulator demonstrate that our proposed RL- and KDE-based approaches reduce the optimality gap by factors up to 30.15 when compared with other commonly used off-the-shelf optimizers.

arXiv:1911.11071•Proceedings of the AAAI Conference on Artificial Intelligence•Apr 3, 2020•117 citations
The quantum approximate optimization algorithm (QAOA), as a hybrid quantum/classical algorithm, has received much interest recently. QAOA can also be viewed as a variational ansatz for quantum control. However, its direct application to emergent quantum technology encounters additional physical constraints: (i) the states of the quantum system are not observable; (ii) obtaining the derivatives of the objective function can be computationally expensive or even inaccessible in experiments, and (iii) the values of the objective function may be sensitive to various sources of uncertainty, as is the case for noisy intermediate-scale quantum (NISQ) devices. Taking such constraints into account, we show that policy-gradient-based reinforcement learning (RL) algorithms are well suited for optimizing the variational parameters of QAOA in a noise-robust fashion, opening up the way for developing RL techniques for continuous quantum control. This is advantageous to help mitigate and monitor the potentially unknown sources of errors in modern quantum simulators. We analyze the performance of the algorithm for quantum state transfer problems in single- and multi-qubit systems, subject to various sources of noise such as error terms in the Hamiltonian, or quantum uncertainty in the measurement process. We show that, in noisy setups, it is capable of outperforming state-of-the-art existing optimization algorithms.
Quantum computing exploits basic quantum phenomena such as state superposition and entanglement to perform computations. The Quantum Approximate Optimization Algorithm (QAOA) is arguably one of the leading quantum algorithms that can outperform classical state-of-the-art methods in the near term. QAOA is a hybrid quantum-classical algorithm that combines a parameterized quantum state evolution with a classical optimization routine to approximately solve combinatorial problems. The quality of the solution obtained by QAOA within a fixed budget of calls to the quantum computer depends on the performance of the classical optimization routine used to optimize the variational parameters. In this work, we propose an approach based on reinforcement learning (RL) to train a policy network that can be used to quickly find high-quality variational parameters for unseen combinatorial problem instances. The RL agent is trained on small problem instances which can be simulated on a classical computer, yet the learned RL policy is generalizable and can be used to efficiently solve larger instances. Extensive simulations using the IBM Qiskit Aer quantum circuit simulator demonstrate that our trained RL policy can reduce the optimality gap by a factor up to 8.61 compared with other off-the-shelf optimizers tested.
Abstract The paper introduces a novel hybrid Feedback-Based Quantum Optimization (FBQO) framework that integrates reinforcement learning (RL) for adaptive parameter control and Kalman filters for noise mitigation to enhance quantum optimization on noisy intermediate-scale quantum (NISQ) devices. Unlike traditional quantum Approximate optimization algorithm (QAOA), the proposed method dynamically tunes parameters through learned policies and statistically filtered feedback, enabling faster convergence and greater noise resilience. Key contributions include the formulation of quantum optimization as a Markov Decision Process, integration of deep quantum networks (DQNs) for adaptive control, and the use of Kalman filtering for robust state estimation. Experimental results on the Max-Cut problem demonstrate superior performance in convergence rate, stability, and optimization accuracy over existing techniques. The main finding is that RL-FBQO achieves convergence in 10–20 iterations, compared to 20–30 for FBQO and 40–50 for QAOA, with superior stability.
Variational quantum eigensolvers have recently received increased attention, as they enable the use of quantum computing devices to find solutions to complex problems, such as the ground energy and ground state of strongly-correlated quantum many-body systems. In many applications, it is the optimization of both continuous and discrete parameters that poses a formidable challenge. Using reinforcement learning (RL), we present a hybrid policy gradient algorithm capable of simultaneously optimizing continuous and discrete degrees of freedom in an uncertainty-resilient way. The hybrid policy is modeled by a deep autoregressive neural network to capture causality. We employ the algorithm to prepare the ground state of the nonintegrable quantum Ising model in a unitary process, parametrized by a generalized quantum approximate optimization ansatz: the RL agent solves the discrete combinatorial problem of constructing the optimal sequences of unitaries out of a predefined set and, at the same time, it optimizes the continuous durations for which these unitaries are applied. We demonstrate the noise-robust features of the agent by considering three sources of uncertainty: classical and quantum measurement noise, and errors in the control unitary durations. Our work exhibits the beneficial synergy between reinforcement learning and quantum control.
Quantum variational algorithms have garnered significant interest recently, due to their feasibility of being implemented and tested on noisy intermediate scale quantum (NISQ) devices. We examine the robustness of the quantum approximate optimization algorithm (QAOA), which can be used to solve certain quantum control problems, state preparation problems, and combinatorial optimization problems. We demonstrate that the error of QAOA simulation can be significantly reduced by robust control optimization techniques, specifically, by sequential convex programming (SCP), to ensure error suppression in situations where the source of the error is known but not necessarily its magnitude. We show that robust optimization improves both the objective landscape of QAOA as well as overall circuit fidelity in the presence of coherent errors and errors in initial state preparation.
Quantum variational algorithms have garnered significant interest recently, due to their feasibility of being implemented and tested on noisy intermediate scale quantum (NISQ) devices. We examine the robustness of the quantum approximate optimization algorithm (QAOA), which can be used to solve certain quantum control problems, state preparation problems, and combinatorial optimization problems. We demonstrate that the error of QAOA simulation can be significantly reduced by robust control optimization techniques, specifically, by sequential convex programming (SCP), to ensure error suppression in situations where the source of the error is known but not necessarily its magnitude. We show that robust optimization improves both the objective landscape of QAOA as well as overall circuit fidelity in the presence of coherent errors and errors in initial state preparation.
Quantum computing is a computational paradigm with the potential to outperform classical methods for a variety of problems. Proposed recently, the Quantum Approximate Optimization Algorithm (QAOA) is considered as one of the leading candidates for demonstrating quantum advantage in the near term. QAOA is a variational hybrid quantum-classical algorithm for approximately solving combinatorial optimization problems. The quality of the solution obtained by QAOA for a given problem instance depends on the performance of the classical optimizer used to optimize the variational parameters. In this paper, we formulate the problem of finding optimal QAOA parameters as a learning task in which the knowledge gained from solving training instances can be leveraged to find high-quality solutions for unseen test instances. To this end, we develop two machine-learning-based approaches. Our first approach adopts a reinforcement learning (RL) framework to learn a policy network to optimize QAOA circuits. Our second approach adopts a kernel density estimation (KDE) technique to learn a generative model of optimal QAOA parameters. In both approaches, the training procedure is performed on small-sized problem instances that can be simulated on a classical computer; yet the learned RL policy and the generative model can be used to efficiently solve larger problems. Extensive simulations using the IBM Qiskit Aer quantum circuit simulator demonstrate that our proposed RL- and KDE-based approaches reduce the optimality gap by factors up to 30.15 when compared with other commonly used off-the-shelf optimizers.