Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1812.10797 · Physical Review A

Quantum adiabatic algorithm design using reinforcement learning

Jian Hui Lin, Zhong Yuan Lai, Xiaopeng Li

Published May 18, 2020• 39 citations
arXivHTMLPDF

Abstract

Quantum algorithm design plays a crucial role in exploiting the computational advantage of quantum devices. Here we develop a deep-reinforcement-learning based approach for quantum adiabatic algorithm design. Our approach is generically applicable to a class of problems with solution hard-to-find but easy-to-verify, e.g., searching and NP-complete problems. We benchmark this approach in Grover-search and 3-SAT problems, and find that the adiabatic algorithm obtained by our RL approach leads to significant improvement in the resultant success probability. In application to Grover search, our RL design automatically produces an adiabatic quantum algorithm that has the quadratic speedup. We find for all our studied cases that quantitatively the RL-designed algorithm has a better performance compared to the analytically constructed nonlinear Hamiltonian path when the encoding Hamiltonian is solvable, and that this RL-design approach remains applicable even when the nonlinear Hamiltonian path is not analytically available. In 3-SAT we find RL design has fascinating transferability---the adiabatic algorithm obtained by training on a specific choice of clause number leads to better performance consistently over the linear algorithm on different clause numbers. These findings suggest the applicability of reinforcement learning for automated quantum adiabatic algorithm design. Further considering the established complexity equivalence of circuit and adiabatic quantum algorithms, we expect the RL-designed adiabatic algorithm to inspire novel circuit algorithms as well. Our approach is potentially applicable to different quantum hardware from trapped ions and optical lattices to superconducting-qubit devices.

DOI
10.1103/physreva.101.052327

Related papers

Reinforcement learning architecture for automated quantum-adiabatic-algorithm design.

Jian Hui Lin, Zhong Yuan Lai, Xiaopeng Li

Quantum algorithm design lies in the hallmark of applications of quantum computation and quantum simulation. Here we put forward a deep reinforcement learning (RL) architecture for automated algorithm design in the framework of quantum adiabatic algorithm, where the optimal Hamiltonian path to reach a quantum ground state that encodes a compution problem is obtained by RL techniques. We benchmark our approach in Grover search and 3-SAT problems, and find that the adiabatic algorithm obtained by our RL approach leads to significant improvement in the success probability and computing speedups for both moderate and large number of qubits compared to conventional algorithms. The RL-designed algorithm is found to be qualitatively distinct from the linear algorithm in the resultant distribution of success probability. Considering the established complexity-equivalence of circuit and adiabatic quantum algorithms, we expect the RL-designed adiabatic algorithm to inspire novel circuit algorithms as well. Our approach offers a recipe to design quantum algorithms for generic problems through a machinery RL process, which paves a novel way to automated quantum algorithm design using artificial intelligence, potentially applicable to different quantum simulation and computation platforms from trapped ions and optical lattices to superconducting-qubit devices.

•Dec 27, 2018•0 citations

Reinforcement-Learning-Based Quantum Adiabatic Algorithm Design for Integer Programming

Jiawei Zhu

Adiabatic quantum computing (AQC) is a computation protocol to solve difficult problems exploiting quantum advantage, directly applicable to optimization problems. In performing the AQC, different configurations of the Hamiltonian path could lead to dramatic differences in the computation efficiency. It is thus crucial to configure the Hamiltonian path to optimize the computation performance of AQC. Here we apply a reinforcement learning approach to configure AQC for integer programming, where we find the learning process automatically converges to a quantum algorithm that exhibits scaling advantage over the trivial AQC using a linear Hamiltonian path. This reinforcement-learning-based approach for quantum adiabatic algorithm design for integer programming can well be adapted to the quantum resources in different quantum computation devices, due to its built-in flexibility.

SPIN•Sep 1, 2021•0 citations

Quantum Architecture Search via Continual Reinforcement Learning

Esther Ye, Samuel Yen-Chi Chen

Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.

arXiv:2112.05779•Dec 10, 2021•15 citations

QAS-QTNs: Curriculum Reinforcement Learning-Driven Quantum Architecture Search for Quantum Tensor Networks

Siddhant Dutta, Nouhaila Innan, Sadok Ben Yahia +1 more

Quantum Architecture Search (QAS) is an emerging field aimed at automating the design of quantum circuits for optimal performance. This paper introduces a novel QAS framework employing hybrid quantum reinforcement learning with quantum curriculum learning strategies, enabling learning agents to tackle increasingly complex quantum circuit design tasks. We benchmark four state-of-the-art classical reinforcement learning algorithms (A2C, PPO, DDQN, TD3) against their quantum-enhanced counterparts (QA2C, QPPO, QDDQN, QTD3) for optimizing variational quantum circuits (VQCs). Our approach progressively increases circuit depth and gate complexity during training, leveraging parameterized quantum circuits as function approximations. To improve learning efficiency and stability, all algorithms, both classical and quantum, are augmented with Prioritized Experience Replay (PER). Experimental results show that quantum-enhanced RL significantly outperforms classical methods. In a 2-qubit environment, PERQDDQN achieves a success probability of 0.46 with 3,000 optimal successes, surpassing classical PERDDQN ($\mathbf{0. 4 2}, \mathbf{2, 4 0 0}$). In the more complex 3-qubit setting, PERQDDQN and PERQTD3 reach success probabilities of 0.47, with optimal success counts of 3,800 and 3,600, respectively, outperforming their classical counterparts. Additionally, we apply our QAS-QTN approach to a classification problem, where the optimized quantum circuit achieves an accuracy of 90.33%, outperforming quantum models consisting of random ansatz. This hybrid classical-quantum approach leads to faster convergence and more efficient quantum circuit designs, demonstrating its potential for advancing automated quantum architecture search.

arXiv:2507.12013•IEEE International Conference on Quantum Computing and Engineering (QCE)•Aug 30, 2025•1 citation

Reinforcement learning-assisted quantum architecture search for variational quantum algorithms

Akash Kundu

A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an $ε$-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.

arXiv:2402.13754•Feb 21, 2024•2 citations

Quantum Reinforcement Learning with Quantum World Model

Peigen Zeng, Ying He, Fei Richard Yu +1 more

Quantum reinforcement learning (QRL) can outperform classical reinforcement learning (RL) by utilizing quantum parallel theory and quantum phenomena such as superposition and entanglement. Although some excellent work has been done on QRL, most existing works either fail to show the exponential advantage of quantum computation over classical computation in terms of performance or are too demanding on quantum devices. In this paper, we provide a novel perspective on combining quantum computing and RL with faster convergence speed and relatively relaxed demands on quantum devices. Specifically, we propose a method to construct a world model with quantum circuit that allows it to interact in a quantum way. In addition, we use Grover's algorithm to efficiently extract high-value information from the quantum world model. Extensive simulation results show that the proposed method can have superior performance compared to classical RL algorithms.

Globecom•Dec 4, 2023•1 citation
Quantum algorithm design plays a crucial role in exploiting the computational advantage of quantum devices. Here we develop a deep-reinforcement-learning based approach for quantum adiabatic algorithm design. Our approach is generically applicable to a class of problems with solution hard-to-find but easy-to-verify, e.g., searching and NP-complete problems. We benchmark this approach in Grover-search and 3-SAT problems, and find that the adiabatic algorithm obtained by our RL approach leads to significant improvement in the resultant success probability. In application to Grover search, our RL design automatically produces an adiabatic quantum algorithm that has the quadratic speedup. We find for all our studied cases that quantitatively the RL-designed algorithm has a better performance compared to the analytically constructed nonlinear Hamiltonian path when the encoding Hamiltonian is solvable, and that this RL-design approach remains applicable even when the nonlinear Hamiltonian path is not analytically available. In 3-SAT we find RL design has fascinating transferability---the adiabatic algorithm obtained by training on a specific choice of clause number leads to better performance consistently over the linear algorithm on different clause numbers. These findings suggest the applicability of reinforcement learning for automated quantum adiabatic algorithm design. Further considering the established complexity equivalence of circuit and adiabatic quantum algorithms, we expect the RL-designed adiabatic algorithm to inspire novel circuit algorithms as well. Our approach is potentially applicable to different quantum hardware from trapped ions and optical lattices to superconducting-qubit devices.
Quantum algorithm design lies in the hallmark of applications of quantum computation and quantum simulation. Here we put forward a deep reinforcement learning (RL) architecture for automated algorithm design in the framework of quantum adiabatic algorithm, where the optimal Hamiltonian path to reach a quantum ground state that encodes a compution problem is obtained by RL techniques. We benchmark our approach in Grover search and 3-SAT problems, and find that the adiabatic algorithm obtained by our RL approach leads to significant improvement in the success probability and computing speedups for both moderate and large number of qubits compared to conventional algorithms. The RL-designed algorithm is found to be qualitatively distinct from the linear algorithm in the resultant distribution of success probability. Considering the established complexity-equivalence of circuit and adiabatic quantum algorithms, we expect the RL-designed adiabatic algorithm to inspire novel circuit algorithms as well. Our approach offers a recipe to design quantum algorithms for generic problems through a machinery RL process, which paves a novel way to automated quantum algorithm design using artificial intelligence, potentially applicable to different quantum simulation and computation platforms from trapped ions and optical lattices to superconducting-qubit devices.
Adiabatic quantum computing (AQC) is a computation protocol to solve difficult problems exploiting quantum advantage, directly applicable to optimization problems. In performing the AQC, different configurations of the Hamiltonian path could lead to dramatic differences in the computation efficiency. It is thus crucial to configure the Hamiltonian path to optimize the computation performance of AQC. Here we apply a reinforcement learning approach to configure AQC for integer programming, where we find the learning process automatically converges to a quantum algorithm that exhibits scaling advantage over the trivial AQC using a linear Hamiltonian path. This reinforcement-learning-based approach for quantum adiabatic algorithm design for integer programming can well be adapted to the quantum resources in different quantum computation devices, due to its built-in flexibility.
Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.
Quantum Architecture Search (QAS) is an emerging field aimed at automating the design of quantum circuits for optimal performance. This paper introduces a novel QAS framework employing hybrid quantum reinforcement learning with quantum curriculum learning strategies, enabling learning agents to tackle increasingly complex quantum circuit design tasks. We benchmark four state-of-the-art classical reinforcement learning algorithms (A2C, PPO, DDQN, TD3) against their quantum-enhanced counterparts (QA2C, QPPO, QDDQN, QTD3) for optimizing variational quantum circuits (VQCs). Our approach progressively increases circuit depth and gate complexity during training, leveraging parameterized quantum circuits as function approximations. To improve learning efficiency and stability, all algorithms, both classical and quantum, are augmented with Prioritized Experience Replay (PER). Experimental results show that quantum-enhanced RL significantly outperforms classical methods. In a 2-qubit environment, PERQDDQN achieves a success probability of 0.46 with 3,000 optimal successes, surpassing classical PERDDQN (0.42,2,400\mathbf{0. 4 2}, \mathbf{2, 4 0 0}0.42,2,400). In the more complex 3-qubit setting, PERQDDQN and PERQTD3 reach success probabilities of 0.47, with optimal success counts of 3,800 and 3,600, respectively, outperforming their classical counterparts. Additionally, we apply our QAS-QTN approach to a classification problem, where the optimized quantum circuit achieves an accuracy of 90.33%, outperforming quantum models consisting of random ansatz. This hybrid classical-quantum approach leads to faster convergence and more efficient quantum circuit designs, demonstrating its potential for advancing automated quantum architecture search.
A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an εεε-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.
Quantum reinforcement learning (QRL) can outperform classical reinforcement learning (RL) by utilizing quantum parallel theory and quantum phenomena such as superposition and entanglement. Although some excellent work has been done on QRL, most existing works either fail to show the exponential advantage of quantum computation over classical computation in terms of performance or are too demanding on quantum devices. In this paper, we provide a novel perspective on combining quantum computing and RL with faster convergence speed and relatively relaxed demands on quantum devices. Specifically, we propose a method to construct a world model with quantum circuit that allows it to interact in a quantum way. In addition, we use Grover's algorithm to efficiently extract high-value information from the quantum world model. Extensive simulation results show that the proposed method can have superior performance compared to classical RL algorithms.