Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2103.16089

Reinforcement learning for optimization of variational quantum circuit architectures

Mateusz Ostaszewski, Lea M. Trenkwalder, Wojciech Masarczyk, Eleanor Scerri, Vedran Dunjko

Published Mar 30, 2021• 65 citations
arXivHTMLPDF

Abstract

The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.

DOI
10.48550/arxiv.2103.16089

Related papers

Benchmarking Adaptive Quantum Circuit Optimization Algorithms for Quantum Chemistry

Waheeda Saib, Xavier Bonet-Monroig, Vedran Dunjko +3 more

In the noisy near-term quantum computing era variational quantum algorithms are promising to explore the boundaries of quantum advantage. One prevalent instance is the variational quantum eigensolver (VQE) employed in quantum chemistry to approximate ground-state energies of molecules. The performance of VQE depends on the design of the parametrized quantum circuit, as well as the optimization algorithm used to optimize those parameters. Circuit optimization methods have been developed to generate quantum circuits to reach accurate approximations to the ground-states. In this work, we conduct an empirical comparison between two prominent adaptive quantum circuit optimization algorithms, namely qubit-ADAPT-VQE and a reinforcement learning (RL) based method, on the problem of finding the ground state energy of Lithium Hydride (LiH). In our experiment, we focus on the circuit depth generated by each circuit optimization method and the number of two-qubit quantum gates in the best-found circuit.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•2 citations

Investigation of Automated Design of Quantum Circuits for Imaginary Time Evolution Methods Using Deep Reinforcement Learning

Ryo Suzuki, Shohei Watabe

Efficient ground state search is fundamental to advancing combinatorial optimization problems and quantum chemistry. While the Variational Imaginary Time Evolution (VITE) method offers a useful alternative to Variational Quantum Eigensolver (VQE), and Quantum Approximate Optimization Algorithm (QAOA), its implementation on Noisy Intermediate-Scale Quantum (NISQ) devices is severely limited by the gate counts and depth of manually designed ansatz. Here, we present an automated framework for VITE circuit design using Double Deep-Q Networks (DDQN). Our approach treats circuit construction as a multi-objective optimization problem, simultaneously minimizing energy expectation values and optimizing circuit complexity. By introducing adoptive thresholds, we demonstrate significant hardware overhead reductions. In Max-Cut problems, our agent autonomously discovered circuits with approximately 37\% fewer gates and 43\% less depth than standard hardware-efficient ansatz on average. For molecular hydrogen ($H_2$), the DDQN also achieved the Full-CI limit, with maintaining a significantly shallower circuit. These results suggest that deep reinforcement learning can be helpful to find non-intuitive, optimal circuit structures, providing a pathway toward efficient, hardware-aware quantum algorithm design.

quant-pharXiv:2604.07951•Apr 10, 2026•0 citations

Reinforcement learning of quantum circuit architectures for molecular potential energy curves

Maureen Krumtünger, Alissa Wilms, Paul K. Faehrmann +4 more

Quantum chemistry and optimization are two of the most prominent applications of quantum computers. Variational quantum algorithms have been proposed for solving problems in these domains. However, the design of the quantum circuit ansatz remains a challenge. Of particular interest is developing a method to generate circuits for any given instance of a problem, not merely a circuit tailored to a specific instance of the problem. To this end, we present a reinforcement learning (RL) approach to learning a problem-dependent quantum circuit mapping, which outputs a circuit for the ground state of a Hamiltonian from a given family of parameterized Hamiltonians. For quantum chemistry, our RL framework takes as input a molecule and a discrete set of bond distances, and it outputs a bond-distance-dependent quantum circuit for arbitrary bond distances along the potential energy curve. The inherently non-greedy approach of our RL method contrasts with existing greedy approaches to adaptive, problem-tailored circuit constructions. We demonstrate its effectiveness for the four-qubit and six-qubit lithium hydride molecules, as well as an eight-qubit H$_4$ chain. Our learned circuits are interpretable in a physically meaningful manner, thus paving the way for applying RL to the development of novel quantum circuits for the ground states of large-scale molecular systems.

quant-pharXiv:2511.16559•Nov 21, 2025•0 citations

Reinforcement learning-assisted quantum architecture search for variational quantum algorithms

Akash Kundu

A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an $ε$-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.

arXiv:2402.13754•Feb 21, 2024•2 citations

NN-AE-VQE: Neural network parameter prediction on autoencoded variational quantum eigensolvers

Koen J. Mesman, Yinglu Tang, Matthias Möller +2 more

A longstanding computational challenge is the accurate simulation of many-body particle systems. Especially for deriving key characteristics of high-impact but complex systems such as battery materials and high entropy alloys (HEA). While simple models allow for simulations of the required scale, these methods often fail to capture the complex dynamics that determine the characteristics. A long-theorized approach is to use quantum computers for this purpose, which allows for a more efficient encoding of quantum mechanical systems. In recent years, the field of quantum computing has become significantly more mature. Furthermore, the rise in integration of machine learning with quantum computing further pushes to a near-term advantage. In this work we aim to improve the well-established quantum computing method for calculating the inter-atomic potential, the variational quantum eigensolver, by presenting an auto-encoded VQE with neural-network predictions: NN-AE-VQE. We apply a quantum autoencoder for a compressed quantum state representation of the atomic system, to which a naive circuit ansatz is applied. This reduces the number of circuit parameters to optimize, while still minimal reduction in accuracy. Additionally, we train a classical neural network to predict the circuit parameters to avoid computationally expensive parameter optimization. We demonstrate these methods on a H2 molecule, achieving chemical accuracy. We believe this method shows promise of efficiently capturing highly accurate systems while omitting current bottlenecks of variational quantum algorithms. Finally, we explore options for exploiting the algorithm structure and further algorithm improvements.

arXiv:2411.15667•Nov 23, 2024•1 citation

Hybrid Action Reinforcement Learning for Quantum Architecture Search

Jiayang Niu, Yan Wang, Jie Li +4 more

Reinforcement learning-based Quantum Architecture Search (QAS) offers a promising avenue for automating the design of variational quantum circuits, but existing methods typically decouple discrete structure search from continuous parameter optimization, resulting in inefficient or brittle solutions. We propose HyRLQAS (Hybrid-Action Reinforcement Learning for Quantum Architecture Search), a unified reinforcement learning framework that jointly learns gate placement and parameter initialization within a hybrid discrete-continuous action space, while enabling dynamic refinement of previously placed gates. Trained in a variational quantum eigensolver setting, the agent constructs circuits that directly optimize molecular ground-state energies. Across multiple molecular benchmarks, HyRLQAS demonstrates strong and competitive performance against state-of-the-art QAS methods, achieving lower energy errors with fewer gates. Notably, HyRLQAS reaches chemical-accuracy-level convergence down to 1e-8 energy error after classical optimization, and policy-guided initialization reduces the iteration count of downstream classical optimizers. These results demonstrate that hybrid-action reinforcement learning provides a principled and effective mechanism for coupling circuit topology design with optimization-aware parameterization.

quant-pharXiv:2511.04967•Nov 10, 2025•0 citations
The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.
In the noisy near-term quantum computing era variational quantum algorithms are promising to explore the boundaries of quantum advantage. One prevalent instance is the variational quantum eigensolver (VQE) employed in quantum chemistry to approximate ground-state energies of molecules. The performance of VQE depends on the design of the parametrized quantum circuit, as well as the optimization algorithm used to optimize those parameters. Circuit optimization methods have been developed to generate quantum circuits to reach accurate approximations to the ground-states. In this work, we conduct an empirical comparison between two prominent adaptive quantum circuit optimization algorithms, namely qubit-ADAPT-VQE and a reinforcement learning (RL) based method, on the problem of finding the ground state energy of Lithium Hydride (LiH). In our experiment, we focus on the circuit depth generated by each circuit optimization method and the number of two-qubit quantum gates in the best-found circuit.
Efficient ground state search is fundamental to advancing combinatorial optimization problems and quantum chemistry. While the Variational Imaginary Time Evolution (VITE) method offers a useful alternative to Variational Quantum Eigensolver (VQE), and Quantum Approximate Optimization Algorithm (QAOA), its implementation on Noisy Intermediate-Scale Quantum (NISQ) devices is severely limited by the gate counts and depth of manually designed ansatz. Here, we present an automated framework for VITE circuit design using Double Deep-Q Networks (DDQN). Our approach treats circuit construction as a multi-objective optimization problem, simultaneously minimizing energy expectation values and optimizing circuit complexity. By introducing adoptive thresholds, we demonstrate significant hardware overhead reductions. In Max-Cut problems, our agent autonomously discovered circuits with approximately 37\% fewer gates and 43\% less depth than standard hardware-efficient ansatz on average. For molecular hydrogen (H2H_2H2​), the DDQN also achieved the Full-CI limit, with maintaining a significantly shallower circuit. These results suggest that deep reinforcement learning can be helpful to find non-intuitive, optimal circuit structures, providing a pathway toward efficient, hardware-aware quantum algorithm design.
Quantum chemistry and optimization are two of the most prominent applications of quantum computers. Variational quantum algorithms have been proposed for solving problems in these domains. However, the design of the quantum circuit ansatz remains a challenge. Of particular interest is developing a method to generate circuits for any given instance of a problem, not merely a circuit tailored to a specific instance of the problem. To this end, we present a reinforcement learning (RL) approach to learning a problem-dependent quantum circuit mapping, which outputs a circuit for the ground state of a Hamiltonian from a given family of parameterized Hamiltonians. For quantum chemistry, our RL framework takes as input a molecule and a discrete set of bond distances, and it outputs a bond-distance-dependent quantum circuit for arbitrary bond distances along the potential energy curve. The inherently non-greedy approach of our RL method contrasts with existing greedy approaches to adaptive, problem-tailored circuit constructions. We demonstrate its effectiveness for the four-qubit and six-qubit lithium hydride molecules, as well as an eight-qubit H4_44​ chain. Our learned circuits are interpretable in a physically meaningful manner, thus paving the way for applying RL to the development of novel quantum circuits for the ground states of large-scale molecular systems.
A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an εεε-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.
A longstanding computational challenge is the accurate simulation of many-body particle systems. Especially for deriving key characteristics of high-impact but complex systems such as battery materials and high entropy alloys (HEA). While simple models allow for simulations of the required scale, these methods often fail to capture the complex dynamics that determine the characteristics. A long-theorized approach is to use quantum computers for this purpose, which allows for a more efficient encoding of quantum mechanical systems. In recent years, the field of quantum computing has become significantly more mature. Furthermore, the rise in integration of machine learning with quantum computing further pushes to a near-term advantage. In this work we aim to improve the well-established quantum computing method for calculating the inter-atomic potential, the variational quantum eigensolver, by presenting an auto-encoded VQE with neural-network predictions: NN-AE-VQE. We apply a quantum autoencoder for a compressed quantum state representation of the atomic system, to which a naive circuit ansatz is applied. This reduces the number of circuit parameters to optimize, while still minimal reduction in accuracy. Additionally, we train a classical neural network to predict the circuit parameters to avoid computationally expensive parameter optimization. We demonstrate these methods on a H2 molecule, achieving chemical accuracy. We believe this method shows promise of efficiently capturing highly accurate systems while omitting current bottlenecks of variational quantum algorithms. Finally, we explore options for exploiting the algorithm structure and further algorithm improvements.
Reinforcement learning-based Quantum Architecture Search (QAS) offers a promising avenue for automating the design of variational quantum circuits, but existing methods typically decouple discrete structure search from continuous parameter optimization, resulting in inefficient or brittle solutions. We propose HyRLQAS (Hybrid-Action Reinforcement Learning for Quantum Architecture Search), a unified reinforcement learning framework that jointly learns gate placement and parameter initialization within a hybrid discrete-continuous action space, while enabling dynamic refinement of previously placed gates. Trained in a variational quantum eigensolver setting, the agent constructs circuits that directly optimize molecular ground-state energies. Across multiple molecular benchmarks, HyRLQAS demonstrates strong and competitive performance against state-of-the-art QAS methods, achieving lower energy errors with fewer gates. Notably, HyRLQAS reaches chemical-accuracy-level convergence down to 1e-8 energy error after classical optimization, and policy-guided initialization reduces the iteration count of downstream classical optimizers. These results demonstrate that hybrid-action reinforcement learning provides a principled and effective mechanism for coupling circuit topology design with optimization-aware parameterization.