Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2511.16559

Reinforcement learning of quantum circuit architectures for molecular potential energy curves

Maureen Krumtünger, Alissa Wilms, Paul K. Faehrmann, Jens Eisert, Jakob Kottmann, Paolo Andrea Erdman, Sumeet Khatri

quant-phSubmitted Nov 20, 2025• Announced Nov 21, 2025• 0 citations
arXivHTMLPDF

Abstract

Quantum chemistry and optimization are two of the most prominent applications of quantum computers. Variational quantum algorithms have been proposed for solving problems in these domains. However, the design of the quantum circuit ansatz remains a challenge. Of particular interest is developing a method to generate circuits for any given instance of a problem, not merely a circuit tailored to a specific instance of the problem. To this end, we present a reinforcement learning (RL) approach to learning a problem-dependent quantum circuit mapping, which outputs a circuit for the ground state of a Hamiltonian from a given family of parameterized Hamiltonians. For quantum chemistry, our RL framework takes as input a molecule and a discrete set of bond distances, and it outputs a bond-distance-dependent quantum circuit for arbitrary bond distances along the potential energy curve. The inherently non-greedy approach of our RL method contrasts with existing greedy approaches to adaptive, problem-tailored circuit constructions. We demonstrate its effectiveness for the four-qubit and six-qubit lithium hydride molecules, as well as an eight-qubit H$_4$ chain. Our learned circuits are interpretable in a physically meaningful manner, thus paving the way for applying RL to the development of novel quantum circuits for the ground states of large-scale molecular systems.

Comments
16+14 pages, 21 figures. Comments welcome!

Related papers

Benchmarking Adaptive Quantum Circuit Optimization Algorithms for Quantum Chemistry

Waheeda Saib, Xavier Bonet-Monroig, Vedran Dunjko +3 more

In the noisy near-term quantum computing era variational quantum algorithms are promising to explore the boundaries of quantum advantage. One prevalent instance is the variational quantum eigensolver (VQE) employed in quantum chemistry to approximate ground-state energies of molecules. The performance of VQE depends on the design of the parametrized quantum circuit, as well as the optimization algorithm used to optimize those parameters. Circuit optimization methods have been developed to generate quantum circuits to reach accurate approximations to the ground-states. In this work, we conduct an empirical comparison between two prominent adaptive quantum circuit optimization algorithms, namely qubit-ADAPT-VQE and a reinforcement learning (RL) based method, on the problem of finding the ground state energy of Lithium Hydride (LiH). In our experiment, we focus on the circuit depth generated by each circuit optimization method and the number of two-qubit quantum gates in the best-found circuit.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•2 citations

Reinforcement Learning for Variational Quantum Circuits Design

Simone Foderà, Gloria Turati, Riccardo Nembrini +2 more

Variational Quantum Algorithms have emerged as promising tools for solving optimization problems on quantum computers. These algorithms leverage a parametric quantum circuit called ansatz, where its parameters are adjusted by a classical optimizer with the goal of optimizing a certain cost function. However, a significant challenge lies in designing effective circuits for addressing specific problems. In this study, we leverage the powerful and flexible Reinforcement Learning paradigm to train an agent capable of autonomously generating quantum circuits that can be used as ansatzes in variational algorithms to solve optimization problems. The agent is trained on diverse problem instances, including Maximum Cut, Maximum Clique and Minimum Vertex Cover, built from different graph topologies and sizes. Our analysis of the circuits generated by the agent and the corresponding solutions shows that the proposed method is able to generate effective ansatzes. While our goal is not to propose any new specific ansatz, we observe how the agent has discovered a novel family of ansatzes effective for Maximum Cut problems, which we call $R_{yz}$-connected. We study the characteristics of one of these ansatzes by comparing it against state-of-the-art quantum algorithms across instances of varying graph topologies, sizes, and problem types. Our results indicate that the $R_{yz}$-connected circuit achieves high approximation ratios for Maximum Cut problems, further validating our proposed agent. In conclusion, our study highlights the potential of Reinforcement Learning techniques in assisting researchers to design effective quantum circuits which could have applications in a wide number of tasks.

arXiv:2409.05475•Sep 9, 2024•1 citation

Reinforcement learning for optimization of variational quantum circuit architectures

Mateusz Ostaszewski, Lea M. Trenkwalder, Wojciech Masarczyk +2 more

The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.

arXiv:2103.16089•Mar 30, 2021•65 citations

Contextualized Quantum Circuit Optimization via Reinforcement Learning

Jincheng Zhang

Quantum circuit optimization remains a significant challenge in the development of practical quantum computing. Traditional methods often rely on heuristics or exhaustive searches, which are computationally expensive and may not effectively adapt to the specific characteristics of the quantum data being processed. This paper proposes a novel approach leveraging reinforcement learning (RL) to achieve contextualized quantum circuit optimization. The core idea is to train an RL agent to dynamically modify quantum circuits in real-time, guided by a reward function that balances circuit fidelity and execution time. Crucially, the agent incorporates a learned contextual representation of the input data, allowing it to tailor the circuit design to the specific context. We demonstrate the potential of this approach by outlining the key components, the reward function design, and the overall architecture. The potential benefits include improved circuit fidelity, reduced execution time, and the ability to adapt to diverse quantum datasets. This represents a shift from static circuit design to adaptive, context-aware optimization, potentially unlocking the full potential of quantum computation.

•Aug 31, 2026•0 citations

Separating Ansatz Discovery from Deployment on Larger Problems: Reinforcement Learning for Modular Circuit Design

Gloria Turati, Simone Foderà, Riccardo Nembrini +2 more

As quantum computing continues to gain attention, there is growing interest in how classical machine learning can assist quantum workflows in practice. Automated circuit design, sometimes referred to as Quantum Architecture Search (QAS), is a natural application but relies on the ability to model the quantum system to support learning as the number of qubits grows. This challenge is central to QAS, and much of the current literature that proposes new ways to model the ansatz focuses on small systems, often around ten qubits. In this work, we propose a complementary approach that separates a small-scale structure discovery phase, where a reusable modular circuit block is learned on small instances where classical learning is feasible, from a deployment phase, where the blocks are used to create the ansatz required for larger problems. To this end, we introduce Reinforcement Learning for Variational Quantum Circuits (RLVQC), formulating QAS as a sequential decision-making problem. We evaluate our methodology on Quadratic Unconstrained Binary Optimization (QUBO) instances derived from Maximum Cut, Maximum Clique, and Minimum Vertex Cover. Our RLVQC Block model is trained to discover a modular two-qubit block that can generalize QAOA-style methods and that is often beneficial compared to learning non-modular ansatzes. The blocks discovered on n=8 instances remain effective when deployed on larger instances (n=12 and n=16), supporting the feasibility of reusing learned modular structure across problem sizes. While we do not aim to establish a new state-of-the-art solver or an advantage over classical methods, our results provide evidence that modular ansatz structure can be learned on smaller instances and then extended to larger ones without requiring learning on systems with a large number of qubits, where quantum computing becomes interesting but classical computation becomes impractical.

arXiv:2507.16001•Jul 21, 2025•0 citations

Distributed quantum architecture search using multi-agent reinforcement learning

Mikhail Sergeev, Georgii Paradezhenko, Daniil Rabinovich +1 more

Quantum architecture search (QAS) automates the design of parameterized quantum circuits for variational quantum algorithms. The framework finds a well-suited problem-specific structure of a variational ansatz. Among possible implementations of QAS the reinforcement learning (RL) stands out as one of the most promising. Current RL approaches are single-agent-based and show poor scalability with a number of qubits due to the increase of the action space dimension and the computational cost. We propose a novel multi-agent RL algorithm for QAS with each agent acting separately on its own block of a quantum circuit. This procedure allows to significantly accelerate the convergence of the RL-based QAS and reduce its computational cost. We benchmark the proposed algorithm on MaxCut problem on 3-regular graphs and on ground energy estimation for the Schwinger Hamiltonian. In addition, the proposed multi-agent approach naturally fits into the set-up of distributed quantum computing, favoring its implementation on modern intermediate scale quantum devices.

quant-pharXiv:2511.22708•Dec 1, 2025•0 citations
Quantum chemistry and optimization are two of the most prominent applications of quantum computers. Variational quantum algorithms have been proposed for solving problems in these domains. However, the design of the quantum circuit ansatz remains a challenge. Of particular interest is developing a method to generate circuits for any given instance of a problem, not merely a circuit tailored to a specific instance of the problem. To this end, we present a reinforcement learning (RL) approach to learning a problem-dependent quantum circuit mapping, which outputs a circuit for the ground state of a Hamiltonian from a given family of parameterized Hamiltonians. For quantum chemistry, our RL framework takes as input a molecule and a discrete set of bond distances, and it outputs a bond-distance-dependent quantum circuit for arbitrary bond distances along the potential energy curve. The inherently non-greedy approach of our RL method contrasts with existing greedy approaches to adaptive, problem-tailored circuit constructions. We demonstrate its effectiveness for the four-qubit and six-qubit lithium hydride molecules, as well as an eight-qubit H4_44​ chain. Our learned circuits are interpretable in a physically meaningful manner, thus paving the way for applying RL to the development of novel quantum circuits for the ground states of large-scale molecular systems.
In the noisy near-term quantum computing era variational quantum algorithms are promising to explore the boundaries of quantum advantage. One prevalent instance is the variational quantum eigensolver (VQE) employed in quantum chemistry to approximate ground-state energies of molecules. The performance of VQE depends on the design of the parametrized quantum circuit, as well as the optimization algorithm used to optimize those parameters. Circuit optimization methods have been developed to generate quantum circuits to reach accurate approximations to the ground-states. In this work, we conduct an empirical comparison between two prominent adaptive quantum circuit optimization algorithms, namely qubit-ADAPT-VQE and a reinforcement learning (RL) based method, on the problem of finding the ground state energy of Lithium Hydride (LiH). In our experiment, we focus on the circuit depth generated by each circuit optimization method and the number of two-qubit quantum gates in the best-found circuit.
Variational Quantum Algorithms have emerged as promising tools for solving optimization problems on quantum computers. These algorithms leverage a parametric quantum circuit called ansatz, where its parameters are adjusted by a classical optimizer with the goal of optimizing a certain cost function. However, a significant challenge lies in designing effective circuits for addressing specific problems. In this study, we leverage the powerful and flexible Reinforcement Learning paradigm to train an agent capable of autonomously generating quantum circuits that can be used as ansatzes in variational algorithms to solve optimization problems. The agent is trained on diverse problem instances, including Maximum Cut, Maximum Clique and Minimum Vertex Cover, built from different graph topologies and sizes. Our analysis of the circuits generated by the agent and the corresponding solutions shows that the proposed method is able to generate effective ansatzes. While our goal is not to propose any new specific ansatz, we observe how the agent has discovered a novel family of ansatzes effective for Maximum Cut problems, which we call RyzR_{yz}Ryz​-connected. We study the characteristics of one of these ansatzes by comparing it against state-of-the-art quantum algorithms across instances of varying graph topologies, sizes, and problem types. Our results indicate that the RyzR_{yz}Ryz​-connected circuit achieves high approximation ratios for Maximum Cut problems, further validating our proposed agent. In conclusion, our study highlights the potential of Reinforcement Learning techniques in assisting researchers to design effective quantum circuits which could have applications in a wide number of tasks.
The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans{ä}tze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.
Quantum circuit optimization remains a significant challenge in the development of practical quantum computing. Traditional methods often rely on heuristics or exhaustive searches, which are computationally expensive and may not effectively adapt to the specific characteristics of the quantum data being processed. This paper proposes a novel approach leveraging reinforcement learning (RL) to achieve contextualized quantum circuit optimization. The core idea is to train an RL agent to dynamically modify quantum circuits in real-time, guided by a reward function that balances circuit fidelity and execution time. Crucially, the agent incorporates a learned contextual representation of the input data, allowing it to tailor the circuit design to the specific context. We demonstrate the potential of this approach by outlining the key components, the reward function design, and the overall architecture. The potential benefits include improved circuit fidelity, reduced execution time, and the ability to adapt to diverse quantum datasets. This represents a shift from static circuit design to adaptive, context-aware optimization, potentially unlocking the full potential of quantum computation.
As quantum computing continues to gain attention, there is growing interest in how classical machine learning can assist quantum workflows in practice. Automated circuit design, sometimes referred to as Quantum Architecture Search (QAS), is a natural application but relies on the ability to model the quantum system to support learning as the number of qubits grows. This challenge is central to QAS, and much of the current literature that proposes new ways to model the ansatz focuses on small systems, often around ten qubits. In this work, we propose a complementary approach that separates a small-scale structure discovery phase, where a reusable modular circuit block is learned on small instances where classical learning is feasible, from a deployment phase, where the blocks are used to create the ansatz required for larger problems. To this end, we introduce Reinforcement Learning for Variational Quantum Circuits (RLVQC), formulating QAS as a sequential decision-making problem. We evaluate our methodology on Quadratic Unconstrained Binary Optimization (QUBO) instances derived from Maximum Cut, Maximum Clique, and Minimum Vertex Cover. Our RLVQC Block model is trained to discover a modular two-qubit block that can generalize QAOA-style methods and that is often beneficial compared to learning non-modular ansatzes. The blocks discovered on n=8 instances remain effective when deployed on larger instances (n=12 and n=16), supporting the feasibility of reusing learned modular structure across problem sizes. While we do not aim to establish a new state-of-the-art solver or an advantage over classical methods, our results provide evidence that modular ansatz structure can be learned on smaller instances and then extended to larger ones without requiring learning on systems with a large number of qubits, where quantum computing becomes interesting but classical computation becomes impractical.
Quantum architecture search (QAS) automates the design of parameterized quantum circuits for variational quantum algorithms. The framework finds a well-suited problem-specific structure of a variational ansatz. Among possible implementations of QAS the reinforcement learning (RL) stands out as one of the most promising. Current RL approaches are single-agent-based and show poor scalability with a number of qubits due to the increase of the action space dimension and the computational cost. We propose a novel multi-agent RL algorithm for QAS with each agent acting separately on its own block of a quantum circuit. This procedure allows to significantly accelerate the convergence of the RL-based QAS and reduce its computational cost. We benchmark the proposed algorithm on MaxCut problem on 3-regular graphs and on ground energy estimation for the Schwinger Hamiltonian. In addition, the proposed multi-agent approach naturally fits into the set-up of distributed quantum computing, favoring its implementation on modern intermediate scale quantum devices.