This paper explores the application of reinforcement learning (RL) algorithms to the domain of quantum sensor circuit design. We focus on the design of four qubits quantum circuits that can efficiently generate states with optimal sensitivity, which maximizes the Quantum Fisher Information (QFI). We formulate the design task as a Markov Decision Process, implementing and comparing two distinct RL strategies: pure policy gradient and actor-critic agents to optimize a Ramsey sequence for maximum QFI. The results demonstrate that both agents are able to find quantum circuit designs with an optimal QFI value of 1. However, the agents differ in the gate sequence of the final Quantum sensor circuit configurations that indicate the optimal state. The policy gradient outperforms the actor-critic method in convergence time and the number of gate sequences for the encoding circuit.
This paper explores the application of reinforcement learning (RL) algorithms to the domain of quantum sensor circuit design. We focus on the design of four qubits quantum circuits that can efficiently generate states with optimal sensitivity, which maximizes the Quantum Fisher Information (QFI). We formulate the design task as a Markov Decision Process, implementing and comparing two distinct RL strategies: pure policy gradient and actor-critic agents to optimize a Ramsey sequence for maximum QFI. The results demonstrate that both agents are able to find quantum circuit designs with an optimal QFI value of 1. However, the agents differ in the gate sequence of the final Quantum sensor circuit configurations that indicate the optimal state. The policy gradient outperforms the actor-critic method in convergence time and the number of gate sequences for the encoding circuit.
Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.
As the number of qubits in a sensor increases, the complexity of designing and controlling the quantum circuits grows exponentially. Manually optimizing these circuits becomes infeasible. Optimizing entanglement distribution in large-scale quantum circuits is critical for enhancing the sensitivity and efficiency of quantum sensors (Yi, in Quantum Sci Technol, 045001, 2022, Thomas et al, in 2021). This paper presents an engineering integration of reinforcement learning with tensor-network-based simulation (MPS) for scalable circuit optimization for optimizing quantum sensor circuits with up to 60 qubits. To enable efficient simulation and scalability, we adopt tensor network methods—specifically, the Matrix Product State (MPS) representation—instead of traditional state vector or density matrix approaches. Our reinforcement learning agent learns to restructure circuits to maximize Quantum Fisher Information (QFI) and entanglement entropy while reducing gate counts and circuit depth. Experimental results show consistent improvements, with QFI values approaching 1, entanglement entropy in the 0.8–1.0 range, and up to 90% reduction in depth and gate count. These results highlight the potential of combining quantum machine learning and tensor networks to optimize complex quantum circuits under realistic constraints.
This study proposes a QRA approach for designing optimal Quantum Sensor Circuits (QSCs) to address complex quantum physics problems. The QRA generates QSCs by selecting sequences of gates that maximize the Quantum Fisher Information (QFI) while minimizing the number of gates. The QSCs generated by the QRA are capable of producing entangled quantum states, specifically the squeezed states, by performing generalized Ramsey measurements on qubits. High QFI indicates increased sensitivity to parameter changes, making the circuit useful for quantum state estimation and control tasks. Evaluation of the QRA on a QSC that consists of two qubits and a sequence ofRx,RyandSgates demonstrates its efficiency in generating optimal QSCs with a QFI of 1. This work illustrates the potential computational power of quantum agents for solving quantum physics problems.
We propose a novel Reinforcement Learning (RL) method for optimizing quantum circuits using graph-theoretic simplification rules of ZX-diagrams. The agent, trained using the Proximal Policy Optimization (PPO) algorithm, employs Graph Neural Networks to approximate the policy and value functions. We demonstrate the capacity of our approach by comparing it against the best performing ZX-Calculus-based algorithm for the problem in hand. After training on small Clifford+T circuits of 5-qubits and few tenths of gates, the agent consistently improves the state-of-the-art for this type of circuits, for at least up to 80-qubit and 2100 gates, whilst remaining competitive in terms of computational performance. Additionally, we illustrate the versatility of the agent by incorporating additional optimization routines on the workflow during training, improving the two-qubit gate count state-of-the-art on multiple structured quantum circuits for relevant applications of much larger dimension and different gate distributions than the circuits the agent trains on. This conveys the potential of tailoring the reward function to the specific characteristics of each application and hardware backend. Our approach is a valuable tool for the implementation of quantum algorithms in the near-term intermediate-scale range (NISQ).
This study proposes a GPA for designing optimal Quantum Sensor Circuits (QSCs) to address complex quantum physics problems. The GPA consists of two parts: the Quantum Policy Evaluation (QPE) and the Quantum Policy Improvement (QPI). The QPE performs phase estimation to generate the search space, while the QPI utilizes Grover search and amplitude amplification techniques to efficiently identify an optimal policy that generates optimal QSCs. The GPA generates QSCs by selecting sequences of gates that maximize the Quantum Fisher Information (QFI) while minimizing the number of gates. The QSCs generated by the GPA are capable of producing entangled quantum states, specifically the squeezed states. High QFI indicates increased sensitivity to parameter changes, making the circuit useful for quantum state estimation and control tasks. Evaluation of the GPA on a QSC that consists of two qubits and a sequence of R_x, R_y, and S gates demonstrates its efficiency in generating optimal QSCs with a QFI of 1. Compared to existing quantum agents, the GPA achieves higher QFI with fewer gates, demonstrating a more efficient and scalable approach to the design of QSCs. This work illustrates the potential computational power of quantum agents for solving quantum physics problems
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.