Quantum operator synthesis is a key task in quantum transpilers, enabling the translation of high-level algorithms into hardware-executable gate sequences by decomposing arbitrary operators into native gates while optimizing for efficiency and constraints. Recently, reinforcement learning (RL) has emerged as a promising approach for quantum circuit synthesis, particularly for Clifford operators. However, the crisp-reward functions used in current RL frameworks may fail to capture the subtleties of complex quantum circuit synthesis tasks, leading to suboptimal designs. In this paper, this gap is addressed by integrating a fuzzy reward into an RL algorithm to synthesize Clifford operators. The fuzzy reward is computed by a fuzzy rule-based system, designed from the analysis of the error rate in current IBM quantum devices. The experiments show that our fuzzy reward accelerates convergence, achieving the same performance in terms of depth and controlled-X (CX) depth as the crisp-reward algorithm, but with fewer training steps. This highlights the efficiency of fuzzy rewards and their potential to improve the synthesis capabilities of RL systems.
Quantum operator synthesis is a key task in quantum transpilers, enabling the translation of high-level algorithms into hardware-executable gate sequences by decomposing arbitrary operators into native gates while optimizing for efficiency and constraints. Recently, reinforcement learning (RL) has emerged as a promising approach for quantum circuit synthesis, particularly for Clifford operators. However, the crisp-reward functions used in current RL frameworks may fail to capture the subtleties of complex quantum circuit synthesis tasks, leading to suboptimal designs. In this paper, this gap is addressed by integrating a fuzzy reward into an RL algorithm to synthesize Clifford operators. The fuzzy reward is computed by a fuzzy rule-based system, designed from the analysis of the error rate in current IBM quantum devices. The experiments show that our fuzzy reward accelerates convergence, achieving the same performance in terms of depth and controlled-X (CX) depth as the crisp-reward algorithm, but with fewer training steps. This highlights the efficiency of fuzzy rewards and their potential to improve the synthesis capabilities of RL systems.
The optimization of the limited resources of current quantum devices hinges on the implementation of quantum-compiling techniques. The process of transforming high-level quantum algorithms into executable quantum circuits has a significant impact on the performance and fidelity of computations. One of the most critical aspects of the compiling stack is the synthesis of quantum circuits, which aims to minimize the depth and number of gates, as well as the error rate. Nevertheless, the optimal synthesis algorithms are extremely computationally expensive. This is why synthesis algorithms based on reinforcement learning (RL) have recently been introduced. However, such algorithms are constrained by the topology of quantum processors, necessitating retraining of the RL agent for different topologies. To achieve a balance between adaptability, optimality, and computational cost in RL-based quantum circuit synthesis techniques, this work proposes an RL-based quantum circuit synthesis algorithm of Clifford operators that addresses different topologies simultaneously. Experimental tests demonstrate that our synthesizer outperforms state-of-the-art greedy techniques while being significantly faster than boolean satisfiability solvers (SAT).
A reinforcement learning (RL) framework is introduced for the efficient synthesis of quantum circuits that generate specified target quantum states from a fixed initial state, addressing a central challenge in both the Noisy Intermediate-Scale Quantum (NISQ) era and future fault-tolerant quantum computing. The approach utilizes tabular Q-learning, based on action sequences, within a discretized quantum state space, to effectively manage the exponential growth of the space dimension. The framework introduces a hybrid reward mechanism, combining a static, domain-informed reward that guides the agent toward the target state with customizable dynamic penalties that discourage inefficient circuit structures such as gate congestion and redundant state revisits. This is a circuit-aware reward, in contrast to the current trend of works on this topic, which are primarily fidelity-based. By leveraging sparse matrix representations and state-space discretization, the method enables practical navigation of high-dimensional environments while minimizing computational overhead. Benchmarking on graph-state preparation tasks for up to seven qubits, we demonstrate that the algorithm consistently discovers minimal-depth circuits with optimized gate counts. Moreover, extending the framework to a universal gate set still yields low depth circuits, highlighting the algorithm’s robustness and adaptability. The results confirm that this RL-driven approach, with our completely circuit-aware method, efficiently explores the complex quantum state space and synthesizes near-optimal quantum circuits, providing a resource-efficient foundation for quantum circuit optimization.
One of the most important initial steps in quantum computing is high-fidelity quantum state preparation, because errors in it can affect the final computational result. Therefore, one of the major challenges is to design a quantum circuit that can achieve the desired target state accurately with a smaller number of gates. Traditionally, circuits are designed using predefined rules and mathematical decompositions, but this makes it difficult to design circuits for different and complex target states. This paper presents TATVA, a reinforcement learning (RL) system for synthesizing quantum circuits. TATVA views circuit synthesis as a sequential decision problem in which the agents select one gate at a time and calculate the fidelity after each applied gate, with the gate producing the highest fidelity selected for the circuit. The system introduces a parallel architecture that uses two RL agents, Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), along with Qiskit's statevector simulator. The system can synthesize up to 5 qubits while achieving a fidelity of 0.999999 and producing compact circuits. After achieving the desired fidelity, it performs a post-hoc optimization step that reduces the circuit depth while preserving the achieved fidelity. The circuit is finalized using fidelity, success rate, gate count, and circuit depth. Target states that are not used during training are used to test whether the system has learned effectively and can generalize beyond the training states. This approach aims to enable automatic quantum circuit synthesis with high fidelity using reinforcement learning.
This paper demonstrates the integration of Reinforcement Learning (RL) into quantum transpiling workflows, significantly enhancing the synthesis and routing of quantum circuits. By employing RL, we achieve near-optimal synthesis of Linear Function, Clifford, and Permutation circuits, up to 9, 11 and 65 qubits respectively, while being compatible with native device instruction sets and connectivity constraints, and orders of magnitude faster than optimization methods such as SAT solvers. We also achieve significant reductions in two-qubit gate depth and count for circuit routing up to 133 qubits with respect to other routing heuristics such as SABRE. We find the method to be efficient enough to be useful in practice in typical quantum transpiling pipelines. Our results set the stage for further AI-powered enhancements of quantum computing workflows.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
Reinforcement learning (RL) shows promise for automated quantum circuit design but often stalls due to a “fidelity trap”: by optimizing only state fidelity, agents overlook entanglement structure and become stranded in suboptimal, overly complex circuits, resulting in significantly reduced search efficiency. In this paper, we propose a scheme that overcomes this barrier by implementing an entanglement-aware learning framework and enhancing the agent's reward function with a direct, quantitative measure of entanglement. This approach offers a more comprehensive physical description of the state space. We demonstrate the efficacy of this principle on three- and four-qubit state-synthesis tasks within an expanded gate set. For this problem, where the fidelity-driven agent systematically fails to discover the minimal-depth circuit, our entanglement-aware agent consistently succeeds. This transformative result is highly robust against variations in initial random seeds and extends to multiqubit systems even in the presence of noise. Our findings establish a generalizable principle that incorporating entanglement as an auxiliary reward can significantly enhance RL-based solutions for a broad class of fidelity-centric tasks in quantum physics and pave the way for scalable, automated discovery on near-term quantum devices.