In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.
In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.
A reinforcement learning (RL) framework is introduced for the efficient synthesis of quantum circuits that generate specified target quantum states from a fixed initial state, addressing a central challenge in both the Noisy Intermediate-Scale Quantum (NISQ) era and future fault-tolerant quantum computing. The approach utilizes tabular Q-learning, based on action sequences, within a discretized quantum state space, to effectively manage the exponential growth of the space dimension. The framework introduces a hybrid reward mechanism, combining a static, domain-informed reward that guides the agent toward the target state with customizable dynamic penalties that discourage inefficient circuit structures such as gate congestion and redundant state revisits. This is a circuit-aware reward, in contrast to the current trend of works on this topic, which are primarily fidelity-based. By leveraging sparse matrix representations and state-space discretization, the method enables practical navigation of high-dimensional environments while minimizing computational overhead. Benchmarking on graph-state preparation tasks for up to seven qubits, we demonstrate that the algorithm consistently discovers minimal-depth circuits with optimized gate counts. Moreover, extending the framework to a universal gate set still yields low depth circuits, highlighting the algorithm’s robustness and adaptability. The results confirm that this RL-driven approach, with our completely circuit-aware method, efficiently explores the complex quantum state space and synthesizes near-optimal quantum circuits, providing a resource-efficient foundation for quantum circuit optimization.
The optimization of the limited resources of current quantum devices hinges on the implementation of quantum-compiling techniques. The process of transforming high-level quantum algorithms into executable quantum circuits has a significant impact on the performance and fidelity of computations. One of the most critical aspects of the compiling stack is the synthesis of quantum circuits, which aims to minimize the depth and number of gates, as well as the error rate. Nevertheless, the optimal synthesis algorithms are extremely computationally expensive. This is why synthesis algorithms based on reinforcement learning (RL) have recently been introduced. However, such algorithms are constrained by the topology of quantum processors, necessitating retraining of the RL agent for different topologies. To achieve a balance between adaptability, optimality, and computational cost in RL-based quantum circuit synthesis techniques, this work proposes an RL-based quantum circuit synthesis algorithm of Clifford operators that addresses different topologies simultaneously. Experimental tests demonstrate that our synthesizer outperforms state-of-the-art greedy techniques while being significantly faster than boolean satisfiability solvers (SAT).
Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.
The advancement of quantum computing into the utility scale requires compilation frameworks that can effectively manage the discrepancy between high-level algorithmic intent and the low-level physical constraints of contemporary hardware. Quantum circuit partitioning is a pivotal stage in this compilation pipeline, particularly when leveraging high-performance synthesis tools that are computationally bounded by the number of qubits. Existing partitioning approaches, such as ScanPartitioner [19] and QuickPartitioner [21], while effective, do not leverage structural patterns in circuits, limiting their ability to make globally informed local partitioning decisions. To address this gap, we propose a novel structural-aware quantum circuit partitioning method using a reinforcement learning (RL) framework that harnesses global circuit knowledge to guide local partitioning decisions, enabling more optimization opportunities at the sub-circuit level. Experimental results on benchmark circuits transpiled to satisfy IBM quantum hardware constraints show that our approach reduces the native two-qubit gate count compared to existing quantum circuit partitioners (ScanPartitioner and QuickPartitioner) with an average two-qubit gate reduction of 18.55% over baselines. This research establishes a scalable and robust methodology for partitioning quantum circuits, bridging the gap between exact and approximate synthesis in the Noisy Intermediate-Scale Quantum (NISQ) era and beyond.
Qubit routing is the process of transforming quantum circuits to ensure that all two-qubit operations obey the connectivity constraints of a target architecture. The severe limitations of NISQ computers make qubit routing a vital compilation step. We propose a method to route quantum circuits based on deep reinforcement learning (RL). Moreover, we augment our approach with device calibration data to enable routing decisions based on the reliability of individual gates and we improve upon a previous technique for encoding circuits as a numerical matrix. We also implement enhancements such as BRIDGE operations, qubit embeddings, and commutation analysis. Using proximal policy optimization (PPO), we trained agents for five IBM topologies with between 5 and 27 qubits. For random circuits, the proposed strategy reduced the number of additional two-qubit gates by up to 37.3% and improved estimated success probability by up to 26.8% compared to the best algorithm from the Qiskit framework. For a dataset of realistic circuits, we achieved up to 27.5% and 15.2% improvements in gate count and circuit fidelity, respectively. We found no significant advantage from using calibration-aware models for IBM architectures, which deserves further exploration. Our work helps cement RL as a valuable qubit routing strategy, serving as another stepping stone toward practical applications for quantum computing.
One of the most important initial steps in quantum computing is high-fidelity quantum state preparation, because errors in it can affect the final computational result. Therefore, one of the major challenges is to design a quantum circuit that can achieve the desired target state accurately with a smaller number of gates. Traditionally, circuits are designed using predefined rules and mathematical decompositions, but this makes it difficult to design circuits for different and complex target states. This paper presents TATVA, a reinforcement learning (RL) system for synthesizing quantum circuits. TATVA views circuit synthesis as a sequential decision problem in which the agents select one gate at a time and calculate the fidelity after each applied gate, with the gate producing the highest fidelity selected for the circuit. The system introduces a parallel architecture that uses two RL agents, Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), along with Qiskit's statevector simulator. The system can synthesize up to 5 qubits while achieving a fidelity of 0.999999 and producing compact circuits. After achieving the desired fidelity, it performs a post-hoc optimization step that reduces the circuit depth while preserving the achieved fidelity. The circuit is finalized using fidelity, success rate, gate count, and circuit depth. Target states that are not used during training are used to test whether the system has learned effectively and can generalize beyond the training states. This approach aims to enable automatic quantum circuit synthesis with high fidelity using reinforcement learning.