We introduce the "quantum circuit daemon" (QC-Daemon), a reinforcement learning agent for compiling quantum device operations aimed at efficient quantum hardware execution. We apply QC-Daemon to the move synthesis problem called the Atom Game, which involves orchestrating parallel circuits on reconfigurable neutral atom arrays. In our numerical simulation, the QC-Daemon is implemented by two different types of transformers with a physically motivated architecture and trained by a reinforcement learning algorithm. We observe a reduction of the logarithmic infidelity for various benchmark problems up to 100 qubits by intelligently changing the layout of atoms. Additionally, we demonstrate the transferability of our approach: a Transformer-based QC-Daemon trained on a diverse set of circuits successfully generalizes its learned strategy to previously unseen circuits.
We introduce the "quantum circuit daemon" (QC-Daemon), a reinforcement learning agent for compiling quantum device operations aimed at efficient quantum hardware execution. We apply QC-Daemon to the move synthesis problem called the Atom Game, which involves orchestrating parallel circuits on reconfigurable neutral atom arrays. In our numerical simulation, the QC-Daemon is implemented by two different types of transformers with a physically motivated architecture and trained by a reinforcement learning algorithm. We observe a reduction of the logarithmic infidelity for various benchmark problems up to 100 qubits by intelligently changing the layout of atoms. Additionally, we demonstrate the transferability of our approach: a Transformer-based QC-Daemon trained on a diverse set of circuits successfully generalizes its learned strategy to previously unseen circuits.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.
Quantum circuit compilation is a crucial step in realizing quantum algorithms on real quantum hardware. Traditionally, compilation methods are largely static, relying on pre-defined mappings and optimizations that fail to account for the inherent noise and variability present in quantum devices. This paper proposes a novel approach leveraging Reinforcement Learning (RL) to dynamically generate and optimize quantum circuits. An RL agent is trained to learn a policy for circuit design, adapting to specific hardware characteristics and real-time measurement data. The agent's goal is to minimize the overall error rate of the compiled circuit. This dynamic compilation framework offers a feedback loop between hardware performance and circuit design, resulting in adaptive and robust quantum computation. The core claim of this work is that traditional static compilation methods are insufficient, and dynamic compilation driven by RL offers a superior solution. The approach presented here represents a significant advancement towards realizing the full potential of quantum computing by mitigating the effects of hardware imperfections. The framework includes a detailed description of the RL agent architecture, the reward function design, and the exploration strategy employed. The effectiveness of the proposed method is demonstrated through simulations, highlighting its ability to outperform static compilation strategies in noisy environments.
Quantum circuit compilation is a critical step in realizing quantum computation, aiming to translate high-level quantum algorithms into a sequence of gates executable on a specific quantum hardware. However, traditional compilation methods often struggle with the inherent complexities of quantum circuits, including their high dimensionality, susceptibility to noise, and the diverse characteristics of different quantum devices. This work proposes a novel approach to adaptive quantum circuit compilation that leverages the power of tensor networks (TNs) and reinforcement learning (RL). The core idea is to represent quantum circuits as TNs, dynamically adjusting the network's structure to capture the relevant circuit features while minimizing computational overhead and mitigating noise effects. A reinforcement learning agent is trained to iteratively construct and refine these TN representations, guided by feedback signals derived from circuit fidelity metrics and computational cost estimates. This dynamic adaptation allows for a more efficient and robust compilation process, potentially leading to improved performance and reduced error rates in quantum computations. The key contribution lies in the integration of TNs with RL for a self-optimizing compilation pipeline. This approach offers a promising direction for tackling the challenges inherent in quantum hardware limitations and diverse quantum device architectures.
Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.