This paper presents an innovative way of quantum circuit optimization; we propose an automated superconducting circuit calibration using deep reinforcement learning (ASCC-DRL). A simplified model of a transmon qubit coupled to a cavity resonator is used to demonstrate a quantum circuit. We analyzed the circuit through its equivalent two-mode lumped-element representation of the distributed circuit and derived the equivalent Hamiltonian translation of the circuit with the non-linearity of the Josephson Junctions. We adopted the parametrized qubit junctions and used the component physical value to tune the circuits to maximize the fidelity of the desired entangle state and the resulting quantum circuit state. However, quantum systems are susceptible to environmental parasitics and noise; therefore, a reliable optimization technique is essential to maintain high fidelity across the system. For optimization, we integrate deep reinforcement learning using deep deterministic policy gradient techniques as our mechanism for our AI module to optimize the fidelity of the superconducting quantum circuit. We demonstrate the feasibility of this approach by training the agent in three distinct environments to achieve the highest possible best rewards. We achieved an average fidelity (best rewards) of 0.92, which is 94% of the statically tuned circuit.
This paper presents an innovative way of quantum circuit optimization; we propose an automated superconducting circuit calibration using deep reinforcement learning (ASCC-DRL). A simplified model of a transmon qubit coupled to a cavity resonator is used to demonstrate a quantum circuit. We analyzed the circuit through its equivalent two-mode lumped-element representation of the distributed circuit and derived the equivalent Hamiltonian translation of the circuit with the non-linearity of the Josephson Junctions. We adopted the parametrized qubit junctions and used the component physical value to tune the circuits to maximize the fidelity of the desired entangle state and the resulting quantum circuit state. However, quantum systems are susceptible to environmental parasitics and noise; therefore, a reliable optimization technique is essential to maintain high fidelity across the system. For optimization, we integrate deep reinforcement learning using deep deterministic policy gradient techniques as our mechanism for our AI module to optimize the fidelity of the superconducting quantum circuit. We demonstrate the feasibility of this approach by training the agent in three distinct environments to achieve the highest possible best rewards. We achieved an average fidelity (best rewards) of 0.92, which is 94% of the statically tuned circuit.
Quantum computing promises significant computational advantages over classical methods in certain applications but is currently too unstable for executing complex tasks. High-fidelity quantum circuits are critical to overcoming this challenge, yet they are computationally challenging to design. On current noisy intermediate-scale quantum (NISQ) hardware, shallower circuits with fewer two-qubit gates generally run with higher fidelity. Increased fidelity can be sought by sequentially rewriting the circuit, which involves a vast search space. This costly search space exploration could be circumvented by using reinforcement learning (RL) to teach an agent to rewrite the circuit in a sequential decision-making process. This thesis aims to implement a Deep Q-Network (DQN) algorithm to optimize BernsteinVazirani (BVZ) circuits. The DQN algorithm interacts with a Cirq-based environment. The environment provides valid rewrite actions to the DQN algorithm, which gains a scalar reward emphasizing depth and gate reduction. In the thesis we compare two different state encodings as input for the neural network: a compact state summarizing vector representation with three features, and an image-like encoding that maps circuit state topology with tokens through a fixed embedding. Both input encodings use the same action rewrite set and reward design. The network is implemented in PyTorch and trained with Adam optimizer, and runs on GPU via CUDA. Empirically, the agent shows promising initial optimizing performance. On several BVZ circuits with varying qubit count and topology, the agent drives the depth toward the theoretical optimum of three. The simplistic vector encoding already reaches reasonable circuit depth plateaus, with the image-like encoding together with hyperparameter tuning improving reliability, showing avenues for continued improvement. We conclude that DQN is a viable foundation for automated circuit optimization on BVZ circuits. To achieve reliable circuit optimization at scale, future research should investigate even richer encoding methods and reward function shaping. Finally, the results support applying the DQN to be applied to circuit families beyond BVZ.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
Quantum compilation requires solving the initial mapping problem, which is crucial for optimizing qubit assignment and minimizing fidelity in compiled quantum circuits. This study investigates the application of Deep Reinforcement Learning (DRL) to address initial mapping across various qubit topologies, while considering varying qubit connection fidelities. By leveraging policy gradient algorithms-Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO), and PPO with action masking-DRL agents effectively compute high-quality mappings for both small-and medium-scale quantum architectures, but their efficiency decreases as the system size grows, highlighting the need for further optimization strategies. Fine-tuning hyperparameters and incorporating action masking are shown to be essential for preventing illegal actions and enhancing mapping accuracy, particularly in larger systems. This research also aids the continued development of initial mapping techniques by introducing a comprehensive qubit connectivity database for systematic evaluation of DRL methods across diverse architectures.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.
Qubit routing is the process of transforming quantum circuits to ensure that all two-qubit operations obey the connectivity constraints of a target architecture. The severe limitations of NISQ computers make qubit routing a vital compilation step. We propose a method to route quantum circuits based on deep reinforcement learning (RL). Moreover, we augment our approach with device calibration data to enable routing decisions based on the reliability of individual gates and we improve upon a previous technique for encoding circuits as a numerical matrix. We also implement enhancements such as BRIDGE operations, qubit embeddings, and commutation analysis. Using proximal policy optimization (PPO), we trained agents for five IBM topologies with between 5 and 27 qubits. For random circuits, the proposed strategy reduced the number of additional two-qubit gates by up to 37.3% and improved estimated success probability by up to 26.8% compared to the best algorithm from the Qiskit framework. For a dataset of realistic circuits, we achieved up to 27.5% and 15.2% improvements in gate count and circuit fidelity, respectively. We found no significant advantage from using calibration-aware models for IBM architectures, which deserves further exploration. Our work helps cement RL as a valuable qubit routing strategy, serving as another stepping stone toward practical applications for quantum computing.