Reinforcement learning (RL) is a promising method for quantum circuit optimisation. However, the state space that has to be explored by an RL agent is extremely large when considering all the possibilities in which a quantum circuit can be transformed through local rewrite operations. This state space explosion slows down the learning of RL-based optimisation strategies. We present for the first time how to use graph neural network (GNN) autoencoders for the optimisation of quantum circuits. We construct directed acyclic graphs from the quantum circuits, encode the graphs and use the encodings to represent RL states. We illustrate our proof of concept implementation on Bernstein-Vazirani circuits and, from preliminary results, we conclude that our autoencoder approach: a) maintains the optimality of the original RL method; b) reduces by 20 \% the size of the table that encodes the learned optimisation strategy. Our method is the first realistic first step towards very large scale RL quantum circuit optimisation.
Reinforcement learning (RL) is a promising method for quantum circuit optimisation. However, the state space that has to be explored by an RL agent is extremely large when considering all the possibilities in which a quantum circuit can be transformed through local rewrite operations. This state space explosion slows down the learning of RL-based optimisation strategies. We present for the first time how to use graph neural network (GNN) autoencoders for the optimisation of quantum circuits. We construct directed acyclic graphs from the quantum circuits, encode the graphs and use the encodings to represent RL states. We illustrate our proof of concept implementation on Bernstein-Vazirani circuits and, from preliminary results, we conclude that our autoencoder approach: a) maintains the optimality of the original RL method; b) reduces by 20 % the size of the table that encodes the learned optimisation strategy. Our method is the first realistic first step towards very large scale RL quantum circuit optimisation.
Quantum computing is currently strongly limited by the impact of noise, in particular introduced by the application of two-qubit gates. For this reason, reducing the number of two-qubit gates is of paramount importance on noisy intermediate-scale quantum hardware. To advance towards more reliable quantum computing, we introduce a framework based on ZX calculus, graph-neural networks and reinforcement learning for quantum circuit optimization. By combining reinforcement learning and tree search, our method addresses the challenge of selecting optimal sequences of ZX calculus rewrite rules. Instead of relying on existing heuristic rules for minimizing circuits, our method trains a novel reinforcement learning policy that directly operates on ZX-graphs, therefore allowing us to search through the space of all possible circuit transformations to find a circuit significantly minimizing the number of CNOT gates. This way we can scale beyond hard-coded rules towards discovering arbitrary optimization rules. We demonstrate our method's competetiveness with state-of-the-art circuit optimizers and generalization capabilities on large sets of diverse random circuits.
Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.
Successfully harnessing the power of present-day quantum computing hardware requires efficient utilisation of the limited available resources. Most notably, in the quantum circuit model for computation, circuit depth is severely limited by noisy equipment. Techniques to optimise and reduce depth are consequently necessary. Various approaches exist, but many suffer from poor scalability in practice. In a search for better techniques, this thesis extends preliminary research efforts to implement a quantum circuit compressor or optimiser that utilises a graph variational autoencoder to manipulate circuits in a computationally more manageable latent space. Such latent spaces aim to capture and compress relevant information into far fewer dimensions. This reduces the computational cost of search space exploration and sidesteps the combinatorial explosion that traditional optimisers typically face. As an intermediate implementation and demonstration of the technique, the work presented in this thesis does not yet optimise circuits; it does, however, successfully compress chosen subsets of quantum circuits into a low-dimensional space with a near-100% perfect reconstruction rate. Further work would extend this to additionally manipulate—and eventually optimise—the compressed circuits.
Quantum Computing (QC) stands to revolutionize computing, but is currently still limited. To develop and test quantum algorithms today, quantum circuits are often simulated on classical computers. Simulating a complex quantum circuit requires computing the contraction of a large network of tensors. The order (path) of contraction can have a drastic effect on the computing cost, but finding an efficient order is a challenging combinatorial optimization problem. We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges and obtain significant improvements over state-of-the-art techniques in three varieties of circuits, including the largest scale networks used in contemporary QC.
Large scale optimisation of quantum circuits is a computationally challenging problem. Reinforcement Learning (RL) is a recent approach for learning strategies to optimise quantum circuits by increasing the reward of an optimisation agent. The reward is a function of the quantum circuit costs, such as gate and qubit counts, or circuit depth. Our goal is to improve the agent’s optimization strategy, by including hints about how quantum circuits are optimized manually: there are situations when the cost of a circuit should be allowed to temporary explode, before applying optimisations which significantly reduce the circuit’s cost. We bring numerical evidence, using Bernstein-Vazirani circuits, to support the advantage of this strategy. Our results are preliminary, and show that allowing cost explosions offers significant advantages for RL training, such as reaching optimum circuits. Cost explosion strategies have the potential to be an essential tool for RL of large-scale quantum circuit optimisation.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.