Reinforcement learning for quantum circuit optimization

Akash Kundu, Aritra Sarkar, Prayag Tiwari +1 more

Khoa Dang Tao, Sumin Jin, Muhammad Raza +1 more

The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.

Jincheng Zhang

Quantum circuit optimization remains a significant challenge in the development of practical quantum computing. Traditional methods often rely on heuristics or exhaustive searches, which are computationally expensive and may not effectively adapt to the specific characteristics of the quantum data being processed. This paper proposes a novel approach leveraging reinforcement learning (RL) to achieve contextualized quantum circuit optimization. The core idea is to train an RL agent to dynamically modify quantum circuits in real-time, guided by a reward function that balances circuit fidelity and execution time. Crucially, the agent incorporates a learned contextual representation of the input data, allowing it to tailor the circuit design to the specific context. We demonstrate the potential of this approach by outlining the key components, the reward function design, and the overall architecture. The potential benefits include improved circuit fidelity, reduced execution time, and the ability to adapt to diverse quantum datasets. This represents a shift from static circuit design to adaptive, context-aware optimization, potentially unlocking the full potential of quantum computation.

Hrvoje Kukina, Susanne F. Yelin

Sarvesh Bendre, Samiksha Apake, Ashish Mishra +2 more

Ioana Moflic, Alexandru Paler

Large scale optimisation of quantum circuits is a computationally challenging problem. Reinforcement Learning (RL) is a recent approach for learning strategies to optimise quantum circuits by increasing the reward of an optimisation agent. The reward is a function of the quantum circuit costs, such as gate and qubit counts, or circuit depth. Our goal is to improve the agent’s optimization strategy, by including hints about how quantum circuits are optimized manually: there are situations when the cost of a circuit should be allowed to temporary explode, before applying optimisations which significantly reduce the circuit’s cost. We bring numerical evidence, using Bernstein-Vazirani circuits, to support the advantage of this strategy. Our results are preliminary, and show that allowing cost explosions offers significant advantages for RL training, such as reaching optimum circuits. Cost explosion strategies have the potential to be an essential tool for RL of large-scale quantum circuit optimisation.