Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2210.06784 · IEEE/ACM Symposium on Edge Computing (SEC)

Efficient circuit implementation for coined quantum walks on binary trees and application to reinforcement learning

Thomas Mullor, David Vigouroux, Louis Béthune

Published Dec 1, 2022• 1 citation
arXivHTMLPDF

Abstract

Quantum walks on binary trees are used in many quantum algorithms to achieve important speedup over classical algorithms. The formulation of this kind of algorithms as quantum circuit presents the advantage of being easily readable, executable on circuit based quantum computers and simulators and optimal on the usage of resources. We propose a strategy to compose quantum circuit that performs quantum walk on binary trees following universal gate model quantum computation principles. We give a particular attention to NAND formula evaluation algorithm as it could have many applications in game theory and reinforcement learning. We therefore propose an application of this algorithm and show how it can be used to train a quantum reinforcement learning agent in a two player game environment.

DOI
10.1109/sec54971.2022.00066

Related papers

Quantum 2-Player Games and Realizations with Circuits

Jinliang Zhang, Tian Chen, Wenyuan Deng +2 more

Game theory problems are widely applied in many research areas such as computer science and finance, with the key issue being how to quickly make decisions. Here, we present a novel quantum algorithm for game theory problems based on a continuous quantum walk. Our algorithm exhibits quantum advantage compared to classical game algorithms. Furthermore, we exploit the analogy between the wave function of the Schrödinger equation and the voltage in Kirchhoff's law to effectively translate the design of quantum game trees into classical circuit networks. We have theoretically simulated the quantum game trees and experimentally validated the quantum functionality speedup on classical circuit networks. Due to the robust scalability and stability inherent in classical circuit networks, quantum game trees implemented within this framework hold promise for addressing more intricate application scenarios.

Research•Jan 1, 2024•9 citations

Quantum circuit optimization using deep reinforcement learning: Applications across multiple gate sets

Khoa Dang Tao, Sumin Jin, Muhammad Raza +1 more

The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.

quant-pharXiv:2608.19103•Aug 20, 2026•0 citations

Quantum circuits as a game: A reinforcement learning agent for quantum compilation and its application to reconfigurable neutral atom arrays

Kouhei Nakaji, Jonathan Wurtz, Haozhe Huang +4 more

We introduce the "quantum circuit daemon" (QC-Daemon), a reinforcement learning agent for compiling quantum device operations aimed at efficient quantum hardware execution. We apply QC-Daemon to the move synthesis problem called the Atom Game, which involves orchestrating parallel circuits on reconfigurable neutral atom arrays. In our numerical simulation, the QC-Daemon is implemented by two different types of transformers with a physically motivated architecture and trained by a reinforcement learning algorithm. We observe a reduction of the logarithmic infidelity for various benchmark problems up to 100 qubits by intelligently changing the layout of atoms. Additionally, we demonstrate the transferability of our approach: a Transformer-based QC-Daemon trained on a diverse set of circuits successfully generalizes its learned strategy to previously unseen circuits.

arXiv:2506.05536•Jun 5, 2025•0 citations

Design of Quantum Circuits for Random Walk Algorithms

Amlan Chakrabarti, Chia-Chun Lin, Niraj Kumar Jha

A quantum algorithm is defined by a sequence of operations that runs on a realistic model of quantum computation. Since the first quantum algorithm proposed by David Deutsch(1985), a large number of impressive quantum algorithms have been developed. Quantum random walks on a graph, which are analogous to classical stochastic walk, form the basis for some of the recent quantum algorithms that promise to significantly outperform existing classical random walk algorithms. Though research has been done on the application of quantum random walk to important computational problems, very little work has been done on its quantum circuit design. There are two types of quantum random walk algorithms: discrete-time and continuous-time. In this paper, we propose quantum circuit designs for both types of random walk algorithms that operate on various graphs. We consider in detail two important problems to which random walk algorithms are applicable: the triangle finding problem and binary welded tree problem. Though there exist a few research works related to quantum circuit design for random walk on graphs, to the best of our knowledge, the circuit designs we present here are first of their kind. We also provide an estimate of the quantum cost of these circuits for several physical machine descriptions (PMDs)of quantum systems, based on the number of quantum operations and execution cycles.

IEEE Computer Society Annual Symposium on VLSI•Aug 1, 2012•9 citations

Quantum Enhancements for AlphaZero

James Chao, Ramiro Rodriguez, Sean Crowe

Reinforcement learning algorithms including AlphaZero are powerful artificial intelligence (AI) algorithms, but are known to be resource intensive and unable to train within a reasonable budget. Speeding up learning would be valuable to further expand application areas for real world problems. In this paper, we investigate two quantum computing methods to enhance AlphaZero, with the goal of speeding up training. We evaluate the results by playing the board game Othello, an adversarial multi-agent turn based game similar to the game Go. First, parameterized quantum circuits (PQC) have been shown to train hybrid quantum-classical reinforcement learning agents in standard benchmark environments. With this inspiration, we replace the classical neural network with a PQC quantum neural network (QNN) in the AlphaZero architecture. Second, tensor-network quantum circuits have been used to extract important features for convolutional neural networks (CNNs) in image classification tasks. Using this as inspiration, we use a tree tensor network (TTN) to extract features from the Othello game board, generating a new set of feature vectors for a classical neural network to estimate the policy and value. Results show both novel methods converge to master the game and achieve the same level of play compared to the classical AlphaZero agent.

Genetic and Evolutionary Computation Conference Companion (GECCO Companion)•Jul 15, 2023•3 citations

Quantum Reinforcement Learning with Quantum World Model

Peigen Zeng, Ying He, Fei Richard Yu +1 more

Quantum reinforcement learning (QRL) can outperform classical reinforcement learning (RL) by utilizing quantum parallel theory and quantum phenomena such as superposition and entanglement. Although some excellent work has been done on QRL, most existing works either fail to show the exponential advantage of quantum computation over classical computation in terms of performance or are too demanding on quantum devices. In this paper, we provide a novel perspective on combining quantum computing and RL with faster convergence speed and relatively relaxed demands on quantum devices. Specifically, we propose a method to construct a world model with quantum circuit that allows it to interact in a quantum way. In addition, we use Grover's algorithm to efficiently extract high-value information from the quantum world model. Extensive simulation results show that the proposed method can have superior performance compared to classical RL algorithms.

Globecom•Dec 4, 2023•1 citation
Quantum walks on binary trees are used in many quantum algorithms to achieve important speedup over classical algorithms. The formulation of this kind of algorithms as quantum circuit presents the advantage of being easily readable, executable on circuit based quantum computers and simulators and optimal on the usage of resources. We propose a strategy to compose quantum circuit that performs quantum walk on binary trees following universal gate model quantum computation principles. We give a particular attention to NAND formula evaluation algorithm as it could have many applications in game theory and reinforcement learning. We therefore propose an application of this algorithm and show how it can be used to train a quantum reinforcement learning agent in a two player game environment.
Game theory problems are widely applied in many research areas such as computer science and finance, with the key issue being how to quickly make decisions. Here, we present a novel quantum algorithm for game theory problems based on a continuous quantum walk. Our algorithm exhibits quantum advantage compared to classical game algorithms. Furthermore, we exploit the analogy between the wave function of the Schrödinger equation and the voltage in Kirchhoff's law to effectively translate the design of quantum game trees into classical circuit networks. We have theoretically simulated the quantum game trees and experimentally validated the quantum functionality speedup on classical circuit networks. Due to the robust scalability and stability inherent in classical circuit networks, quantum game trees implemented within this framework hold promise for addressing more intricate application scenarios.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
We introduce the "quantum circuit daemon" (QC-Daemon), a reinforcement learning agent for compiling quantum device operations aimed at efficient quantum hardware execution. We apply QC-Daemon to the move synthesis problem called the Atom Game, which involves orchestrating parallel circuits on reconfigurable neutral atom arrays. In our numerical simulation, the QC-Daemon is implemented by two different types of transformers with a physically motivated architecture and trained by a reinforcement learning algorithm. We observe a reduction of the logarithmic infidelity for various benchmark problems up to 100 qubits by intelligently changing the layout of atoms. Additionally, we demonstrate the transferability of our approach: a Transformer-based QC-Daemon trained on a diverse set of circuits successfully generalizes its learned strategy to previously unseen circuits.
A quantum algorithm is defined by a sequence of operations that runs on a realistic model of quantum computation. Since the first quantum algorithm proposed by David Deutsch(1985), a large number of impressive quantum algorithms have been developed. Quantum random walks on a graph, which are analogous to classical stochastic walk, form the basis for some of the recent quantum algorithms that promise to significantly outperform existing classical random walk algorithms. Though research has been done on the application of quantum random walk to important computational problems, very little work has been done on its quantum circuit design. There are two types of quantum random walk algorithms: discrete-time and continuous-time. In this paper, we propose quantum circuit designs for both types of random walk algorithms that operate on various graphs. We consider in detail two important problems to which random walk algorithms are applicable: the triangle finding problem and binary welded tree problem. Though there exist a few research works related to quantum circuit design for random walk on graphs, to the best of our knowledge, the circuit designs we present here are first of their kind. We also provide an estimate of the quantum cost of these circuits for several physical machine descriptions (PMDs)of quantum systems, based on the number of quantum operations and execution cycles.
Reinforcement learning algorithms including AlphaZero are powerful artificial intelligence (AI) algorithms, but are known to be resource intensive and unable to train within a reasonable budget. Speeding up learning would be valuable to further expand application areas for real world problems. In this paper, we investigate two quantum computing methods to enhance AlphaZero, with the goal of speeding up training. We evaluate the results by playing the board game Othello, an adversarial multi-agent turn based game similar to the game Go. First, parameterized quantum circuits (PQC) have been shown to train hybrid quantum-classical reinforcement learning agents in standard benchmark environments. With this inspiration, we replace the classical neural network with a PQC quantum neural network (QNN) in the AlphaZero architecture. Second, tensor-network quantum circuits have been used to extract important features for convolutional neural networks (CNNs) in image classification tasks. Using this as inspiration, we use a tree tensor network (TTN) to extract features from the Othello game board, generating a new set of feature vectors for a classical neural network to estimate the policy and value. Results show both novel methods converge to master the game and achieve the same level of play compared to the classical AlphaZero agent.
Quantum reinforcement learning (QRL) can outperform classical reinforcement learning (RL) by utilizing quantum parallel theory and quantum phenomena such as superposition and entanglement. Although some excellent work has been done on QRL, most existing works either fail to show the exponential advantage of quantum computation over classical computation in terms of performance or are too demanding on quantum devices. In this paper, we provide a novel perspective on combining quantum computing and RL with faster convergence speed and relatively relaxed demands on quantum devices. Specifically, we propose a method to construct a world model with quantum circuit that allows it to interact in a quantum way. In addition, we use Grover's algorithm to efficiently extract high-value information from the quantum world model. Extensive simulation results show that the proposed method can have superior performance compared to classical RL algorithms.