Quantum Reinforcement Learning, QLearning Project

Sabine Wölk

Leonhard Kunczik

Joongheon Kim

Annette Zapf, Sabine Wölk

Pairwise sequence alignment is central in bioinformatics for analyzing DNA, RNA, and proteins, revealing similarities that inform evolutionary relationships, gene or protein functions, and disease mechanisms. Classical dynamic programming guarantees optimal alignments but scales poorly, while heuristics reduce runtime at the cost of optimality and still require high resources. Quantum algorithms exploit superposition and entanglement to accelerate optimization or find exact solutions faster. We present a hybrid quantum reinforcement learning (RL) agent based on (Hamann & Wölk, 2022), optimized for NISQ hardware. The agent learns a policy mapping states to action sequences from feedback of an environment. Using a Grover-like subroutine, it evaluates all sequences in parallel, reinforcing those leading to a solution, which reduces environment interactions and can achieve a quadratic speedup over classical RL. To enable execution on current hardware, we introduce a novel Parameterized Quantum Policy embedding entire action sequences, combined with a simplified Grover routine using a black-box oracle. This reduces resource demands but requires prior knowledge and retraining per alignment, limiting generalization. We also outline design ideas for a more general Grover oracle and circuit with broader generalization, which, while not NISQ-compatible, suggest a promising direction for future work.

Daoyi Dong, Chunlin Chen, Zonghai Chen

Joaquim M. Gaspar, Alexandre Bergerault, Vassilis Apostolou +1 more

Quantum Reinforcement Learning (QRL) is an emerging field with many novel approaches suitable for noisy intermediate-scale quantum (NISQ) devices being proposed recently. Projective Simulation (PS) is a classical machine learning method based on random walks in a Directed Acyclic Graph (DAG), and is mainly used to train agents in a Reinforcement Learning setup. This graph corresponds to the Episodic Compositional Memory (ECM) of the agent. For the quantum version of PS we use quantum walks of single photons in quantum optical circuits, which go through tunable components (beamsplitters and phase shifters). We call this specific model Quantum Optical Projective Simulation (QOPS). In this work, we propose a general framework to translate any 2-layer ECM to a quantum optical circuit and an algorithm for solving quantum strategic games using QOPS. Moreover, we introduce QOPS with an entangled state as input. The application that we are using to evaluate the performance of the framework is the Prisoner's Dilemma. We executed the implementation using Altair's noisy simulator, designed to emulate a single photon-based quantum processor by Quandela. The algorithm provided promising results even in the presence of noise, which makes it a strong candidate for runs on single-photon-based quantum processors.