Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms — Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search — into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in $5\times 5$ MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms — Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search — into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in$5\times 5$MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.

arXiv:2408.01187IEEE International Conference on Quantum Computing and Engineering (QCE)Sep 15, 20243 citations

Reinforcement learning for quantum circuit optimization

Thomas Fösel, Murphy Yuezhen Niu, Florian Marquardt +1 more

Bulletin of the American Physical SocietyMar 16, 20210 citations

Optimizing Initial Qubit Mappings With Variable Connection Fidelity Using Deep Reinforcement Learning

Rares Adrian Oancea, Stan van der Linde, Willem de Kok +2 more

Quantum compilation requires solving the initial mapping problem, which is crucial for optimizing qubit assignment and minimizing fidelity in compiled quantum circuits. This study investigates the application of Deep Reinforcement Learning (DRL) to address initial mapping across various qubit topologies, while considering varying qubit connection fidelities. By leveraging policy gradient algorithms-Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO), and PPO with action masking-DRL agents effectively compute high-quality mappings for both small-and medium-scale quantum architectures, but their efficiency decreases as the system size grows, highlighting the need for further optimization strategies. Fine-tuning hyperparameters and incorporating action masking are shown to be essential for preventing illegal actions and enhancing mapping accuracy, particularly in larger systems. This research also aids the continued development of initial mapping techniques by introducing a comprehensive qubit connectivity database for systematic evaluation of DRL methods across diverse architectures.Quantum compilation requires solving the initial mapping problem, which is crucial for optimizing qubit assignment and minimizing fidelity in compiled quantum circuits. This study investigates the application of Deep Reinforcement Learning (DRL) to address initial mapping across various qubit topologies, while considering varying qubit connection fidelities. By leveraging policy gradient algorithms-Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO), and PPO with action masking-DRL agents effectively compute high-quality mappings for both small-and medium-scale quantum architectures, but their efficiency decreases as the system size grows, highlighting the need for further optimization strategies. Fine-tuning hyperparameters and incorporating action masking are shown to be essential for preventing illegal actions and enhancing mapping accuracy, particularly in larger systems. This research also aids the continued development of initial mapping techniques by introducing a comprehensive qubit connectivity database for systematic evaluation of DRL methods across diverse architectures.

Nov 20, 20240 citations

Reinforcement Learning for Quantum Circuit Optimization: A Review

Akash Kundu, Aritra Sarkar, Prayag Tiwari +1 more

Jan 1, 20260 citations

5\times 5

MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.