Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2408.01187 · IEEE International Conference on Quantum Computing and Engineering (QCE)

Optimizing Variational Quantum Circuits Using Metaheuristic Strategies in Reinforcement Learning

Michael Kölle, Daniel Seidl, Maximilian Zorn, Philipp J. Altmann, Jonas Stein, Thomas Gabor

Published Sep 15, 2024• 3 citations
arXivHTMLPDF

Abstract

Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms — Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search — into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in$5\times 5$MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.

DOI
10.1109/qce60285.2024.10300

Related papers

Optimization of Reinforcement Learning Using Quantum Computation

Roopa Ravish, Nischal R. Bhat, N. Nandakumar +3 more

Exploring the convergence of quantum computing and machine learning, this paper delves into Quantum Reinforcement Learning (QRL) with a specific focus on Variational Quantum Circuits (VQC). Alongside a comprehensive examination of the field, the study presents a concrete implementation of QRL tailored to foundational gym environments. Leveraging principles from Quantum Computing, experiments reveal QRL’s potential in enhancing reinforcement learning tasks within these environments, showcasing notable space optimization for RL models. Quantitative results indicate a reduction in the number of trainable parameters by up to 90% when compared to classical approaches, significantly improving efficiency, mainly due to quantum principles of superposition and entanglement. This highlights the promising implications of integrating quantum computing techniques to address challenges and advance capabilities in fundamental reinforcement learning scenarios. Furthermore, the study discusses the adaptability of QRL algorithms to diverse problem domains, suggesting their potential for scalability and applicability beyond simple environments. Such versatility underscores the robustness and practical relevance of QRL methodologies in real-world scenarios, positioning them as valuable tools for tackling complex reinforcement learning challenges.

IEEE Access•Jan 1, 2024•10 citations

Quantum-Enhanced Reinforcement Learning (QRL) for Robotics

Jincheng Zhang

This paper explores the application of Quantum-Enhanced Reinforcement Learning (QRL) to robotics, aiming to accelerate learning speeds and enable more intelligent decision-making in complex environments. Traditional reinforcement learning algorithms often struggle with scalability and convergence, particularly when dealing with high-dimensional state spaces and complex environments. Quantum computing offers the potential to overcome these limitations by leveraging quantum mechanics to perform computations that are intractable for classical computers. We propose a novel QRL framework that utilizes quantum-enhanced reinforcement learning to improve exploration, adaptation, and overall performance. The core mechanism involves employing quantum circuits to represent and manipulate the state space, facilitating more efficient learning and potentially uncovering novel strategies. This research investigates the feasibility and potential impact of QRL on robotic control, focusing on scenarios requiring rapid adaptation and robust decision-making in dynamic environments. We present preliminary results demonstrating the efficacy of our approach in a simulated robotic task involving navigation and obstacle avoidance.

•Sep 3, 2026•0 citations

Multi-Agent Quantum Reinforcement Learning Using Evolutionary Optimization

Michael Kölle, Felix Topp, Thomy Phan +3 more

Multi-Agent Reinforcement Learning is becoming increasingly more important in times of autonomous driving and other smart industrial applications. Simultaneously a promising new approach to Reinforcement Learning arises using the inherent properties of quantum mechanics, reducing the trainable parameters of a model significantly. However, gradient-based Multi-Agent Quantum Reinforcement Learning methods often have to struggle with barren plateaus, holding them back from matching the performance of classical approaches. While gradient free Quantum Reinforcement Learning methods may alleviate some of these challenges, they too are not immune to the difficulties posed by barren plateaus. We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning using evolutionary optimization. We evaluate our genetic variations in the Coin Game environment and also compare them to classical approaches. We showed that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters. Compared to the larger neural network, our approaches archive similar results using $97.88\%$ less parameters.

arXiv:2311.05546•International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2024•6 citations

Variational quantum reinforcement learning via evolutionary optimization

Samuel Yen-Chi Chen, Chih-Min Huang, Chia-Wei Hsing +2 more

Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

arXiv:2109.00540•Machine Learning Science and Technology•Dec 21, 2021•108 citations

Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization

Vincent Gurgul, Ying Chen, Stefan Lessmann

This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public

cs.LGCross-listedarXiv:2601.18811•Jan 21, 2026•0 citations

Quantum Channel Optimization: Integrating Quantum-Inspired Machine Learning With Genetic Adaptive Strategies

R. Vijay Anand, G. Magesh, I. Alagiri +3 more

There is an ever-growing need for optimizing efficient nano and quantum communication systems and this demand calls for more advanced optimization techniques than those that have been around for quite some time. The conventional genetic algorithms (GAs) proved to be quite helpful in general but proved to be poor when it comes to fast convergence and adaptability to the changing communication environment. Moreover, fault tolerance and resource efficiency in these communication channels are poorly addressed, giving rise to a huge roadblock to practical implementations in a number of different use cases. This research presents a set of novel hybrid optimization methods aimed at addressing the issues previously mentioned. In the first place, a Quantum-inspired Genetic Algorithm (QIGA) with Simulated Annealing (SA) Initialization is proposed through augmenting traditional genetic operations with the quantum computing paradigm. In the first instance, the hybrid approach works by utilizing the parameters derived from SA while initializing the GA population, hence effective exploration of solution space. Additionally, the Adaptive Genetic Algorithm with Reinforcement Learning (AGA-RL) is presented, which uses real-time feedback from the current environment to modify the GA parameters. This way, the system will be much more adaptive and hence be able to adjust the GA so that it can match the rapidly changing channel conditions. Simultaneously, the Fault-Tolerant Genetic Algorithm with Error-Correcting Codes (FTGA-ECC) is developed to ensure reliable data transmission, especially in an error-prone environment. To make up for resource constraints, the Energy-Aware Genetic Algorithm with Dynamic Resource Allocation (EAGA-DRA) strategy ensures that resources are well-managed by dynamically changing allocation based on the optimization process and the particular needs of the channel, hence reducing energy overhead and spatial overhead without compromising channel quality. A great leap is in the integration of Machine Learning through the Deep Reinforcement Learning-guided Genetic Algorithm (DRL-GA). The above approach exploits deep reinforcement learning to fine-tune the exploration and exploitation strategies of the GA, hence providing faster convergence and improved solution quality. All of these methods are validated by the Quantum-inspired Benchmarking Framework (QIBF), which forms a systematized comparative platform to evaluate different optimization techniques from the perspective of QCA-based communication systems. The above approaches not only provide the improvement in efficiency and reliability but also contribute to enhancing the adaptability of communication systems and provide a very good base for future research and development in the field of quantum communications and beyond, with potential applications extending to secure communication, quantum computing, and nanotechnology-based devices & scenarios. This work is multifold, promising significant advancements in the field of quantum communication and beyond. The empirical findings showed that the suggested algorithms perform significantly better than traditional approaches in several important performance metrics, such as robustness to channel errors, adaptability to changing signal-to-noise ratios, convergence speed under dynamic conditions, and solution quality in highly variable environments.INDEX TERMS Quantum-inspired genetic algorithm, adaptive genetic algorithm, fault-tolerance, energy-efficient optimization, deep reinforcement learning.

IEEE Access•Jan 1, 2024•21 citations
Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms — Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search — into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in5×55\times 55×5MiniGrid Reinforcement Learning environments show that, all algorithms yield nearoptimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation.
Exploring the convergence of quantum computing and machine learning, this paper delves into Quantum Reinforcement Learning (QRL) with a specific focus on Variational Quantum Circuits (VQC). Alongside a comprehensive examination of the field, the study presents a concrete implementation of QRL tailored to foundational gym environments. Leveraging principles from Quantum Computing, experiments reveal QRL’s potential in enhancing reinforcement learning tasks within these environments, showcasing notable space optimization for RL models. Quantitative results indicate a reduction in the number of trainable parameters by up to 90% when compared to classical approaches, significantly improving efficiency, mainly due to quantum principles of superposition and entanglement. This highlights the promising implications of integrating quantum computing techniques to address challenges and advance capabilities in fundamental reinforcement learning scenarios. Furthermore, the study discusses the adaptability of QRL algorithms to diverse problem domains, suggesting their potential for scalability and applicability beyond simple environments. Such versatility underscores the robustness and practical relevance of QRL methodologies in real-world scenarios, positioning them as valuable tools for tackling complex reinforcement learning challenges.
This paper explores the application of Quantum-Enhanced Reinforcement Learning (QRL) to robotics, aiming to accelerate learning speeds and enable more intelligent decision-making in complex environments. Traditional reinforcement learning algorithms often struggle with scalability and convergence, particularly when dealing with high-dimensional state spaces and complex environments. Quantum computing offers the potential to overcome these limitations by leveraging quantum mechanics to perform computations that are intractable for classical computers. We propose a novel QRL framework that utilizes quantum-enhanced reinforcement learning to improve exploration, adaptation, and overall performance. The core mechanism involves employing quantum circuits to represent and manipulate the state space, facilitating more efficient learning and potentially uncovering novel strategies. This research investigates the feasibility and potential impact of QRL on robotic control, focusing on scenarios requiring rapid adaptation and robust decision-making in dynamic environments. We present preliminary results demonstrating the efficacy of our approach in a simulated robotic task involving navigation and obstacle avoidance.
Multi-Agent Reinforcement Learning is becoming increasingly more important in times of autonomous driving and other smart industrial applications. Simultaneously a promising new approach to Reinforcement Learning arises using the inherent properties of quantum mechanics, reducing the trainable parameters of a model significantly. However, gradient-based Multi-Agent Quantum Reinforcement Learning methods often have to struggle with barren plateaus, holding them back from matching the performance of classical approaches. While gradient free Quantum Reinforcement Learning methods may alleviate some of these challenges, they too are not immune to the difficulties posed by barren plateaus. We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning using evolutionary optimization. We evaluate our genetic variations in the Coin Game environment and also compare them to classical approaches. We showed that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters. Compared to the larger neural network, our approaches archive similar results using 97.88%97.88\%97.88% less parameters.
Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.
This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public
There is an ever-growing need for optimizing efficient nano and quantum communication systems and this demand calls for more advanced optimization techniques than those that have been around for quite some time. The conventional genetic algorithms (GAs) proved to be quite helpful in general but proved to be poor when it comes to fast convergence and adaptability to the changing communication environment. Moreover, fault tolerance and resource efficiency in these communication channels are poorly addressed, giving rise to a huge roadblock to practical implementations in a number of different use cases. This research presents a set of novel hybrid optimization methods aimed at addressing the issues previously mentioned. In the first place, a Quantum-inspired Genetic Algorithm (QIGA) with Simulated Annealing (SA) Initialization is proposed through augmenting traditional genetic operations with the quantum computing paradigm. In the first instance, the hybrid approach works by utilizing the parameters derived from SA while initializing the GA population, hence effective exploration of solution space. Additionally, the Adaptive Genetic Algorithm with Reinforcement Learning (AGA-RL) is presented, which uses real-time feedback from the current environment to modify the GA parameters. This way, the system will be much more adaptive and hence be able to adjust the GA so that it can match the rapidly changing channel conditions. Simultaneously, the Fault-Tolerant Genetic Algorithm with Error-Correcting Codes (FTGA-ECC) is developed to ensure reliable data transmission, especially in an error-prone environment. To make up for resource constraints, the Energy-Aware Genetic Algorithm with Dynamic Resource Allocation (EAGA-DRA) strategy ensures that resources are well-managed by dynamically changing allocation based on the optimization process and the particular needs of the channel, hence reducing energy overhead and spatial overhead without compromising channel quality. A great leap is in the integration of Machine Learning through the Deep Reinforcement Learning-guided Genetic Algorithm (DRL-GA). The above approach exploits deep reinforcement learning to fine-tune the exploration and exploitation strategies of the GA, hence providing faster convergence and improved solution quality. All of these methods are validated by the Quantum-inspired Benchmarking Framework (QIBF), which forms a systematized comparative platform to evaluate different optimization techniques from the perspective of QCA-based communication systems. The above approaches not only provide the improvement in efficiency and reliability but also contribute to enhancing the adaptability of communication systems and provide a very good base for future research and development in the field of quantum communications and beyond, with potential applications extending to secure communication, quantum computing, and nanotechnology-based devices & scenarios. This work is multifold, promising significant advancements in the field of quantum communication and beyond. The empirical findings showed that the suggested algorithms perform significantly better than traditional approaches in several important performance metrics, such as robustness to channel errors, adaptability to changing signal-to-noise ratios, convergence speed under dynamic conditions, and solution quality in highly variable environments.INDEX TERMS Quantum-inspired genetic algorithm, adaptive genetic algorithm, fault-tolerance, energy-efficient optimization, deep reinforcement learning.