This paper presents an adaptive multi-objective reinforcement learning (MORL) framework integrating classical Deep Q-Networks (DQN) and Variational Quantum Circuit DQN (VQC-DQN).A modified CartPole environment provides two reward signals combined through an adaptive weighting mechanism that updates every 10 episodes and converges in 14 of 15 runs; a corrected replay buffer rescalarizes stored transitions with the current weight instead of a stale one.Across five weight configurations, classical DQN achieves the highest whole-training balance reward, while a parameter-matched Tiny-Classical baseline (37 parameters) attains the best normalized hypervolume (0.412 vs. 0.376 for VQC-DQN and 0.262 for classical); VQC-DQN outperforms Tiny-Classical on raw balance in 3 of 5 weight settings.Retraining VQC-DQN for 200 episodes instead of 150 increases near-convergence balance by 32-163%.At 200 episodes, VQC-DQN has higher point-estimate near-convergence balance than classical DQN across the tested weight settings; however, this comparison is suggestive rather than statistically established because of unequal seed counts, a single 20-episode evaluation window, and unavailable per-seed variance for this metric.A qubit-count sweep under a matched gradient-step budget finds no evidence of barren plateaus within the tested 2-10 qubit range (gradient norm increases, not decreases, with qubit count), while a Deep Sea Treasure benchmark shows classical DQN fails under extreme weights whereas VQC-DQN consistently reaches a treasure but converges to the shallowest policy.These results suggest that classical-versus-quantum comparisons are budget-dependent rather than absolute, pending variance-controlled, multi-seed confirmation.
This paper presents an adaptive multi-objective reinforcement learning (MORL) framework integrating classical Deep Q-Networks (DQN) and Variational Quantum Circuit DQN (VQC-DQN).A modified CartPole environment provides two reward signals combined through an adaptive weighting mechanism that updates every 10 episodes and converges in 14 of 15 runs; a corrected replay buffer rescalarizes stored transitions with the current weight instead of a stale one.Across five weight configurations, classical DQN achieves the highest whole-training balance reward, while a parameter-matched Tiny-Classical baseline (37 parameters) attains the best normalized hypervolume (0.412 vs. 0.376 for VQC-DQN and 0.262 for classical); VQC-DQN outperforms Tiny-Classical on raw balance in 3 of 5 weight settings.Retraining VQC-DQN for 200 episodes instead of 150 increases near-convergence balance by 32-163%.At 200 episodes, VQC-DQN has higher point-estimate near-convergence balance than classical DQN across the tested weight settings; however, this comparison is suggestive rather than statistically established because of unequal seed counts, a single 20-episode evaluation window, and unavailable per-seed variance for this metric.A qubit-count sweep under a matched gradient-step budget finds no evidence of barren plateaus within the tested 2-10 qubit range (gradient norm increases, not decreases, with qubit count), while a Deep Sea Treasure benchmark shows classical DQN fails under extreme weights whereas VQC-DQN consistently reaches a treasure but converges to the shallowest policy.These results suggest that classical-versus-quantum comparisons are budget-dependent rather than absolute, pending variance-controlled, multi-seed confirmation.
Hybrid quantum-classical frameworks leverage quantum computing for machine learning; however, variational quantum circuits (VQCs) are limited by local measurements. We introduce an adaptive non-local observable (ANO) paradigm within VQCs for quantum reinforcement learning (QRL), jointly optimizing circuit parameters and multi-qubit measurements. The ANO-VQC architecture serves as the function approximator in Deep Q-Network (DQN) and Asynchronous Advantage Actor-Critic (A3C) algorithms. On multiple benchmark tasks, ANOVQC agents outperform baseline VQCs. Ablation studies reveal that adaptive measurements enhance the function space without increasing circuit depth. Our results demonstrate that adaptive multi-qubit observables can enable practical quantum advantages in reinforcement learning.
Reinforcement learning (RL) has transformed sequential decisionmaking, yet traditional algorithms like Deep Q-Networks (DQNs) and Proximal Policy Optimization (PPO) often struggle with efficient exploration, stability, and adaptability in dynamic environments. This study presents ARDNS-FN-Quantum (Adaptive Reward-Driven Neural Simulator with Quantum enhancement), a novel framework that integrates a 2-qubit quantum circuit for action selection, a dual-memory system inspired by human cognition, and adaptive exploration strategies modulated by reward variance and curiosity. Evaluated in a 10X10 grid-world over 20,000 episodes, ARDNS-FN-Quantum achieves a 99.5% success rate (versus 81.3% for DQN and 97.0% for PPO), a mean reward of 9.0528 across all episodes (versus 1.2941 for DQN and 7.6196 for PPO), and an average of 46.7 steps to goal (versus 135.9 for DQN and 62.5 for PPO). In the last 100 episodes, it records a mean reward of 9.1652 (versus 7.0916 for DQN and 9.0310 for PPO) and 37.2 steps to goal (versus 52.7 for DQN and 53.4 for PPO). Graphical analyses, including learning curves, steps-to-goal trends, reward variance, and reward distributions, demonstrate ARDNS-FN-Quantum's superior stability (reward variance 5.424 across all episodes versus 252.262 for DQN and 76.583 for PPO) and efficiency. By bridging quantum computing, cognitive science, and RL, ARDNS-FN-Quantum offers a scalable, human-like approach to adaptive learning in uncertain environments, with potential applications in robotics, autonomous systems, and decision-making under uncertainty.
The comparative evaluation between classical and quantum reinforcement learning (QRL) paradigms was conducted to investigate their convergence behavior, robustness under observational noise, and computational efficiency in a benchmark control environment. The study employed a multilayer perceptron (MLP) agent as a classical baseline and a parameterized variational quantum circuit (VQC) as a quantum counterpart, both trained on the CartPole-v1 environment over 500 episodes. Empirical results demonstrated that the classical MLP achieved near-optimal policy convergence with a mean return of 498.7 +/- 3.2, maintaining stable equilibrium throughout training. In contrast, the VQC exhibited limited learning capability, with an average return of 14.6 +/- 4.8, primarily constrained by circuit depth and qubit connectivity. Noise robustness analysis further revealed that the MLP policy deteriorated gracefully under Gaussian perturbations, while the VQC displayed higher sensitivity at equivalent noise levels. Despite the lower asymptotic performance, the VQC exhibited significantly lower parameter count and marginally increased training time, highlighting its potential scalability for low-resource quantum processors. The results suggest that while classical neural policies remain dominant in current control benchmarks, quantum-enhanced architectures could offer promising efficiency advantages once hardware noise and expressivity limitations are mitigated.
The purpose of this research is to show the capabilities that quantum computing has to offer for AI. Therefore, the scheduling problem was chosen in order to demonstrate and compare the classical reinforcement learning (CRL) agent with the quantum computing reinforcement learning (QC-RL) agent. This constrained combinatorial optimization problem involves scheduling an oil field with several drilling rigs which are limited in capacity, area limitations, mandatory maintenance periods, weather restrictions and a mobilization costs which is considered along with well priorities and deadlines. Deep reinforcement learning (DRL) can learn adaptive scheduling policies, but value-based agents such as Deep Q-Networks (DQN) are known to exhibit high variance across random initializations. This paper investigates whether a hybrid quantum-classical value function can improve this reliability. We formulate realistic multi-rig drilling scheduling as a Markov Decision Process with action masking over an eight-constraint simulator (24 wells, 6 rigs, 3 areas, 80-day horizon) and compare a classical DQN against a hybrid Quantum DQN in which the value network’s core is a variational quantum circuit via a 4-qubit ZFeatureMap encoding followed by a RealAmplitudes ansatz with 12 trainable quantum parameters, executed using a PyTorch-Qiskit interface. Both agents share an identical training and evaluation protocol (3 seeds × 30 episodes, greedy evaluation) to ensure a controlled comparison. Under greedy evaluation, the hybrid Quantum DQN completes 22.3/24 wells on average versus 17/24 for the classical DQN, achieves an 12.87% higher mean reward (9.12M vs 8.08M), and most significantly reduces the across-seed reward standard deviation by roughly 6.8× (0.202M vs 1.38M). These results indicate that the structurally constrained quantum value function acts as an implicit regularizer, yielding scheduling policies that are markedly more consistent across initializations than a substantially larger classical network. This simulated quantum computing experiment demonstrates the potential for quantum computing in the field of AI.
There is a lack of adequate studies on dynamic environments control for Quantum Reinforcement Learning (QRL) algorithms, representing a significant gap in this field. This study contributes to bridging this gap by demonstrating the potential of quantum RL algorithms to effectively handle dynamic environments. In this research, the performance and robustness of Quantum Deep Q-learning Networks (DQN) were examined in two dynamic environments, Cart Pole and Lunar Lander, by using three distinct quantum Ansatz layers: RealAmplitudes, EfficientSU2, and TwoLocal. The quantum DQNs were compared with classical DQN algorithms in terms of convergence speed, loss minimization, and Q-value behavior. It was observed that the RealAmplitudes Ansatz outperformed the other quantum circuits, demonstrating faster convergence and superior performance in minimizing the loss function. To assess robustness, the pole length was increased in the Cart Pole environment, and a wind function was added to the Lunar Lander environment after the 50th episode. All three quantum Ansatz layers were found to maintain robust performance under disturbed conditions, with consistent reward values, loss minimization, and stable Q-value distributions. Although the proposed QRL demonstrates competitive results overall, classical RL can surpass them in convergence speed under specific conditions.
Deep Reinforcement Learning (RL) has considerably advanced over the past decade. At the same time, state-of-the-art RL algorithms require a large computational budget in terms of training time to converge. Recent work has started to approach this problem through the lens of quantum computing, which promises theoretical speed-ups for several traditionally hard tasks. In this work, we examine a class of hybrid quantum-classical RL algorithms that we collectively refer to as variational quantum deep Q-networks (VQ-DQN). We show that VQ-DQN approaches are subject to instabilities that cause the learned policy to diverge, study the extent to which this afflicts reproducibility of established results based on classical simulation, and perform systematic experiments to identify potential explanations for the observed instabilities. Additionally, and in contrast to most existing work on quantum reinforcement learning, we execute RL algorithms on an actual quantum processing unit (an IBM Quantum Device) and investigate differences in behaviour between simulated and physical quantum systems that suffer from implementation deficiencies. Our experiments show that, contrary to opposite claims in the literature, it cannot be conclusively decided if known quantum approaches, even if simulated without physical imperfections, can provide an advantage as compared to classical approaches. Finally, we provide a robust, universal and well-tested implementation of VQ-DQN as a reproducible testbed for future experiments.