This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.
This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.
Quantum Computing aims to streamline machine learning, making it more effective with fewer trainable parameters. This reduction of parameters can speed up the learning process and reduce the use of computational resources. However, in the current phase of quantum computing development, known as the noisy intermediate-scale quantum era (NISQ), learning is difficult due to a limited number of qubits and widespread quantum noise. To overcome these challenges, researchers are focusing on variational quantum circuits (VQCs). VQCs are hybrid algorithms that merge a quantum circuit, which can be adjusted through parameters, with traditional classical optimization techniques. These circuits require only few qubits for effective learning. Recent studies have presented new ways of applying VQCs to reinforcement learning, showing promising results that warrant further exploration. This study investigates the effects of various techniques — data re-uploading, input scaling, output scaling — and introduces exponential learning rate decay in the quantum proximal policy optimization algorithm’s actor-VQC. We assess these methods in the popular Frozen Lake and Cart Pole environments. Our focus is on their ability to reduce the number of parameters in the VQC without losing effectiveness. Our findings indicate that data re-uploading and an exponential learning rate decay significantly enhance hyperparameter stability and overall performance. While input scaling does not improve parameter efficiency, output scaling effectively manages greediness, leading to increased learning speed and robustness.
Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.
Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.
The comparative evaluation between classical and quantum reinforcement learning (QRL) paradigms was conducted to investigate their convergence behavior, robustness under observational noise, and computational efficiency in a benchmark control environment. The study employed a multilayer perceptron (MLP) agent as a classical baseline and a parameterized variational quantum circuit (VQC) as a quantum counterpart, both trained on the CartPole-v1 environment over 500 episodes. Empirical results demonstrated that the classical MLP achieved near-optimal policy convergence with a mean return of 498.7 +/- 3.2, maintaining stable equilibrium throughout training. In contrast, the VQC exhibited limited learning capability, with an average return of 14.6 +/- 4.8, primarily constrained by circuit depth and qubit connectivity. Noise robustness analysis further revealed that the MLP policy deteriorated gracefully under Gaussian perturbations, while the VQC displayed higher sensitivity at equivalent noise levels. Despite the lower asymptotic performance, the VQC exhibited significantly lower parameter count and marginally increased training time, highlighting its potential scalability for low-resource quantum processors. The results suggest that while classical neural policies remain dominant in current control benchmarks, quantum-enhanced architectures could offer promising efficiency advantages once hardware noise and expressivity limitations are mitigated.
This paper proposes a quantum neural network (QNN)-based reinforcement learning (RL) controller for the real-time control of DC–DC buck converters supplying constant power loads (CPLs). Unlike conventional RL approaches that rely on classical neural networks, the proposed method employs a parameterized quantum circuit to represent the control policy, enabling a compact and expressive function approximator. The QNN is implemented as a classically simulated variational quantum circuit; no physical quantum hardware or quantum computational speedup is claimed. It is trained offline using a deterministic policy gradient framework and subsequently deployed in a real-time control loop under strict sampling constraints. The proposed 36-parameter QNN actor reduces trainable parameters by 85% versus a computation-matched classical RL baseline (245 parameters), while achieving sub-millisecond settling times and near-zero steady-state error across reference tracking, load disturbance, and supply variation scenarios validated on a dSPACE DS1202 hardware platform at 100-kHz switching frequency. Small-signal stability analysis at three tested operating points confirms phase margins exceeding 30° and a gain margin that is theoretically infinite at those points, as the linearized phase does not cross −180°. The results highlight the ability of the QNN to capture complex system dynamics with fewer trainable parameters while maintaining stable closed-loop operation, providing, to the best of the authors’ knowledge, a first step toward integrating quantum machine learning concepts into real-time power electronics control.
Diffusion models typically employ static or heuristic classifier-free guidance (CFG) schedules, which often fail to adapt across timesteps and noise conditions. In this work, we introduce a quantum reinforcement learning (QRL) controller that dynamically adjusts CFG at each denoising step. The controller adopts a hybrid quantum-classical actor-critic architecture: a shallow variational quantum circuit (VQC) with ring entanglement generates policy features, which are mapped by a compact multilayer perceptron (MLP) into Gaussian actions over ΔCFG, while a classical critic estimates value functions. The policy is optimized using Proximal Policy Optimization (PPO) with Generalized Advantage Estimation (GAE), guided by a reward that balances classification confidence, perceptual improvement, and action regularization. Experiments on CIFAR-10 demonstrate that our QRL policy improves perceptual quality (LPIPS, PSNR, SSIM) while reducing parameter count compared to classical RL actors and fixed schedules. Ablation studies on qubit number and circuit depth reveal trade-offs between accuracy and efficiency, and extended evaluations confirm robust generation under long diffusion schedules.