Reusable rockets have emerged as a pivotal technology for cost reduction and efficiency improvement in the space industry. Joint control, communication, and computing technologies must be considered for the mission performance of these reusable rockets. This paper introduces a quantum reinforcement learning for the joint control, communication, and computing (QRL-C3) algorithm designed for quantum reusable rockets. The QRL-C3 aims to solve the issue of diminished training performance in high-dimensional states and action spaces prevalent in conventional reinforcement learning (RL). Leveraging quantum neural networks (QNN) with superposition and entanglement, QRL-C3 enhances information representation and processing. The framework integrates a (i) robust quantum control system for precise re-entry and landing, a (ii) quantum multi-agent reinforcement learning (QMARL)-based scheduler for seamless communication with multiple ground stations (GSs) for enabling Internet-of-Things (IoT) services, and (iii) energy-efficient computing algorithms considering queue backlog for optimal image processing and transmission power management. These components synergistically improve mission performance and rocket re-usability. Experimental validation using SpaceX's Falcon 9 model demonstrates that QRL-C3 outperforms conventional RL approaches, achieving higher training efficiency. This study is the first to integrate control, communication, and computing systems in reusable rockets through quantum artificial intelligence (QAI), paving the way for future advancements in QAI-based space technologies.
Reusable rockets have emerged as a pivotal technology for cost reduction and efficiency improvement in the space industry. Joint control, communication, and computing technologies must be considered for the mission performance of these reusable rockets. This paper introduces a quantum reinforcement learning for the joint control, communication, and computing (QRL-C3) algorithm designed for quantum reusable rockets. The QRL-C3 aims to solve the issue of diminished training performance in high-dimensional states and action spaces prevalent in conventional reinforcement learning (RL). Leveraging quantum neural networks (QNN) with superposition and entanglement, QRL-C3 enhances information representation and processing. The framework integrates a (i) robust quantum control system for precise re-entry and landing, a (ii) quantum multi-agent reinforcement learning (QMARL)-based scheduler for seamless communication with multiple ground stations (GSs) for enabling Internet-of-Things (IoT) services, and (iii) energy-efficient computing algorithms considering queue backlog for optimal image processing and transmission power management. These components synergistically improve mission performance and rocket re-usability. Experimental validation using SpaceX's Falcon 9 model demonstrates that QRL-C3 outperforms conventional RL approaches, achieving higher training efficiency. This study is the first to integrate control, communication, and computing systems in reusable rockets through quantum artificial intelligence (QAI), paving the way for future advancements in QAI-based space technologies.
The increasing demands for ultra-reliable low-latency communication (URLLC), high-capacity, connectivity, and mobility in next-generation wireless networks have driven the integration of unmanned aerial vehicles (UAVs) as flexible and adaptive communication nodes. UAV-assisted networks offer enhanced coverage, dynamic deployment, and cost-effective infrastructure for the sixth generation (6G) and beyond. However, optimizing UAV trajectory planning, resource allocation, and interference management is particularly challenging, especially in dynamic and large-scale environments. Classical reinforcement learning (RL) techniques, while effective in adaptive optimization, suffer from computational inefficiencies and slow convergence in high-dimensional state-action spaces. To address these constraints, quantum reinforcement learning (QRL) emerges as a novel paradigm that leverages quantum computing principles, such as superposition and entanglement, to enhance decision-making efficiency. This study provides a comprehensive overview of QRL algorithms and investigates their transformative potential in UAV-assisted wireless communication networks. We analyze UAV deployment strategies, network architectures, and topologies, highlighting their advantages as well as inherent challenges. Furthermore, we examine the limitations of classical RL algorithms and assess how QRL overcomes them while improving computational efficiency and exploration capabilities. In addition, we present a detailed review of existing QRL algorithms applicable to next-generation UAV-enhanced wireless networks, identifying key challenges that must be addressed for practical deployment. Finally, we identify open research challenges and offer future directions for integrating QRL into UAV-assisted communication, paving the way for scalable, intelligent, and high-performance wireless systems.
Recent research in quantum computing has gained significant momentum, promising transformative advances across various fields, including artificial intelligence (AI). This paper focuses on quantum reinforcement learning (QRL), which utilizes quantum neural network (QNN) to address the inherent chal-lenges in conventional reinforcement learning (RL) frameworks. For a comprehensive examination of QRL, in this paper, the structure of QNN is explored in-depth, including the functional roles of key quantum circuit gates such as the unitary, rotation, controlled-X, and Hadamard gates. This paper also discusses the primary components of QNN, namely state encoding, parameterized quantum circuits (PQC), and measurement processes. Furthermore, the application of IBM qiskit for the visualization of quantum states and circuits is highlighted, providing practical insights into the deployment of quantum principles in RL. Consequently, this paper encapsulates the potential of QRL to revolutionize AI, positioning it as a pivotal area of future research and development in quantum computing.
Quantum Reinforcement Learning (QRL) represents a groundbreaking approach at the intersection of quantum computing and artificial intelligence, promising to revolutionize decision-making processes across various domains. This chapter explores the fundamental principles of QRL and its diverse applications in critical sectors, including transportation networks, healthcare, financial decision-making, and smart energy systems. By leveraging the computational advantages of quantum algorithms, QRL enhances traditional reinforcement learning techniques, enabling more efficient data processing and improved strategic decision-making capabilities. Case studies within this chapter illustrate the transformative potential of QRL in optimizing logistics, improving medical imaging, and enhancing energy management strategies. The findings underscore the need for further exploration of quantum applications to fully harness their capabilities and address contemporary challenges. This comprehensive examination provides a solid foundation for researchers and practitioners aiming to integrate quantum technologies into real-world applications, paving the way for innovative solutions and sustainable advancements.
Reinforcement learning (RL) enables agents to learn optimal policies through environmental interaction. However, RL suffers from reduced learning efficiency due to the curse of dimensionality in high-dimensional spaces. Quantum reinforcement learning (QRL) addresses this issue by leveraging superposition and entanglement in quantum computing, allowing efficient handling of high-dimensional problems with fewer resources. QRL combines quantum neural networks (QNNs) with RL, where the parameterized quantum circuit (PQC) acts as the core computational module. The PQC performs linear and nonlinear transformations through gate operations, similar to hidden layers in classical neural networks. Previous QRL studies, however, have used fixed PQC structures based on empirical intuition without verifying their optimality. This paper proposes a QRL-NAS algorithm that integrates quantum neural architecture search (QNAS) to optimize PQC structures within QRL. Experiments demonstrate that QRL-NAS achieves higher rewards than QRL with fixed circuits, validating its effectiveness and practical utility.
Reinforcement learning (RL) enables agents to learn optimal policies through environmental interaction. However, RL suffers from reduced learning efficiency due to the curse of dimensionality in high-dimensional spaces. Quantum reinforcement learning (QRL) addresses this issue by leveraging superposition and entanglement in quantum computing, allowing efficient handling of high-dimensional problems with fewer resources. QRL combines quantum neural networks (QNNs) with RL, where the parameterized quantum circuit (PQC) acts as the core computational module. The PQC performs linear and nonlinear transformations through gate operations, similar to hidden layers in classical neural networks. Previous QRL studies, however, have used fixed PQC structures based on empirical intuition without verifying their optimality. This paper proposes a QRLNAS algorithm that integrates quantum neural architecture search (QNAS) to optimize PQC structures within QRL. Experiments demonstrate that QRL-NAS achieves higher rewards than QRL with fixed circuits, validating its effectiveness and practical utility.
Attitude control is vital for ensuring flight stability in aircraft operating under dynamic disturbances such as turbulence and vortices. While reinforcement learning (RL)-based controllers offer adaptivity to uncertain dynamics, their deployment on airborne platforms is often constrained by the large number of training parameters and associated computational overhead. To address this limitation, this article implements a quantum reinforcement learning (QRL)-based attitude controller in which a quantum neural network (QNN) replaces the classical neural network (NN) within an actor-critic framework. Exploiting quantum principles such as superposition and entanglement, the QRL controller reduces training parameters, thereby enhancing computational efficiency and enabling resilient, lightweight flight control for next-generation aircraft systems. The proposed controller is evaluated in a simulated flight-control environment that reflects the aerodynamic specifications of a Boeing B777-300X airframe and is subjected to stochastic vortical disturbances. The simulation results demonstrate that the QNN-based controller can reduce the number of training parameters by approximately 760x compared with a controller based on the classical NN. These results indicate that, within the realistic experimental setting, QRL can provide a parameter-efficient and computationally lightweight alternative for aircraft attitude control under uncertain aerodynamic conditions.