A novel quantum-inspired reinforcement learning (QiRL) algorithm is proposed for decision-making of Markovian state transition. The QiRL algorithm adopts a probabilistic action selection policy to better balance the tradeoff between exploration and exploitation, which is inspired by the collapse phenomenon in quantum measurement. Several simulated experiments of Markovian state transition demonstrate that QiRL is more robust to learning rates and initial states than traditional reinforcement learning. The QiRL approach provides an effective method for complex decision-making problems.
A novel quantum-inspired reinforcement learning (QiRL) algorithm is proposed for decision-making of Markovian state transition. The QiRL algorithm adopts a probabilistic action selection policy to better balance the tradeoff between exploration and exploitation, which is inspired by the collapse phenomenon in quantum measurement. Several simulated experiments of Markovian state transition demonstrate that QiRL is more robust to learning rates and initial states than traditional reinforcement learning. The QiRL approach provides an effective method for complex decision-making problems.
A novel quantum-inspired reinforcement learning (QiRL) algorithm is proposed for navigation control of autonomous mobile robots. The QiRL algorithm adopts a probabilistic action selection policy and a new reinforcement strategy, which are inspired, respectively, by the collapse phenomenon in quantum measurement and amplitude amplification in quantum computation. Several simulated experiments of Markovian state transition demonstrate that QiRL is more robust to learning rates and initial states than traditional reinforcement learning. The QiRL approach is then applied to navigation control of a real mobile robot, and the simulated and experimental results show the effectiveness of the proposed approach.
The key approaches for machine learning, particularly learning in unknown probabilistic environments, are new representations and computation mechanisms. In this paper, a novel quantum reinforcement learning (QRL) method is proposed by combining quantum theory and reinforcement learning (RL). Inspired by the state superposition principle and quantum parallelism, a framework of a value-updating algorithm is introduced. The state (action) in traditional RL is identified as the eigen state (eigen action) in QRL. The state (action) set can be represented with a quantum superposition state, and the eigen state (eigen action) can be obtained by randomly observing the simulated quantum state according to the collapse postulate of quantum measurement. The probability of the eigen action is determined by the probability amplitude, which is updated in parallel according to rewards. Some related characteristics of QRL such as convergence, optimality, and balancing between exploration and exploitation are also analyzed, which shows that this approach makes a good tradeoff between exploration and exploitation using the probability amplitude and can speedup learning through the quantum parallelism. To evaluate the performance and practicability of QRL, several simulated experiments are given, and the results demonstrate the effectiveness and superiority of the QRL algorithm for some complex problems. This paper is also an effective exploration on the application of quantum computation to artificial intelligence.
This paper proposes a novel action selection method based on quantum computation and reinforcement learning (RL). Inspired by the advantages of quantum computation, the state/action in a RL system is represented with quantum superposition state. The probability of action eigenvalue is denoted by probability amplitude, which is updated according to rewards. And the action selection is carried out by observing quantum state according to collapse postulate of quantum measurement. The results of simulated experiments show that quantum computation can be effectively used to action selection and decision making through speeding up learning. This method also makes a good tradeoff between exploration and exploitation for RL using probability characteristics of quantum theory.
Recent progress in quantum-inspired computing and neuromorphic machine has rekindled the quest to develop learning systems capable of probabilistic thinking, and which can be run with the energy efficiency of the brain. However, the vast majority of reinforcement learning (RL) models can be described as being constrained by strict update policies and high power usage when implemented on a spiking architecture or an edge-based architecture. To fill this gap, it is proposed in this study that a Quantum-Inspired Reinforcement Learning (QiRL) framework can combine the policy encoding based on amplitude, and the neuromorphic adaptation to reach real-time and low-energy decision-making. In contrast to the traditional RL, QiRL incorporates phase-modulated dynamics of the interference to enhance positive learning and discourage reward oscillation.
According to the existing problems in RL area, such as low learning speed and tradeoff between exploration and exploitation, SIRL and QRL methods are introduced based on the theory of RL and quantum computation in this chapter, which follows the developing roadmap from the superposition-inspired methods to the RL methods in quantum systems. Just as simulated annealing algorithm comes from mimicking the physical annealing process, quantum characteristics also broaden our mind and provide alternative approaches to novel RL methods. In this chapter, SIRL method emphasizes the exploration policy and uses a probabilistic action selection method that is inspired by the state superposition principle and collapse postulate. The experiments, which include a puzzle problem and a mobile robot navigation problem, demanstrate the effectiveness of SIRL algorithm and show that it is superior to basic TD algorithm with -greedy policy. As for QRL, the state/action value is represented with quantum superposition state and the action selection is carried out by observing quantum state according to quantum collapse postulate, which means a QRL system is designed for the real quantum system although it can also be simulated on a traditional computer. The results of simulated experiments verified its feasibility and effectiveness with two examples: Prisoner's Dilemma and the control of a five-qubit system. The contents presented in this chapter are mainly the basic ideas and methods related to the combination of RL theory and quantum computation. More theoretic research and applictions are to be investigated in the future.
This paper investigates the application of quantum-inspired algorithms to the field of reinforcement learning (RL). Traditional RL methods often struggle with exploration challenges and can be computationally expensive, particularly in complex environments. We propose a novel RL algorithm that leverages concepts from quantum mechanics, specifically superposition and entanglement, to address these limitations. The core idea is to represent states, actions, and rewards using quantum-inspired representations, allowing the agent to explore the environment more efficiently and potentially converge to optimal policies faster. This work presents a detailed formulation of the algorithm, including the mathematical framework and key considerations for implementation. The potential benefits of this approach, including improved exploration and reduced computational complexity, are discussed. The algorithm is designed to iteratively refine its policy based on simulated quantum measurements, aiming to minimize the expected return. The exploration strategy is guided by the inherent probabilistic nature of quantum states, leading to a more diverse and potentially faster learning process. Further research is warranted to fully explore the capabilities and limitations of this approach across various RL benchmark problems.