Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1806.03145 · IEEE Transactions on Neural Networks and Learning Systems

Fidelity-Based Probabilistic Q-Learning for Control of Quantum Systems

Chunlin Chen, Daoyi Dong, Han‐Xiong Li, Jian Chu, Tzyh‐Jong Tarn

Published Oct 11, 2013• 151 citations
arXivHTMLPDF

Abstract

The balance between exploration and exploitation is a key problem for reinforcement learning methods, especially for Q-learning. In this paper, a fidelity-based probabilistic Q-learning (FPQL) approach is presented to naturally solve this problem and applied for learning control of quantum systems. In this approach, fidelity is adopted to help direct the learning process and the probability of each action to be selected at a certain state is updated iteratively along with the learning process, which leads to a natural exploration strategy instead of a pointed one with configured parameters. A probabilistic Q-learning (PQL) algorithm is first presented to demonstrate the basic idea of probabilistic action selection. Then the FPQL algorithm is presented for learning control of quantum systems. Two examples (a spin-1/2 system and a Λ-type atomic system) are demonstrated to test the performance of the FPQL algorithm. The results show that FPQL algorithms attain a better balance between exploration and exploitation, and can also avoid local optimal policies and accelerate the learning process.

DOI
10.1109/tnnls.2013.2283574

Related papers

Open quantum system control based on reinforcement learning

Peng Wei, Na Li, Zairong Xi

The fast preparation of a quantum state with high fidelity is a key problem for a quantum system. We propose two modified algorithms based on the cutting-edge reinforcement learning methods for the flip from an initial state to a target one in a quantum spin system. The balance between exploration and exploitation and the size of a state space are key factors for reinforcement learning methods. In the first algorithm, we propose a modified $\epsilon$-greedy strategy instead of the ε-greedy strategy to balance exploration and exploitation. Furthermore, we use the fidelity of the final state as the reward and leverage piecewise-constant driving protocols in every duration. The results show that the rate of correction of this algorithm is greater than that of Q-learning with ε-greedy strategy. However, it will still cost much time resource if the rotation angle from a state to the next state is too small. Thus, we initialize the state-action value table to reduce the state space in the second modified Q-learning algorithm (MQL), which can be found that the efficiency of learning is greatly improved.

Chinese Control Conference (CCC)•Jul 1, 2019•3 citations

Control design of two-level quantum systems with reinforcement learning

Haixu Yu, Xudong Xu, Hailan Ma +2 more

In recent years, some experimental studies and simulations show that reinforcement learning (RL) is an effective learning control approach for solving certain quantum control problems. In this paper, Q-learning with different exploration strategies (e.g., ε-greedy and Softmax), probabilistic Q-learning (PQL) and quantum reinforcement learning (QRL) are applied to solve the state transition problem of two-level quantum systems (e.g., spin-1/2 systems), respectively. These reinforcement learning algorithms are introduced and analyzed regarding the learning control problem of the spin-1/2 system. According to the constraints of the control fields, two typical kinds of controllers, i.e., three-switch controller and Bang-Bang controller, are designed using reinforcement learning. The learning performance of the above RL algorithms for both of the three-switch control and Bang-Bang control of two-level quantum systems are demonstrated and analyzed.

Youth Academic Annual Conference of Chinese Association of Automation (YAC)•May 1, 2018•5 citations

A Quantum-inspired Q-learning Algorithm for Indoor Robot Navigation

Chunlin Chen, Pei Yang, Xianzhong Zhou +1 more

A quantum-inspired Q-learning (QIQL) algorithm is proposed for indoor robot navigation control. Q- learning is an action-dependent reinforcement learning method and has been widely used in robot navigation. Inspired by the fundamental characteristics of quantum computation, e.g. state superposition principle and quantum parallel computation, probability is introduced to Q-learning and along with the learning process the probability of each action to be selected at a certain state is updated, which leads to a natural exploration strategy instead of a pointed one with configured parameters. The simulated navigation experiments show that the proposed QIQL algorithm keeps a good balance of exploration and exploitation, which can avoid the local optimal policies and accelerate the learning process as well.

IEEE International Conference on Networking, Sensing and Control (ICNSC)•Apr 1, 2008•12 citations

Fidelity-Based Ant Colony Optimization for Control of Quantum System

Hang Zhang, Yifeng Tu, Ying Guo

The search time of traditional ant colony Algorithm (ACA) is too long, and it tends to trap in local optimizations. In this paper, a fidelity-based ACA with Q-learning is presented to solve above-mentioned problem and applied for control of quantum system. In the fidelity-based ACA with Q-learning, fidelity is used in heuristic function, which can help the system converge to the optimal solution. And the fidelity-based ACA with Q-learning can simplify the parameter setting of ACA algorithm. An example (a spin-1/2 system) is demonstrated to test the performance of the fidelity-based ACA with Q-learning. The results show that the fidelity-based ACA with Q-learning can efficiently avoid trapping in local optimal policies and increase the speed of convergence process.

International Conference on Smart Grid and Electrical Automation (ICSGEA)•May 1, 2017•1 citation

Guided Reward Design in Continuous Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•0 citations

A Modified Deep Q-Learning Algorithm for Control of Two-qubit Systems

Omar Shindi, Qi Yu, Daoyi Dong

Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 17, 2021•3 citations
The balance between exploration and exploitation is a key problem for reinforcement learning methods, especially for Q-learning. In this paper, a fidelity-based probabilistic Q-learning (FPQL) approach is presented to naturally solve this problem and applied for learning control of quantum systems. In this approach, fidelity is adopted to help direct the learning process and the probability of each action to be selected at a certain state is updated iteratively along with the learning process, which leads to a natural exploration strategy instead of a pointed one with configured parameters. A probabilistic Q-learning (PQL) algorithm is first presented to demonstrate the basic idea of probabilistic action selection. Then the FPQL algorithm is presented for learning control of quantum systems. Two examples (a spin-1/2 system and a Λ-type atomic system) are demonstrated to test the performance of the FPQL algorithm. The results show that FPQL algorithms attain a better balance between exploration and exploitation, and can also avoid local optimal policies and accelerate the learning process.
The fast preparation of a quantum state with high fidelity is a key problem for a quantum system. We propose two modified algorithms based on the cutting-edge reinforcement learning methods for the flip from an initial state to a target one in a quantum spin system. The balance between exploration and exploitation and the size of a state space are key factors for reinforcement learning methods. In the first algorithm, we propose a modified ϵ\epsilonϵ-greedy strategy instead of the ε-greedy strategy to balance exploration and exploitation. Furthermore, we use the fidelity of the final state as the reward and leverage piecewise-constant driving protocols in every duration. The results show that the rate of correction of this algorithm is greater than that of Q-learning with ε-greedy strategy. However, it will still cost much time resource if the rotation angle from a state to the next state is too small. Thus, we initialize the state-action value table to reduce the state space in the second modified Q-learning algorithm (MQL), which can be found that the efficiency of learning is greatly improved.
In recent years, some experimental studies and simulations show that reinforcement learning (RL) is an effective learning control approach for solving certain quantum control problems. In this paper, Q-learning with different exploration strategies (e.g., ε-greedy and Softmax), probabilistic Q-learning (PQL) and quantum reinforcement learning (QRL) are applied to solve the state transition problem of two-level quantum systems (e.g., spin-1/2 systems), respectively. These reinforcement learning algorithms are introduced and analyzed regarding the learning control problem of the spin-1/2 system. According to the constraints of the control fields, two typical kinds of controllers, i.e., three-switch controller and Bang-Bang controller, are designed using reinforcement learning. The learning performance of the above RL algorithms for both of the three-switch control and Bang-Bang control of two-level quantum systems are demonstrated and analyzed.
A quantum-inspired Q-learning (QIQL) algorithm is proposed for indoor robot navigation control. Q- learning is an action-dependent reinforcement learning method and has been widely used in robot navigation. Inspired by the fundamental characteristics of quantum computation, e.g. state superposition principle and quantum parallel computation, probability is introduced to Q-learning and along with the learning process the probability of each action to be selected at a certain state is updated, which leads to a natural exploration strategy instead of a pointed one with configured parameters. The simulated navigation experiments show that the proposed QIQL algorithm keeps a good balance of exploration and exploitation, which can avoid the local optimal policies and accelerate the learning process as well.
The search time of traditional ant colony Algorithm (ACA) is too long, and it tends to trap in local optimizations. In this paper, a fidelity-based ACA with Q-learning is presented to solve above-mentioned problem and applied for control of quantum system. In the fidelity-based ACA with Q-learning, fidelity is used in heuristic function, which can help the system converge to the optimal solution. And the fidelity-based ACA with Q-learning can simplify the parameter setting of ACA algorithm. An example (a spin-1/2 system) is demonstrated to test the performance of the fidelity-based ACA with Q-learning. The results show that the fidelity-based ACA with Q-learning can efficiently avoid trapping in local optimal policies and increase the speed of convergence process.
Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.