We introduce a reinforcement learning algorithm designed to identify the fixed points of a given quantum operation. The method iteratively constructs the unitary transformation that maps the computational basis onto the basis of fixed points through a reward-penalty scheme based on quantum measurements. In cases where the operation corresponds to a Hamiltonian evolution, this task reduces to determining the Hamiltonian eigenstates. The algorithm is first benchmarked on random Hamiltonians acting on two and three qubits and then applied to many-body systems of up to six qubits, including the transverse-field Ising model and the all-to-all pairing Hamiltonian. In both cases, the algorithm is demonstrated to perform successfully; in the pairing model, it can also reveal hidden symmetries, which can be exploited to restrict learning to specific symmetry sectors. Finally, we discuss the possibility of post-selecting high-fidelity states even when full convergence has not been reached.
We introduce a reinforcement learning algorithm designed to identify the fixed points of a given quantum operation. The method iteratively constructs the unitary transformation that maps the computational basis onto the basis of fixed points through a reward-penalty scheme based on quantum measurements. In cases where the operation corresponds to a Hamiltonian evolution, this task reduces to determining the Hamiltonian eigenstates. The algorithm is first benchmarked on random Hamiltonians acting on two and three qubits and then applied to many-body systems of up to six qubits, including the transverse-field Ising model and the all-to-all pairing Hamiltonian. In both cases, the algorithm is demonstrated to perform successfully; in the pairing model, it can also reveal hidden symmetries, which can be exploited to restrict learning to specific symmetry sectors. Finally, we discuss the possibility of post-selecting high-fidelity states even when full convergence has not been reached.
This paper investigates the application of quantum-inspired algorithms to the field of reinforcement learning (RL). Traditional RL methods often struggle with exploration challenges and can be computationally expensive, particularly in complex environments. We propose a novel RL algorithm that leverages concepts from quantum mechanics, specifically superposition and entanglement, to address these limitations. The core idea is to represent states, actions, and rewards using quantum-inspired representations, allowing the agent to explore the environment more efficiently and potentially converge to optimal policies faster. This work presents a detailed formulation of the algorithm, including the mathematical framework and key considerations for implementation. The potential benefits of this approach, including improved exploration and reduced computational complexity, are discussed. The algorithm is designed to iteratively refine its policy based on simulated quantum measurements, aiming to minimize the expected return. The exploration strategy is guided by the inherent probabilistic nature of quantum states, leading to a more diverse and potentially faster learning process. Further research is warranted to fully explore the capabilities and limitations of this approach across various RL benchmark problems.
Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
The key approaches for machine learning, particularly learning in unknown probabilistic environments, are new representations and computation mechanisms. In this paper, a novel quantum reinforcement learning (QRL) method is proposed by combining quantum theory and reinforcement learning (RL). Inspired by the state superposition principle and quantum parallelism, a framework of a value-updating algorithm is introduced. The state (action) in traditional RL is identified as the eigen state (eigen action) in QRL. The state (action) set can be represented with a quantum superposition state, and the eigen state (eigen action) can be obtained by randomly observing the simulated quantum state according to the collapse postulate of quantum measurement. The probability of the eigen action is determined by the probability amplitude, which is updated in parallel according to rewards. Some related characteristics of QRL such as convergence, optimality, and balancing between exploration and exploitation are also analyzed, which shows that this approach makes a good tradeoff between exploration and exploitation using the probability amplitude and can speedup learning through the quantum parallelism. To evaluate the performance and practicability of QRL, several simulated experiments are given, and the results demonstrate the effectiveness and superiority of the QRL algorithm for some complex problems. This paper is also an effective exploration on the application of quantum computation to artificial intelligence.
The computation of electronic excited states and real-time quantum dynamics of many-fermion systems is among the most promising applications of near-term quantum computing. In this work, we generalize the reinforcement learning contracted quantum eigensolver (RL-CQE), previously developed for ground-state problems, to electronic excited states and real-time quantum dynamics, in which a deep Q-network agent adaptively selects the two-body operators at each iteration, yielding more compact ansätze and improved robustness with respect to critical hyperparameters. A key feature of the algorithm is a scalable state representation based on the ACSE residuals, whose dimension grows with the one-particle basis but remains independent of the number of targeted excited states. We also verify the equivalence of sign-free qubit operators in the excited-state setting, extending a result previously established for ground-state problems. Our RL-CQE for time evolution derives from a constant-scaling ansatz that represents the wave function with a fixed number of unitary transformations independent of simulation time t, enabled by the shared unitary structure of the purified ensemble treatment of excited states. Benchmarks on chemical systems demonstrate chemical accuracy with minimal operator counts across a range of bond lengths.
This paper addresses the critical challenge of resource allocation in non-homogeneous quantum computing environments, where quantum bits (qubits) exhibit significantly varying characteristics. We propose a novel reinforcement learning-based algorithm for optimizing resource allocation, specifically designed to identify the optimal execution path for quantum computing tasks and maximize qubit utilization. The core of our approach lies in building a probabilistic model representing the quantum computing environment and employing a reinforcement learning agent to learn the best qubit allocation strategy through trial and error. This algorithm overcomes the limitations of existing methods that assume uniform qubit properties, leading to improved efficiency and success rates in quantum computations. The algorithm's core equation is as follows: * *E* = ∑i=1K *ri* (1 - *pi*) where *E* represents the overall execution efficiency, *K* is the number of qubits, *ri* is the allocated resource for qubit *i*, and *pi* is the probability of failure for qubit *i*. The algorithm's objective is to minimize *E*. Furthermore, the reward function is defined as: * R = (1 - E)* where *R* is the reward and *E* is the execution efficiency. The algorithm learns by maximizing this reward. The probabilistic model incorporates qubit characteristics such as coherence time, gate fidelity, and coupling strength, represented as probability distributions. The reinforcement learning agent utilizes a Q-learning algorithm to estimate the optimal Q-values, representing the expected reward for each state-action pair. The state space includes qubit characteristics and task requirements, while the action space consists of qubit allocation decisions. The algorithm iteratively updates its Q-values based on observed rewards, converging towards the optimal resource allocation strategy. We demonstrate that this approach offers a significant improvement over traditional uniform qubit allocation methods, particularly in scenarios with highly heterogeneous qubit populations. The primary contribution of this work is the integration of probabilistic modeling and reinforcement learning to effectively address the complexities of non-homogeneous quantum computing.
The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.