Abstract We propose a bottom–up approach, based on reinforcement learning, to the design of a chain achieving efficient excitation-transfer performances. We assume distance-dependent interactions among particles arranged in a chain under tight-binding conditions. Starting from two particles and a localised excitation, we gradually increase the number of constitutents of the system so as to improve the transfer probability. We formulate the problem of finding the optimal locations and numbers of particles as a Markov decision process: we use proximal policy optimization to find the optimal chain-building policies and the optimal chain configurations under different scenarios. We consider both the case in which the target is a sink connected to the end of the chain and the case in which the target is the right-most particle in the chain. We address the problem of disorder in the chain induced by particle positioning errors. We apply our methodology to a simplified model of a relevant physical platform, consisting of trapped ions. We are able to achieve extremely high excitation transfer in all cases, with different chain configurations and properties depending on the specific conditions.
Abstract We propose a bottom–up approach, based on reinforcement learning, to the design of a chain achieving efficient excitation-transfer performances. We assume distance-dependent interactions among particles arranged in a chain under tight-binding conditions. Starting from two particles and a localised excitation, we gradually increase the number of constitutents of the system so as to improve the transfer probability. We formulate the problem of finding the optimal locations and numbers of particles as a Markov decision process: we use proximal policy optimization to find the optimal chain-building policies and the optimal chain configurations under different scenarios. We consider both the case in which the target is a sink connected to the end of the chain and the case in which the target is the right-most particle in the chain. We address the problem of disorder in the chain induced by particle positioning errors. We apply our methodology to a simplified model of a relevant physical platform, consisting of trapped ions. We are able to achieve extremely high excitation transfer in all cases, with different chain configurations and properties depending on the specific conditions.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
Scalable trapped-ion quantum computing is commonly realized with modular chips that feature distinct zones with specific functionalities, such as storage, state preparation, and gate execution. To execute a quantum circuit, the ions must be transported between these zones. This process is called ion shuttling. To achieve reliable computation results, the shuttling process must be optimized. However, as the number of ions increases, this becomes a high-dimensional optimization problem where optimal solutions cannot be computed efficiently. We demonstrate, to the best of our knowledge, the first use of reinforcement learning (RL) for the optimization of ion shuttling. RL is well-suited for such scenarios, as it enables learning a strategy through direct interaction with the problem. We show that our RL approach outperforms current state-of-the-art heuristic techniques, yielding a reduction in shuttling operations of up to 36.3 %. Furthermore, we show that our method is easily applicable to various chip architectures. Our approach offers a versatile method to study shuttling efficiency during chip design and, therefore, a highly relevant tool for future, more complex architectures.
One of the ambitious goals of artificial intelligence is to build a machine that outperforms human intelligence, even if limited knowledge and data are provided. Reinforcement learning (RL) provides one such possibility to reach this goal. In this work, we consider a specific task from quantum physics, i.e., quantum state transfer in a one-dimensional spin chain. The mission for the machine is to find transfer schemes with the fastest speeds while maintaining high transfer fidelities. The first scenario we consider is when the Hamiltonian is time independent. We update the coupling strength by minimizing a loss function dependent on both the fidelity and the speed. Compared with a scheme proven to be at the quantum speed limit for the perfect state transfer, the scheme provided by RL is faster while maintaining the infidelity below $5\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}4}$. In the second scenario where a time-dependent external field is introduced, we convert the state transfer process into a Markov decision process that can be understood by the machine. We solve it with the deep Q-learning algorithm. After training, the machine successfully finds transfer schemes with high fidelities and speeds, which are faster than previously known ones. These results show that reinforcement learning can be a powerful tool for quantum control problems.
Quantum networks make use of the quantum entanglement as building block. When two qubits are entangled, their state changes exhibit non-classical correlations used to design new applications not possible with classical communication, such us quantum key distribution or distributed quantum computation. Unfortunately, quantum entanglement is a probabilistic process strongly dependent on the features of involved devices (optical fibers, lasers, quantum memories, …). The management decisions (i.e., the control policy) to set up and keep the entanglement as long as possible with the highest quality constitutes a stochastic control problem. This process can be modelled as Markov Decision Process (MDP) and solved via the Reinforcement Learning (RL) framework (a form of Machine Learning). In this work, we apply this RL framework to learn an entanglement management policy outperforming the State-of-the-Art policy when models characterising precisely the involved quantum devices are not known.
We propose a reinforcement learning (RL) scheme for feedback quantum control within the quantum approximate optimization algorithm (QAOA).We reformulate the QAOA variational minimization as a learning task, where an RL agent chooses the control parameters for the unitaries, given partial information on the system.Such an RL scheme finds a policy converging to the optimal adiabatic solution of the quantum Ising chain that can also be successfully transferred between systems with different sizes, even in the presence of disorder.This allows for immediate experimental verification of our proposal on more complicated models: the RL agent is trained on a small control system, simulated on classical hardware, and then tested on a larger physical sample.
The design of autonomous quantum systems constitutes an entirely new paradigm for next-generation computing by having machine intelligence do the engineering work to build and optimise successful quantum architectures. We address the problem of self-optimizing quantum systems that perform adaptive control, circuit synthesis and decision making in both accurate and uncertain controllable environments via a two-pronged approach: Reinforcement Learning (RL) together with Evolutionary Strategies (ES). In this work, we find that reinforcement learning provides a general framework for sequential decision-making, where agents learn optimal policies from reward signals by interacting with quantum environments. Evolutionary strategies, which are based on ideas from biological evolution, provide an alternative to RL by performing global optimization of quantum circuit parameters and architectures using the standard canonical processes of mutation, selection and recombination. Recent developments in quantum technologies show, again, that machine learning tools (ML), especially RL can be very useful to optimize unavailable closed blow-ups related with quantity functions. Moreover, evolutionary optimization based hybrids with RL have demonstrated better performance on the complex multi-objective quantum problems due to balanced exploitation and exploration. In this paper, we provide an overview of a versatile framework for autonomous quantum system design that showcases example algorithmic architectures, hybrid machine learning models and real world applications in areas such as quantum control, quantum circuit synthesis and energy optimization. In addition, we discuss well-known obstacles such as the scalability problem, the noise sensitivity issue, barren plateaus of quantum optimization and computational complexity. Comparisons to traditional methods show that reinforcement learning (RL) when combined with evolutionary strategies enables a significant improvement in efficiency, adaptability, and robustness for quantum system design as supported by experimental studies and case analyses. Future research directions are then discussed, including quantum-native learning algorithms, multi-agent quantum reinforcement learning and hardware-aware optimization techniques