In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the ultraviolet (UV) range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states which leads to a strong perturbation by the nuclear motions. We modify an open source software by Giannelli et al. [L. Giannelli et al., Phys. Lett. A 434, 128054 (2022)] that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding reservoir either by time-dependent rates, or more exactly, by using the hierarchical equations of motion with the QuTip-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.
In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the ultraviolet (UV) range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states which leads to a strong perturbation by the nuclear motions. We modify an open source software by Giannelli et al. [L. Giannelli et al., Phys. Lett. A 434, 128054 (2022)] that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding reservoir either by time-dependent rates, or more exactly, by using the hierarchical equations of motion with the QuTip-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.
Non-Markovian quantum dynamics, characterized by information backflow from the environment to the system, has emerged as a potential resource for quantum technologies. A key challenge is therefore to control and enhance such memory effects. In this work, we investigate the use of reinforcement learning (RL) to maximize non-Markovianity in a driven two-level system coupled to a structured reservoir. We compare RL-based control strategies with standard optimal control theory (OCT). We show that OCT produces localized but relatively weak revivalsin the instantaneous non-Markovianity rate, whereas RL policies generate significantly stronger and better-timed information backflow by synchronizing the system dynamics with favorable memory intervals of the environment. This enhanced exploitation of memory effects leads to a higher total integrated non-Markovianity for RL than for OCT, with SAC achieving the largest overall enhancement and PPO delivering slightly lower but still strongly improved performance with smoother, experimentally attractive pulses. Our results contribute to the emerging view of non-Markovianity as an operational resource and illustrate how RL can serve as a flexible, model-free tool for non-Markovian quantum control.
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.
We present a general method to efficiently design optimal control sequences for non-Markovian open quantum systems, and illustrate it by optimizing the shape of a laser pulse to prepare a quantum dot in a specific state. The optimization of control procedures for quantum systems with strong coupling to structured environments-where time-local descriptions fail-is a computationally challenging task. We modify the numerically exact time evolving matrix product operator (TEMPO) method, such that it allows the repeated computation of the time evolution of the reduced system density matrix for various sets of control parameters at very low computational cost. This method is potentially useful for studying numerous optimal control problems, in particular in solid state quantum devices where the coupling to vibrational modes is typically strong.
The fast preparation of a quantum state with high fidelity is a key problem for a quantum system. We propose two modified algorithms based on the cutting-edge reinforcement learning methods for the flip from an initial state to a target one in a quantum spin system. The balance between exploration and exploitation and the size of a state space are key factors for reinforcement learning methods. In the first algorithm, we propose a modified ϵ-greedy strategy instead of the ε-greedy strategy to balance exploration and exploitation. Furthermore, we use the fidelity of the final state as the reward and leverage piecewise-constant driving protocols in every duration. The results show that the rate of correction of this algorithm is greater than that of Q-learning with ε-greedy strategy. However, it will still cost much time resource if the rotation angle from a state to the next state is too small. Thus, we initialize the state-action value table to reduce the state space in the second modified Q-learning algorithm (MQL), which can be found that the efficiency of learning is greatly improved.
Recent advances in quantum technologies and related experiments have created a need for highly accurate, versatile, and computationally efficient simulation techniques for the dynamics of open quantum systems. Long-lived correlation effects (non-Markovianity), system-environment hybridization, and the necessity for accuracy beyond the Born-Markov approximation form particular challenges. Approaches to meet these challenges have been introduced, originating from different fields, such as hierarchical equations of motion, Lindblad-pseudomode formulas, chain-mapping approaches, quantum Brownian motion master equations, stochastic unravelings, and refined quantum master equations. This diversity, while indicative of the field's relevance, has inadvertently led to a fragmentation that hinders cohesive advances and their effective cross-community application to current problems for complex systems. How are different approaches related to each other? What are their strengths and limitations? Here we give a systematic overview and concise discussion addressing these questions. We make use of a unified framework which very conveniently allows to link different schemes and, this way, may also catalyze further progress. In line with the state of the art, this framework is formulated not in a fully reduced space of the system but in an extended state space which in a minimal fashion includes effective reservoir modes. This in turn offers a comprehensive understanding of existing methods, elucidating their physical interpretations, interconnections, and applicability.
We address a wide spectrum of quantum control strategies, including various open-loop protocols and advanced adaptive methods. These methodologies apply to few-qubit scenarios and naturally scale to larger N-qubit systems. We benchmark them across fundamental quantum tasks: entanglement preservation/generation, and directed quantum transport in a disordered quantum walk. All simulations are performed in a challenging environment featuring non-Markov colored noise, imperfections, and the Markov Lindblad equation. With a complex task-dependent performance hierarchy, our deterministic protocols proved highly effective for entanglement generation/preservation, and in specific pulse configurations, they even outperformed the RL-optimization. In contrast, more advanced methods demonstrate a marked specialization. For entanglement preservation, a physics-informed hybrid Quantum Error Correction and Dynamical Decoupling (QEC-DD) protocol provides the most stable and effective solution, outperforming all other approaches. Conversely, for dynamic tasks requiring the discovery of non-trivial control sequences, such as DD, Floquet engineering, and rapid entanglement generation or coherent transport, the model-free Reinforcement Learning (RL) agents consistently find superior solutions. We further demonstrate that the control pulse envelope is a non-trivial factor that actively shapes the control landscape, which determines the difficulty for all protocols and highlights the adaptability of the RL agent. We conclude that no single strategy is universally dominant. A clear picture emerges: the future of high-fidelity quantum control lies in a synthesis of physics-informed design, as exemplified by robust hybrid methods, and the specialized, high-performance optimization power of adaptive machine learning.