arXiv:2601.01252

Harnessing Environmental Memory with Reinforcement Learning in Open Quantum Systems

Safae Gaidi, Abdallah Slaoui, Mohammed EL Falaki, Amine Jaouadi

quant-phSubmitted Jan 3, 2026• Announced Jan 6, 2026• 0 citations• Revised Jul 23, 2026 (v3)

Abstract

Non-Markovian quantum dynamics, characterized by information backflow from the environment to the system, has emerged as a potential resource for quantum technologies. A key challenge is therefore to control and enhance such memory effects. In this work, we investigate the use of reinforcement learning (RL) to maximize non-Markovianity in a driven two-level system coupled to a structured reservoir. We compare RL-based control strategies with standard optimal control theory (OCT). We show that OCT produces localized but relatively weak revivalsin the instantaneous non-Markovianity rate, whereas RL policies generate significantly stronger and better-timed information backflow by synchronizing the system dynamics with favorable memory intervals of the environment. This enhanced exploitation of memory effects leads to a higher total integrated non-Markovianity for RL than for OCT, with SAC achieving the largest overall enhancement and PPO delivering slightly lower but still strongly improved performance with smoother, experimentally attractive pulses. Our results contribute to the emerging view of non-Markovianity as an operational resource and illustrate how RL can serve as a flexible, model-free tool for non-Markovian quantum control.

Comments: 16 pages, 11 figures

Related papers

Non-Markovian Quantum Control via Model Maximum Likelihood Estimation and Reinforcement Learning

Tanmay Neema, Susmit Jha, Tuhin Sahai

Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.

arXiv:2402.05084Feb 7, 20240 citations

Non-Markovian dissipation as a resource for quantum reservoir computing

Luca Nigro, Francesco Monzani, Enrico Prati

The control of open-system dynamics provides a powerful mechanism for using quantum information to process sequential tasks. While quantum reservoir computing typically relies on Markovian dissipation to process sequential data, the computational role of non-Markovian memory effects remains largely unexplored. We introduce a framework for quantum reservoir computing where non-Markovianity is explicitly modeled and regulated using fractional derivatives. By employing fractional time subordination, we generate tunable, heavy-tailed relaxation dynamics that govern the information backflow between the system and its environment. Non-Markovian information backflow regulates the overall temporal retention of the system, maximizing short-term linear memory capacity. Within an optimal operating regime, fractional non-Markovianity redistributes the reservoir memory towards recent inputs, improving both linear memory capacity and nonlinear prediction accuracy at short delays, at the expense of long-range temporal retention. System-environment interaction proves being not merely a requirement for quantum reservoir computing, but an active resource that embeds memory retention directly into the quantum evolution.

quant-pharXiv:2609.40172Oct 1, 20260 citations

Hamiltonian-Driven Architectures for Non-Markovian Quantum Reservoir Computing

Daiki Sasaki, Ryosuke Koga, Kuroiwa, Taihei +3 more

We propose a Hamiltonian-level framework for non-Markovian quantum reservoir computing directly tailored for analog hardware implementations. By dividing the reservoir into a system block and an environment block and evolving their joint state under a unified Hamiltonian, our architecture naturally embeds memory backflow by harnessing entanglement-induced information backflow with tunable coupling strengths. Numerical benchmarks on short-term memory tasks demonstrate that operating in non-Markovian regimes yields significantly slower memory decay compared to the Markovian limit. Further analyzing the echo-state property (ESP), showing that the non-Markovian quantum reservoir evolves from two different initial states, they do not converge to the same trajectory even after a long time, strongly suggesting that the ESP is effectively violated. Our work provides the first demonstration in quantum reservoir computing that strong non-Markovianity can fundamentally violate the ESP, such that conventional linear-regression readouts fail to deliver stable training and inference. Finally, we experimentally showed that, with an appropriate time-evolution step size, the non-Markovian reservoir exhibits superior performance on higher-order nonlinear autoregressive moving-average(NARMA) tasks.

arXiv:2505.14450May 20, 20250 citations

Re-exploring control strategies in a non-Markovian open quantum system by reinforcement learning

Amine Jaouadi, Étienne Mangaud, Michèle Desouter-Lecomte

In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the ultraviolet (UV) range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states which leads to a strong perturbation by the nuclear motions. We modify an open source software by Giannelli et al. [L. Giannelli et al., Phys. Lett. A 434, 128054 (2022)] that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding reservoir either by time-dependent rates, or more exactly, by using the hierarchical equations of motion with the QuTip-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.

arXiv:2312.11853Physical Review AJan 16, 20245 citations

Learning Non-Markovian Noise via Ensemble Optimal Control

Da-Wei Luo, Ting Yu

We study the estimation of parameters pertaining to non-Markovian quantum open systems, such as the dissipation rate and environmental memory time. A key challenge is identifying the optimal measurement time, which must allow sufficient time to acquire information about the environment, yet be short enough to avoid dissipation that erases the information. Using machine learning approaches, we develop an optimized control scheme trained over a representative ensemble to fix the optimal measurement time at a prescribed runtime. The protocol is robust to errors in the training process, enhances precision by exploiting non-Markovian memory effects, and achieves measurement uncertainties approaching the quantum limits set by the Cramér-Rao bound.

quant-pharXiv:2604.16784Apr 21, 20260 citations

Memory-enhanced quantum extreme learning machines for characterizing non-Markovian dynamics

Hajar Assil, Abderrahim El Allati, Gian Luca Giorgi

We use a Quantum Extreme Learning Machine for characterizing and estimating parameters of quantum dynamics generated by a tunable collision model. The input to the learning protocol consists of quantum states produced by successive system environment interactions, while the reservoir is implemented as a disordered many body quantum system evolving under a fixed Hamiltonian. We systematically explore how extending the QELM feature space, through the inclusion of temporal information and additional observables, affects estimation performance. Our results demonstrate that temporal extensions of the feature vector consistently and significantly enhance estimation accuracy relative to the baseline protocol. Notably, incorporating memory from earlier time steps yields the most substantial and robust improvements, whereas extensions based solely on additional observables offer only marginal gains. Crucially, the advantage conferred by temporal memory becomes increasingly pronounced as the dynamics become more strongly non Markovian, indicating that environmental memory effects serve as a constructive resource for learning.

quant-pharXiv:2603.17182Physical Review ResearchMar 19, 20260 citations