Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2601.01252

Harnessing Environmental Memory with Reinforcement Learning in Open Quantum Systems

Safae Gaidi, Abdallah Slaoui, Mohammed EL Falaki, Amine Jaouadi

quant-phSubmitted Jan 3, 2026• Announced Jan 6, 2026• 0 citations• Revised Jan 9, 2026 (v2)
v1 · Jan 3, 2026v2 · Jan 9, 2026v3 · Jul 23, 2026
arXivHTMLPDF

Abstract

Non-Markovian memory effects in open quantum systems provide valuable resources for preserving coherence and enhancing controllability. However, exploiting them requires strategies adapted to history-dependent dynamics. We introduce a reinforcement-learning framework that autonomously learns to amplify information backflow in a driven two-level system coupled to a structured reservoir. Using a reward based on the positive time derivative of the trace distance associated with the Breuer-Laine-Piilo measure, we train PPO and SAC agents and benchmark their performance against gradient-based optimal control theory (OCT). While OCT enhances a single dominant backflow peak, RL policies broaden this revival and activate additional contributions in later memory windows, producing sustained positive trace-distance growth over a longer duration. Consequently, the integrated non-Markovianity achieved by RL substantially exceeds that obtained with OCT. These results demonstrate how long-horizon, model-free learning naturally uncovers distributed-backflow strategies and highlight the potential of reinforcement learning for engineering memory effects in open quantum systems.

Comments
13 pages, 8 figures

Related papers

Non-Markovian Quantum Control via Model Maximum Likelihood Estimation and Reinforcement Learning

Tanmay Neema, Susmit Jha, Tuhin Sahai

Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.

arXiv:2402.05084•Feb 7, 2024•0 citations

Non-Markovian dissipation as a resource for quantum reservoir computing

Luca Nigro, Francesco Monzani, Enrico Prati

The control of open-system dynamics provides a powerful mechanism for using quantum information to process sequential tasks. While quantum reservoir computing typically relies on Markovian dissipation to process sequential data, the computational role of non-Markovian memory effects remains largely unexplored. We introduce a framework for quantum reservoir computing where non-Markovianity is explicitly modeled and regulated using fractional derivatives. By employing fractional time subordination, we generate tunable, heavy-tailed relaxation dynamics that govern the information backflow between the system and its environment. Non-Markovian information backflow regulates the overall temporal retention of the system, maximizing short-term linear memory capacity. Within an optimal operating regime, fractional non-Markovianity redistributes the reservoir memory towards recent inputs, improving both linear memory capacity and nonlinear prediction accuracy at short delays, at the expense of long-range temporal retention. System-environment interaction proves being not merely a requirement for quantum reservoir computing, but an active resource that embeds memory retention directly into the quantum evolution.

quant-pharXiv:2609.40172•Oct 1, 2026•0 citations

Re-exploring control strategies in a non-Markovian open quantum system by reinforcement learning

Amine Jaouadi, Étienne Mangaud, Michèle Desouter-Lecomte

In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the ultraviolet (UV) range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states which leads to a strong perturbation by the nuclear motions. We modify an open source software by Giannelli et al. [L. Giannelli et al., Phys. Lett. A 434, 128054 (2022)] that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding reservoir either by time-dependent rates, or more exactly, by using the hierarchical equations of motion with the QuTip-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.

arXiv:2312.11853•Physical Review A•Jan 16, 2024•5 citations

Harnessing non-Markovian quantum memory by environmental coupling

Zhong‐Xiao Man, Yun‐Jie Xia, Rosario Lo Franco

Controlling the non-Markovian dynamics of open quantum systems is essential in quantum information technology since it plays a crucial role in preserving quantum memory. Albeit in many realistic scenarios the quantum system can simultaneously interact with composite environments, this condition remains little understood, particularly regarding the effect of the coupling between environmental parts. We analyze the non-Markovian behavior of a qubit interacting at the same time with two coupled single-mode cavities which in turn dissipate into memoryless or memory-keeping reservoirs. We show that increasing the control parameter, that is the two-mode coupling, allows for triggering and enhancing a non-Markovian dynamics for the qubit starting from a Markovian one in the absence of coupling. Surprisingly, if the qubit dynamics is non-Markovian for the zero control parameter, increasing the latter enables multiple transitions from non-Markovian to Markovian regimes. These results hold independently on the nature of the reservoirs. This work highlights that suitably engineering the coupling between parts of a compound environment can efficiently harness the quantum memory, stored in a qubit, based on non-Markovianity.

arXiv:1506.08293•Physical Review A•Jul 13, 2015•93 citations

Hamiltonian-Driven Architectures for Non-Markovian Quantum Reservoir Computing

Daiki Sasaki, Ryosuke Koga, Kuroiwa, Taihei +3 more

We propose a Hamiltonian-level framework for non-Markovian quantum reservoir computing directly tailored for analog hardware implementations. By dividing the reservoir into a system block and an environment block and evolving their joint state under a unified Hamiltonian, our architecture naturally embeds memory backflow by harnessing entanglement-induced information backflow with tunable coupling strengths. Numerical benchmarks on short-term memory tasks demonstrate that operating in non-Markovian regimes yields significantly slower memory decay compared to the Markovian limit. Further analyzing the echo-state property (ESP), showing that the non-Markovian quantum reservoir evolves from two different initial states, they do not converge to the same trajectory even after a long time, strongly suggesting that the ESP is effectively violated. Our work provides the first demonstration in quantum reservoir computing that strong non-Markovianity can fundamentally violate the ESP, such that conventional linear-regression readouts fail to deliver stable training and inference. Finally, we experimentally showed that, with an appropriate time-evolution step size, the non-Markovian reservoir exhibits superior performance on higher-order nonlinear autoregressive moving-average(NARMA) tasks.

arXiv:2505.14450•May 20, 2025•0 citations

Learning Non-Markovian Noise via Ensemble Optimal Control

Da-Wei Luo, Ting Yu

We study the estimation of parameters pertaining to non-Markovian quantum open systems, such as the dissipation rate and environmental memory time. A key challenge is identifying the optimal measurement time, which must allow sufficient time to acquire information about the environment, yet be short enough to avoid dissipation that erases the information. Using machine learning approaches, we develop an optimized control scheme trained over a representative ensemble to fix the optimal measurement time at a prescribed runtime. The protocol is robust to errors in the training process, enhances precision by exploiting non-Markovian memory effects, and achieves measurement uncertainties approaching the quantum limits set by the Cramér-Rao bound.

quant-pharXiv:2604.16784•Apr 21, 2026•0 citations
Non-Markovian memory effects in open quantum systems provide valuable resources for preserving coherence and enhancing controllability. However, exploiting them requires strategies adapted to history-dependent dynamics. We introduce a reinforcement-learning framework that autonomously learns to amplify information backflow in a driven two-level system coupled to a structured reservoir. Using a reward based on the positive time derivative of the trace distance associated with the Breuer-Laine-Piilo measure, we train PPO and SAC agents and benchmark their performance against gradient-based optimal control theory (OCT). While OCT enhances a single dominant backflow peak, RL policies broaden this revival and activate additional contributions in later memory windows, producing sustained positive trace-distance growth over a longer duration. Consequently, the integrated non-Markovianity achieved by RL substantially exceeds that obtained with OCT. These results demonstrate how long-horizon, model-free learning naturally uncovers distributed-backflow strategies and highlight the potential of reinforcement learning for engineering memory effects in open quantum systems.
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.
The control of open-system dynamics provides a powerful mechanism for using quantum information to process sequential tasks. While quantum reservoir computing typically relies on Markovian dissipation to process sequential data, the computational role of non-Markovian memory effects remains largely unexplored. We introduce a framework for quantum reservoir computing where non-Markovianity is explicitly modeled and regulated using fractional derivatives. By employing fractional time subordination, we generate tunable, heavy-tailed relaxation dynamics that govern the information backflow between the system and its environment. Non-Markovian information backflow regulates the overall temporal retention of the system, maximizing short-term linear memory capacity. Within an optimal operating regime, fractional non-Markovianity redistributes the reservoir memory towards recent inputs, improving both linear memory capacity and nonlinear prediction accuracy at short delays, at the expense of long-range temporal retention. System-environment interaction proves being not merely a requirement for quantum reservoir computing, but an active resource that embeds memory retention directly into the quantum evolution.
In this study, we reexamine a recent optimal control simulation targeting the preparation of a superposition of two excited electronic states in the ultraviolet (UV) range in a complex molecular system. We revisit this control from the perspective of reinforcement learning, offering an efficient alternative to conventional quantum control methods. The two excited states are addressable by orthogonal polarizations and their superposition corresponds to a right or left localization of the electronic density. The pulse duration spans tens of femtoseconds to prevent excitation of higher excited bright states which leads to a strong perturbation by the nuclear motions. We modify an open source software by Giannelli et al. [L. Giannelli et al., Phys. Lett. A 434, 128054 (2022)] that implements reinforcement learning with Lindblad dynamics, to introduce non-Markovianity of the surrounding reservoir either by time-dependent rates, or more exactly, by using the hierarchical equations of motion with the QuTip-BoFiN package. This extension opens the way to wider applications for non-Markovian environments, in particular when the active system interacts with a highly structured noise.
Controlling the non-Markovian dynamics of open quantum systems is essential in quantum information technology since it plays a crucial role in preserving quantum memory. Albeit in many realistic scenarios the quantum system can simultaneously interact with composite environments, this condition remains little understood, particularly regarding the effect of the coupling between environmental parts. We analyze the non-Markovian behavior of a qubit interacting at the same time with two coupled single-mode cavities which in turn dissipate into memoryless or memory-keeping reservoirs. We show that increasing the control parameter, that is the two-mode coupling, allows for triggering and enhancing a non-Markovian dynamics for the qubit starting from a Markovian one in the absence of coupling. Surprisingly, if the qubit dynamics is non-Markovian for the zero control parameter, increasing the latter enables multiple transitions from non-Markovian to Markovian regimes. These results hold independently on the nature of the reservoirs. This work highlights that suitably engineering the coupling between parts of a compound environment can efficiently harness the quantum memory, stored in a qubit, based on non-Markovianity.
We propose a Hamiltonian-level framework for non-Markovian quantum reservoir computing directly tailored for analog hardware implementations. By dividing the reservoir into a system block and an environment block and evolving their joint state under a unified Hamiltonian, our architecture naturally embeds memory backflow by harnessing entanglement-induced information backflow with tunable coupling strengths. Numerical benchmarks on short-term memory tasks demonstrate that operating in non-Markovian regimes yields significantly slower memory decay compared to the Markovian limit. Further analyzing the echo-state property (ESP), showing that the non-Markovian quantum reservoir evolves from two different initial states, they do not converge to the same trajectory even after a long time, strongly suggesting that the ESP is effectively violated. Our work provides the first demonstration in quantum reservoir computing that strong non-Markovianity can fundamentally violate the ESP, such that conventional linear-regression readouts fail to deliver stable training and inference. Finally, we experimentally showed that, with an appropriate time-evolution step size, the non-Markovian reservoir exhibits superior performance on higher-order nonlinear autoregressive moving-average(NARMA) tasks.
We study the estimation of parameters pertaining to non-Markovian quantum open systems, such as the dissipation rate and environmental memory time. A key challenge is identifying the optimal measurement time, which must allow sufficient time to acquire information about the environment, yet be short enough to avoid dissipation that erases the information. Using machine learning approaches, we develop an optimized control scheme trained over a representative ensemble to fix the optimal measurement time at a prescribed runtime. The protocol is robust to errors in the training process, enhances precision by exploiting non-Markovian memory effects, and achieves measurement uncertainties approaching the quantum limits set by the Cramér-Rao bound.