Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2608.19779

An Irreducible Quantum Advantage in Aligning World Models with Reality

Josep Lumbreras, Hailan Ma, Jayne Thompson, Mile Gu

quant-phcs.AIcs.LGSubmitted Aug 20, 2026• Announced Aug 21, 2026• 0 citations• Revised Oct 2, 2026 (v2)
v1 · Aug 20, 2026v2 · Oct 2, 2026
arXivHTMLPDF

Abstract

World models provide digital simulacra of the true world, allowing agents to be trained and tested before costly real-world deployment. At each time step, they receive an action and generate an observation and reward matching the statistics of the true world. In complex environments where present outcomes depend on events far in the past, this requires memory. One might expect that, by increasing memory, we can always build a model accurately enough to align the optimal agent policies of the real and virtual worlds. We show that this is false for classical world models, even when the true world itself is classical. We construct true worlds for which every finite classical model fails along the same possible trajectory: it either loses the ability to distinguish actions when the true world clearly prefers one, or repeatedly assigns the highest expected reward to suboptimal actions. Its expected-reward estimates also retain a nonvanishing average error. In contrast, each such true world admits a quantum world model using a single qutrit that reproduces it exactly: its reward estimates and preferred actions always match those of the true world, ensuring that the optimal policies of the real and virtual worlds remain perfectly aligned.

Comments
36 pages, 8 figures

Related papers

Quantum Adaptive Agents with Efficient Long-Term Memories

Thomas J. Elliott, Mile Gu, Andrew J. P. Garner +1 more

Central to the success of adaptive systems is their ability to interpret signals from their environment and respond accordingly-they act as agents interacting with their surroundings.Such agents typically perform better when able to execute increasingly complex strategies.This comes with a cost: the more information the agent must recall from its past experiences, the more memory it will need.Here we investigate the power of agents capable of quantum information processing.We uncover the most general form a quantum agent need adopt to maximize memory compression advantages and provide a systematic means of encoding their memory states.We show these encodings can exhibit extremely favorable scaling advantages relative to memoryminimal classical agents, particularly when information must be retained about events increasingly far into the past.

arXiv:2108.10876•Physical Review X•Jan 11, 2022•19 citations

Provable superior accuracy in machine learned quantum models

Chengran Yang, Andrew J. P. Garner, Feiyang Liu +5 more

In modelling complex processes, the potential past data that influence future expectations are immense. Models that track all this data are not only computationally wasteful but also shed little light on what past data most influence the future. There is thus enormous interest in dimensional reduction-finding automated means to reduce the memory dimension of our models while minimizing its impact on its predictive accuracy. Here we construct dimensionally reduced quantum models by machine learning methods that can achieve greater accuracy than provably optimal classical counterparts. We demonstrate this advantage on present-day quantum computing hardware. Our algorithm works directly off classical time-series data and can thus be deployed in real-world settings. These techniques illustrate the immediate relevance of quantum technologies to time-series analysis and offer a rare instance where the resulting quantum advantage can be provably established.

arXiv:2105.14434•May 30, 2021•6 citations

Reinforcement learning for quantum processes with memory

Josep Lumbreras, Ruo Cheng Huang, Yanglin Hu +2 more

In reinforcement learning, an agent interacts sequentially with an environment to maximize a reward, receiving only partial, probabilistic feedback. This creates a fundamental exploration-exploitation trade-off: the agent must explore to learn the hidden dynamics while exploiting this knowledge to maximize its target objective. While extensively studied classically, applying this framework to quantum systems requires dealing with hidden quantum states that evolve via unknown dynamics. We formalize this problem via a framework where the environment maintains a hidden quantum memory evolving via unknown quantum channels, and the agent intervenes sequentially using quantum instruments. For this setting, we adapt an optimistic maximum-likelihood estimation algorithm. We extend the analysis to continuous action spaces, allowing us to model general positive operator-valued measures (POVMs). By controlling the propagation of estimation errors through quantum channels and instruments, we prove that the cumulative regret of our strategy scales as $\widetilde{\mathcal{O}}(\sqrt{K})$ over $K$ episodes. Furthermore, via a reduction to the multi-armed quantum bandit problem, we establish information-theoretic lower bounds demonstrating that this sublinear scaling is strictly optimal up to polylogarithmic factors. As a physical application, we consider state-agnostic work extraction. When extracting free energy from a sequence of non-i.i.d. quantum states correlated by a hidden memory, any lack of knowledge about the source leads to thermodynamic dissipation. In our setting, the mathematical regret exactly quantifies this cumulative dissipation. Using our adaptive algorithm, the agent uses past energy outcomes to improve its extraction protocol on the fly, achieving sublinear cumulative dissipation, and, consequently, an asymptotically zero dissipation rate.

quant-pharXiv:2603.25138•Mar 27, 2026•0 citations

Quantum Algorithms for Reinforcement Learning with a Generative Model

Daochen Wang, Aarthi Sundaram, Robin Kothari +2 more

Reinforcement learning studies how an agent should interact with an environment to maximize its cumulative reward. A standard way to study this question abstractly is to ask how many samples an agent needs from the environment to learn an optimal policy for a $γ$-discounted Markov decision process (MDP). For such an MDP, we design quantum algorithms that approximate an optimal policy ($π^*$), the optimal value function ($v^*$), and the optimal $Q$-function ($q^*$), assuming the algorithms can access samples from the environment in quantum superposition. This assumption is justified whenever there exists a simulator for the environment; for example, if the environment is a video game or some other program. Our quantum algorithms, inspired by value iteration, achieve quadratic speedups over the best-possible classical sample complexities in the approximation accuracy ($ε$) and two main parameters of the MDP: the effective time horizon ($\frac{1}{1-γ}$) and the size of the action space ($A$). Moreover, we show that our quantum algorithm for computing $q^*$ is optimal by proving a matching quantum lower bound.

arXiv:2112.08451•Dec 15, 2021•14 citations

How Quantum Agents Can Change Which Strategies Are More Complex

Spiros Kechrimparis, Nix Barnett, Mile Gu +1 more

Whether winning blackjack or navigating busy streets, achieving desired outcomes requires agents to execute adaptive strategies, strategies where actions depend contextually on past events. In complexity science, this motivates memory as an operational quantifier of complexity: given two strategies, the more complex one demands the agent to track more about the past. Here, we show that conclusions about complexity fundamentally depend on whether agents can process and store quantum information. Thus, while classical agents might find Strategy A more complex to execute than Strategy B, quantum agents can reach the opposite conclusion. We derive sufficient conditions for such contradictory conclusions and illustrate the phenomenon across multiple scenarios. As a byproduct, our results yield an information-theoretic lower bound on the minimal memory required by any agent - classical or quantum - to execute a given strategy.

arXiv:2508.08092•Aug 11, 2025•0 citations

Quantum reinforcement learning: the maze problem

Nicola Dalla Pozza, Lorenzo Buffoni, Stefano Martina +1 more

Abstract Quantum machine learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalising the classical concept of reinforcement learning to the quantum domain, i.e. quantum reinforcement learning (QRL). In particular, we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimisation, we consider a hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation noisy intermediate-scale quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.

arXiv:2108.04490•Quantum Machine Intelligence•May 27, 2022•35 citations
World models provide digital simulacra of the true world, allowing agents to be trained and tested before costly real-world deployment. At each time step, they receive an action and generate an observation and reward matching the statistics of the true world. In complex environments where present outcomes depend on events far in the past, this requires memory. One might expect that, by increasing memory, we can always build a model accurately enough to align the optimal agent policies of the real and virtual worlds. We show that this is false for classical world models, even when the true world itself is classical. We construct true worlds for which every finite classical model fails along the same possible trajectory: it either loses the ability to distinguish actions when the true world clearly prefers one, or repeatedly assigns the highest expected reward to suboptimal actions. Its expected-reward estimates also retain a nonvanishing average error. In contrast, each such true world admits a quantum world model using a single qutrit that reproduces it exactly: its reward estimates and preferred actions always match those of the true world, ensuring that the optimal policies of the real and virtual worlds remain perfectly aligned.
Central to the success of adaptive systems is their ability to interpret signals from their environment and respond accordingly-they act as agents interacting with their surroundings.Such agents typically perform better when able to execute increasingly complex strategies.This comes with a cost: the more information the agent must recall from its past experiences, the more memory it will need.Here we investigate the power of agents capable of quantum information processing.We uncover the most general form a quantum agent need adopt to maximize memory compression advantages and provide a systematic means of encoding their memory states.We show these encodings can exhibit extremely favorable scaling advantages relative to memoryminimal classical agents, particularly when information must be retained about events increasingly far into the past.
In modelling complex processes, the potential past data that influence future expectations are immense. Models that track all this data are not only computationally wasteful but also shed little light on what past data most influence the future. There is thus enormous interest in dimensional reduction-finding automated means to reduce the memory dimension of our models while minimizing its impact on its predictive accuracy. Here we construct dimensionally reduced quantum models by machine learning methods that can achieve greater accuracy than provably optimal classical counterparts. We demonstrate this advantage on present-day quantum computing hardware. Our algorithm works directly off classical time-series data and can thus be deployed in real-world settings. These techniques illustrate the immediate relevance of quantum technologies to time-series analysis and offer a rare instance where the resulting quantum advantage can be provably established.
In reinforcement learning, an agent interacts sequentially with an environment to maximize a reward, receiving only partial, probabilistic feedback. This creates a fundamental exploration-exploitation trade-off: the agent must explore to learn the hidden dynamics while exploiting this knowledge to maximize its target objective. While extensively studied classically, applying this framework to quantum systems requires dealing with hidden quantum states that evolve via unknown dynamics. We formalize this problem via a framework where the environment maintains a hidden quantum memory evolving via unknown quantum channels, and the agent intervenes sequentially using quantum instruments. For this setting, we adapt an optimistic maximum-likelihood estimation algorithm. We extend the analysis to continuous action spaces, allowing us to model general positive operator-valued measures (POVMs). By controlling the propagation of estimation errors through quantum channels and instruments, we prove that the cumulative regret of our strategy scales as O~(K)\widetilde{\mathcal{O}}(\sqrt{K})O(K​) over KKK episodes. Furthermore, via a reduction to the multi-armed quantum bandit problem, we establish information-theoretic lower bounds demonstrating that this sublinear scaling is strictly optimal up to polylogarithmic factors. As a physical application, we consider state-agnostic work extraction. When extracting free energy from a sequence of non-i.i.d. quantum states correlated by a hidden memory, any lack of knowledge about the source leads to thermodynamic dissipation. In our setting, the mathematical regret exactly quantifies this cumulative dissipation. Using our adaptive algorithm, the agent uses past energy outcomes to improve its extraction protocol on the fly, achieving sublinear cumulative dissipation, and, consequently, an asymptotically zero dissipation rate.
Reinforcement learning studies how an agent should interact with an environment to maximize its cumulative reward. A standard way to study this question abstractly is to ask how many samples an agent needs from the environment to learn an optimal policy for a γγγ-discounted Markov decision process (MDP). For such an MDP, we design quantum algorithms that approximate an optimal policy (π∗π^*π∗), the optimal value function (v∗v^*v∗), and the optimal QQQ-function (q∗q^*q∗), assuming the algorithms can access samples from the environment in quantum superposition. This assumption is justified whenever there exists a simulator for the environment; for example, if the environment is a video game or some other program. Our quantum algorithms, inspired by value iteration, achieve quadratic speedups over the best-possible classical sample complexities in the approximation accuracy (εεε) and two main parameters of the MDP: the effective time horizon (11−γ\frac{1}{1-γ}1−γ1​) and the size of the action space (AAA). Moreover, we show that our quantum algorithm for computing q∗q^*q∗ is optimal by proving a matching quantum lower bound.
Whether winning blackjack or navigating busy streets, achieving desired outcomes requires agents to execute adaptive strategies, strategies where actions depend contextually on past events. In complexity science, this motivates memory as an operational quantifier of complexity: given two strategies, the more complex one demands the agent to track more about the past. Here, we show that conclusions about complexity fundamentally depend on whether agents can process and store quantum information. Thus, while classical agents might find Strategy A more complex to execute than Strategy B, quantum agents can reach the opposite conclusion. We derive sufficient conditions for such contradictory conclusions and illustrate the phenomenon across multiple scenarios. As a byproduct, our results yield an information-theoretic lower bound on the minimal memory required by any agent - classical or quantum - to execute a given strategy.
Abstract Quantum machine learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalising the classical concept of reinforcement learning to the quantum domain, i.e. quantum reinforcement learning (QRL). In particular, we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimisation, we consider a hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation noisy intermediate-scale quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.