We study a recommender system for quantum data using the linear contextual bandit framework. In each round, a learner receives an observable (the context) and has to recommend from a finite set of unknown quantum states (the actions) which one to measure. The learner has the goal of maximizing the reward in each round, that is the outcome of the measurement on the unknown state. Using this model, we formulate the low energy quantum state recommendation problem where the context is a Hamiltonian and the goal is to recommend the state with the lowest energy. For this task, we study two families of contexts: the Ising model and a generalized cluster model. We observe that if we interpret the actions as different phases of the models, then the recommendation is done by classifying the correct phase of the given Hamiltonian, and the strategy can be interpreted as an online quantum phase classifier.
We study a recommender system for quantum data using the linear contextual bandit framework. In each round, a learner receives an observable (the context) and has to recommend from a finite set of unknown quantum states (the actions) which one to measure. The learner has the goal of maximizing the reward in each round, that is the outcome of the measurement on the unknown state. Using this model, we formulate the low energy quantum state recommendation problem where the context is a Hamiltonian and the goal is to recommend the state with the lowest energy. For this task, we study two families of contexts: the Ising model and a generalized cluster model. We observe that if we interpret the actions as different phases of the models, then the recommendation is done by classifying the correct phase of the given Hamiltonian, and the strategy can be interpreted as an online quantum phase classifier.
We initiate the study of tradeoffs between exploration and exploitation in online learning of properties of quantum states. Given sequential oracle access to an unknown quantum state, in each round, we are tasked to choose an observable from a set of actions aiming to maximize its expectation value on the state (the reward). Information gained about the unknown state from previous rounds can be used to gradually improve the choice of action, thus reducing the gap between the reward and the maximal reward attainable with the given action set (the regret). We provide various information-theoretic lower bounds on the cumulative regret that an optimal learner must incur, and show that it scales at least as the square root of the number of rounds played. We also investigate the dependence of the cumulative regret on the number of available actions and the dimension of the underlying space. Moreover, we exhibit strategies that are optimal for bandits with a finite number of arms and general mixed states.
Supervised quantum learning methods face notable limitations in dynamic, real-world environments due to their reliance on static labels and limited adaptability. To address these challenges, we propose a novel online learning framework-Quantum Contextual Bandit (QCB)-that integrates quantum neural networks (QNNs) with contextual bandit (CB) algorithms. The QCB framework enables adaptive decision-making by incorporating bandit-based exploration into QNN training, making it particularly suitable for applications such as recommender systems and resource allocation in next-generation 5G/6G networks. To mitigate the adverse effects of quantum noise-including depolarizing, Pauli, and shot noise, the framework leverages a gradient-free optimization approach, enhancing robustness and convergence stability. Experimental results on various datasets demonstrate that QCB consistently outperforms traditional QNN training methods with identical circuit architectures. Notably, the model achieves over 99 % accuracy under ideal conditions and sustains high performance under noisy quantum environments. These results underscore the potential of QCB as a scalable, noise-resilient solution for adaptive learning in quantum machine learning systems.
We study quantum multi-armed bandits (QMAB) and quantum linear bandits (QLB), where the learner queries each arm or action through a quantum reward oracle or its inverse. Prior work gives algorithms over horizon T with regret O(KlogT) for QMAB with K arms and O(d2polylogT) for d-dimensional QLB. This leaves open the optimal dependence on K and T and whether the dependence on d can be further improved. In this work, we prove the first tight minimax regret bound of Θ(Klog(1+T/K)) for QMAB and the first lower bound of Ω(dlog(1+T/d)) for finite-action QLB, ruling out regret independent of T. Our lower bounds rely on a high-confidence single-arm quantum testing lower bound for distinguishing a fixed reward mean from an interval of alternatives. A bandit-to-testing reduction then lifts it to the QMAB lower bound, while a linear embedding gives the finite-action QLB lower bound. The matching QMAB upper bound is obtained using a tail bound for the Quantum Monte Carlo (QMC) estimator. For finite-action QLB, we propose a phased elimination algorithm that combines a low-bias low-variance quantum mean estimator with a small-support G-optimal design through a query allocation matched to the design weights. When the action set has size poly(d), its regret is nearly linear in d and matches our lower bound up to polylogarithmic factors.
Applying quantum computing techniques to machine learning has attracted widespread attention recently and quantum machine learning has become a hot research topic. There are three major categories of machine learning: supervised, unsupervised, and reinforcement learning (RL). However, quantum RL has made the least progress when compared to the other two areas. In this study, we implement the well-known RL algorithm Q learning with a quantum neural network and evaluate it in the grid world environment. RL is learning through interactions with the environment, with the aim of discovering a strategy to maximize the expected cumulative rewards. Problems in RL bring in unique challenges to the study with their sequential nature of learning, potentially long delayed reward signals, and large or infinite size of state and action spaces. This study extends our previous work on solving the contextual bandit problem using a quantum neural network, where the reward signals are immediate after each action.
In reinforcement learning, an agent interacts sequentially with an environment to maximize a reward, receiving only partial, probabilistic feedback. This creates a fundamental exploration-exploitation trade-off: the agent must explore to learn the hidden dynamics while exploiting this knowledge to maximize its target objective. While extensively studied classically, applying this framework to quantum systems requires dealing with hidden quantum states that evolve via unknown dynamics. We formalize this problem via a framework where the environment maintains a hidden quantum memory evolving via unknown quantum channels, and the agent intervenes sequentially using quantum instruments. For this setting, we adapt an optimistic maximum-likelihood estimation algorithm. We extend the analysis to continuous action spaces, allowing us to model general positive operator-valued measures (POVMs). By controlling the propagation of estimation errors through quantum channels and instruments, we prove that the cumulative regret of our strategy scales as O(K) over K episodes. Furthermore, via a reduction to the multi-armed quantum bandit problem, we establish information-theoretic lower bounds demonstrating that this sublinear scaling is strictly optimal up to polylogarithmic factors. As a physical application, we consider state-agnostic work extraction. When extracting free energy from a sequence of non-i.i.d. quantum states correlated by a hidden memory, any lack of knowledge about the source leads to thermodynamic dissipation. In our setting, the mathematical regret exactly quantifies this cumulative dissipation. Using our adaptive algorithm, the agent uses past energy outcomes to improve its extraction protocol on the fly, achieving sublinear cumulative dissipation, and, consequently, an asymptotically zero dissipation rate.
AI and collective intelligence systems universally suffer from a deficiency of context. There are innumerable possible contexts that may possibly change the interpretation of some signal, that may change the proper response to some stimulus. For example, an image understanding system that does not recognize an arrest event in a zoomed image of a person's face. How is it possible to know there is more information, outside of what the system can access, that affects the interpretation of data The solution to the context problem in practice today is a pragmatic, engineering one: analyze errors (in recommendations, question answers, image recognition, etc.), classify the kinds of contextual information that caused the wrong behavior, find the most common type of context that causes errors, and add information about that kind of context to the system. Clearly this approach is neither general nor scalable, and ignores the infamous long tail of possible contextual information that may affect a system's understanding and its behavior. In this paper we outline a new, more general, approach to recognizing context. The approach is grounded in a fairly simple intuition: the mathematics underlying quantum mechanics is far more appropriate for modeling, and therefore simulating, human cognitive behavior than the standard toolset from classical statistics. Notions such as Heisenberg's uncertainty principle, superpositions of states, and entanglement have direct and measurable analogs in collective intelligence.