Abstract The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in quantum optical experiments appears to be within the reach of near-term technology.
Abstract The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in quantum optical experiments appears to be within the reach of near-term technology.
Reinforcement learning algorithms are a powerful tool to manage the interaction between a system and its environment. Here we present an approach to apply these algorithms within modern-day photonic technologies. Numerical tests, performed on typical learning tasks, show that the architecture is robust against experimental noise, which can even be beneficial for the learning process. The proposed architecture, based on single-photon evolution on a tree structure of tunable beamsplitters, is simple, easy to implement and an integration in quantum optics applications appears to be within the reach of near-term technology.
As the field of artificial intelligence is pushed forward, the question arises of how fast autonomous machines can learn. Within artificial intelligence, an important paradigm is reinforcement learning, where agents - learning entities capable of decision making - interact with the world they are placed in, called an environment. Thanks to these interactions, agents receive feedback from the environment and thus progressively adjust their behaviour to accomplish a given goal. An important question in reinforcement learning is how fast agents can learn to fulfill their tasks. To answer this question we consider a novel reinforcement learning framework where quantum mechanics is used. In particular, we quantize the agent and the environment and grant them the possibility to also interact quantum-mechanically, that is, by using a quantum channel for their communication. We demonstrate that this feature enables a speed-up in the agent's learning process, and we further show that combining this scenario with classical communication enables the evaluation of such an improvement. This learning protocol is implemented on an integrated re-programmable photonic platform interfaced with photons at telecommunication wavelengths. Thanks to the full tunability of the device, this platform proves the best candidate for the implementation of learning protocols, where a continuous update of the learning process is required.
The growing scale and complexity of photonic quantum systems demand a transition from manual, heuristic control toward intelligent, autonomous operational frameworks. Quantum Artificial Intelligence (QAI) integrates machine learning, quantum-enhanced optimization, and real-time feedback to address key challenges in the calibration, stabilization, and scaling of photonic quantum technologies. Unlike conventional machine-learning-assisted quantum control and self-driving laboratory approaches that rely primarily on classical intelligence, the proposed QAI paradigm establishes a tighter integration between artificial intelligence and quantum hardware. Parameterized quantum circuits executed directly on the photonic processor participate in the agent’s decision-making process, while the same framework supports both real-time control tasks. QAI combines variational quantum neural networks, meta-reinforcement learning, multi-agent control architectures, and quantum optimization methods, such as the Quantum Approximate Optimization Algorithm, to enable adaptive calibration, error mitigation, and high-dimensional optimization in programmable photonic platforms. While this perspective focuses on real-time control and calibration, we also outline a conceptual pathway toward autonomous generative inverse design. To guide future development, we propose a three-stage roadmap that progresses from open-source hybrid quantum–classical control tools to fully autonomous laboratories capable of closed-loop self-optimization and experiment design. Experimental platforms include programmable silicon photonic circuits and lithium-niobate interferometers, where thermal drift, fabrication variability, and crosstalk require intelligent adaptation. Beyond quantum optics, the QAI framework has broader implications for quantum sensing, secure communications, and photonic computing. By introducing a unified QAI framework and a structured classification of Quantum Autonomy Levels (QALs), this work provides a roadmap toward autonomous quantum-optics laboratories and photonic quantum technologies.
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.
Quantum Reinforcement Learning (QRL) is an emerging field with many novel approaches suitable for noisy intermediate-scale quantum (NISQ) devices being proposed recently. Projective Simulation (PS) is a classical machine learning method based on random walks in a Directed Acyclic Graph (DAG), and is mainly used to train agents in a Reinforcement Learning setup. This graph corresponds to the Episodic Compositional Memory (ECM) of the agent. For the quantum version of PS we use quantum walks of single photons in quantum optical circuits, which go through tunable components (beamsplitters and phase shifters). We call this specific model Quantum Optical Projective Simulation (QOPS). In this work, we propose a general framework to translate any 2-layer ECM to a quantum optical circuit and an algorithm for solving quantum strategic games using QOPS. Moreover, we introduce QOPS with an entangled state as input. The application that we are using to evaluate the performance of the framework is the Prisoner's Dilemma. We executed the implementation using Altair's noisy simulator, designed to emulate a single photon-based quantum processor by Quandela. The algorithm provided promising results even in the presence of noise, which makes it a strong candidate for runs on single-photon-based quantum processors.
Variational quantum algorithms show potential in effectively operating on noisy intermediate-scale quantum devices. A variational approach to reinforcement learning has been recently proposed, incorporating linear-optical interferometers and a classical learning model known as projective simulation (PS). PS is a decision-making tool for reinforcement learning and can be classically represented as a random walk on a graph that describes the agent's memory. In its optical quantum version, this approach utilizes quantum walks of single photons on a mesh of tunable beamsplitters and phase shifters to select actions. In this work, we present the implementation of this algorithm on Ascella, a single-photon-based quantum computer from Quandela. The focus is drawn on solving a test bed task to showcase the potential of the quantum agent with respect to the classical agent. Published by the American Physical Society 2024