In this work, we introduce a quantum-enhanced reinforcement learning (RL) framework for process design synthesis. RL-driven methods for generating process designs have gained momentum due to their ability to intelligently identify optimal configurations without requiring pre-defined superstructures or flowsheet configurations. This eliminates reliance on prior expert knowledge, offering a comprehensive and robust design strategy. However, navigating the vast combinatorial design space poses computational challenges. To address this, a novel approach integrating RL with quantum machine learning (QML) is proposed. QML leverages theoretical advantages over classical methods to accelerate searches in large spaces. Built upon our prior work, the approach begins with a maximum set of available unit operations, represented in a flowsheet structure using an input-output stream matrix as RL observations. A Deep Q-Network (DQN) algorithm trains a parameterized quantum circuit (PQC) in place of a classical neural network (NN). The design structures generated by the RL agent are optimized using the IDAES Process Systems Engineering Framework, with the optimization objectives used as rewards to RL (e.g., cost, productivity). This quantum-enhanced algorithm is performed on a hydrodealkylation process case study, showcasing its efficiency and improved performance in navigating complex design spaces.
In this work, we introduce a quantum-enhanced reinforcement learning (RL) framework for process design synthesis. RL-driven methods for generating process designs have gained momentum due to their ability to intelligently identify optimal configurations without requiring pre-defined superstructures or flowsheet configurations. This eliminates reliance on prior expert knowledge, offering a comprehensive and robust design strategy. However, navigating the vast combinatorial design space poses computational challenges. To address this, a novel approach integrating RL with quantum machine learning (QML) is proposed. QML leverages theoretical advantages over classical methods to accelerate searches in large spaces. Built upon our prior work, the approach begins with a maximum set of available unit operations, represented in a flowsheet structure using an input-output stream matrix as RL observations. A Deep Q-Network (DQN) algorithm trains a parameterized quantum circuit (PQC) in place of a classical neural network (NN). The design structures generated by the RL agent are optimized using the IDAES Process Systems Engineering Framework, with the optimization objectives used as rewards to RL (e.g., cost, productivity). This quantum-enhanced algorithm is performed on a hydrodealkylation process case study, showcasing its efficiency and improved performance in navigating complex design spaces.
In this work, we present quantum reinforcement learning (RL) as a solution strategy for process synthesis problems. Building on our prior work, we develop a generalized framework that formally poses process synthesis as a Markov decision process and introduces quantum-enhanced RL algorithms to solve it with improved scalability. Earlier implementations of quantum-based RL for process synthesis were limited by qubit requirements, which scaled poorly with problem complexity. This work overcomes this challenge by introducing state encoding algorithms to decouple qubit requirements from problem size. A classical RL-based solution strategy is used as a baseline to benchmark the quantum algorithms under identical training conditions. All algorithms are evaluated across a flowsheet synthesis problem of increasing unit counts to analyze their performance and scalability. Results show that all approaches are capable of identifying the optimal flowsheet designs in small design spaces. For moderate-scale unit counts, quantum approaches demonstrate competitive performance on a per-episode basis and improved efficiency on a per-parameter basis versus the classical RL benchmark. This work provides a foundation for future quantum computing applications within process systems engineering, establishes a controlled benchmark for comparing classical and quantum algorithms, and shows that the proposed quantum variants remain competitive for the process synthesis problem examined in this work.
In this work, we present quantum reinforcement learning algorithms for process flowsheet synthesis. Particularly, we discuss the implementation of encoding strategies to improve the algorithmic scalability. Reinforcement learning (RL)-driven flowsheet synthesis techniques provide a promising approach for conceptual process design, in addition to traditional optimization-based methods. These RL-based strategies identify the optimal flowsheet configurations from a maximum set of available processing units, without requiring to pre-postulate an interconnected superstructure. However, the resulting combinatorial design space for RL can scale extensively with the increased number of available processing units, which can render the algorithms to be computationally intensive or even intractable. To address this challenge, our prior work has introduced a quantum-enhanced approach to RL-driven process synthesis. However, this algorithm was limited in its capacity to solve larger flowsheeting problems as a large number of qubits were needed. To reduce qubit requirements and improve scalability, this work presents two different encoding strategies to embed information into the gates of quantum circuits instead of embedding it into qubits alone. We demonstrate the efficacy of these algorithms on three different scenarios of a flowsheet synthesis problem and interpret the results obtained from the IonQ quantum computing platform.
Quantum Machine Learning (QML) represents a paradigm shift in machine learning, leveraging quantum mechanical principles to accelerate computation and potentially unlock new algorithms. This paper investigates a novel reinforcement learning (RL) framework, termed "Adaptive Neural Network Reinforcement Learning" (ANNR), which employs dynamic parameter adjustment within a self-adaptive neural network to enhance learning efficiency and generalization capabilities. We demonstrate that this approach overcomes limitations of traditional RL methods, offering a promising path towards more robust and adaptable reinforcement learning agents. The core mechanism focuses on iteratively refining neural network weights through a feedback loop, enabling the system to explore complex environments effectively. The research explores the impact of parameter adjustment on both training speed and final performance across various RL tasks.
Abstract This work introduces a novel hybrid quantum-classical machine learning framework in which a Reinforcement Learning (RL) agent autonomously constructs quantum circuits to optimize kernels for a Quantum Support Vector Classifier (QSVC). Unlike traditional approaches that rely on fixed or manually designed quantum feature maps, the proposed method employs a Deep Q-Network (DQN) agent to dynamically select quantum gates and their parameters. These configurations define a parameterized quantum circuit whose fidelity-based kernel is used to perform classification tasks. The RL agent is trained to maximize classification accuracy by interacting with an environment that evaluates circuit performance. Experiments conducted on a realistic and potentially biased educational dataset demonstrate that the RL-optimized QSVC outperforms both classical SVC and standard QSVC baselines in terms of accuracy and fairness. Furthermore, SHAP-based explainability analysis reveals a reduction in model bias, suggesting improved interpretability and robustness. This study highlights the potential of reinforcement learning for automated quantum circuit design and kernel optimization in quantum machine learning.
The convergence of quantum computing and artificial intelligence presents profound opportunities but faces significant hurdles, particularly in the Noisy Intermediate-Scale Quantum (NISQ) era. Quantum Machine Learning (QML) algorithms, while promising, exhibit sensitivity to noise and scalability challenges, hindering the demonstration of practical quantum advantage. Concurrently, Quantum Error Correction (QEC), essential for fault tolerance, imposes substantial resource overheads and is often developed generically, without specific adaptation to the target application's error sensitivity. This paper reviews the current state of the intersection of AI and quantum computing, examining both QML paradigms, e.g., Variational Quantum Algorithms, Quantum Kernels, and the burgeoning use of AI to enhance quantum computing itself, e.g., quantum control, QEC decoding, circuit design. A critical gap identified is the lack of frameworks that systematically co-design QML algorithms and QEC protocols adaptively. To address this, a novel framework is proposed based on Reinforcement Learning (RL). This framework employs an RL agent to dynamically adjust both the QML circuit architecture, e.g., VQC ansatz, and QEC parameters, e.g., decoding strategy, measurement frequency, based on observed application performance and estimated error characteristics. This adaptive co-design loop aims to optimize the trade-off between QML performance and QEC overhead, enhancing noise resilience and resource efficiency. The potential advantages, feasibility, and limitations of this approach are discussed, alongside with future research directions aimed at realizing robust and practical Quantum AI.
This paper investigates the optimization of quantum control algorithms using reinforcement learning (RL). The core claim is that RL can significantly enhance the control precision and efficiency of quantum devices. We propose a framework where the quantum control algorithm is treated as an intelligent agent, interacting with the quantum environment to learn optimal control strategies. This approach allows for adaptive control, addressing the challenges inherent in designing fixed control protocols for complex quantum systems. The methodology involves defining a reward function that reflects the desired quantum state evolution, using RL algorithms to train the agent, and validating the learned control strategy through simulations. We demonstrate the potential of this approach to improve control accuracy and robustness, particularly in scenarios with noisy environments or rapidly changing system parameters. The results suggest a promising direction for leveraging machine learning techniques to unlock the full potential of quantum control.