Systems and Control Transactions

Process Flowsheet Synthesis via Quantum Reinforcement Learning with Improved Scalability

Austin Braniff, Fengqi You, Yuhe Tian

Published Jun 19, 2026• 1 citation

Abstract

In this work, we present quantum reinforcement learning algorithms for process flowsheet synthesis. Particularly, we discuss the implementation of encoding strategies to improve the algorithmic scalability. Reinforcement learning (RL)-driven flowsheet synthesis techniques provide a promising approach for conceptual process design, in addition to traditional optimization-based methods. These RL-based strategies identify the optimal flowsheet configurations from a maximum set of available processing units, without requiring to pre-postulate an interconnected superstructure. However, the resulting combinatorial design space for RL can scale extensively with the increased number of available processing units, which can render the algorithms to be computationally intensive or even intractable. To address this challenge, our prior work has introduced a quantum-enhanced approach to RL-driven process synthesis. However, this algorithm was limited in its capacity to solve larger flowsheeting problems as a large number of qubits were needed. To reduce qubit requirements and improve scalability, this work presents two different encoding strategies to embed information into the gates of quantum circuits instead of embedding it into qubits alone. We demonstrate the efficacy of these algorithms on three different scenarios of a flowsheet synthesis problem and interpret the results obtained from the IonQ quantum computing platform.

DOI: 10.69997/sct.187362

Related papers

Enhanced Reinforcement Learning-based Process Synthesis via Quantum Computing

Austin Braniff, Fengqi You, Yuhe Tian

In this work, we present quantum reinforcement learning (RL) as a solution strategy for process synthesis problems. Building on our prior work, we develop a generalized framework that formally poses process synthesis as a Markov decision process and introduces quantum-enhanced RL algorithms to solve it with improved scalability. Earlier implementations of quantum-based RL for process synthesis were limited by qubit requirements, which scaled poorly with problem complexity. This work overcomes this challenge by introducing state encoding algorithms to decouple qubit requirements from problem size. A classical RL-based solution strategy is used as a baseline to benchmark the quantum algorithms under identical training conditions. All algorithms are evaluated across a flowsheet synthesis problem of increasing unit counts to analyze their performance and scalability. Results show that all approaches are capable of identifying the optimal flowsheet designs in small design spaces. For moderate-scale unit counts, quantum approaches demonstrate competitive performance on a per-episode basis and improved efficiency on a per-parameter basis versus the classical RL benchmark. This work provides a foundation for future quantum computing applications within process systems engineering, establishes a controlled benchmark for comparing classical and quantum algorithms, and shows that the proposed quantum variants remain competitive for the process synthesis problem examined in this work.

quant-pharXiv:2605.21213Chinese Journal of Chemical EngineeringMay 21, 20260 citations

Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis

Lukas Theißinger, Thore Gerlach, David Berghaus +1 more

Quantum unitary synthesis addresses the problem of translating abstract quantum algorithms into sequences of hardware-executable quantum gates. Solving this task exactly is infeasible in general due to the exponential growth of the underlying combinatorial search space. Existing approaches suffer from misaligned optimization objectives, substantial training costs and limited generalization across different qubit counts. We mitigate these limitations by using supervised learning to approximate the minimum description length of residual unitaries and combining this estimate with stochastic beam search to identify near optimal gate sequences. Our method relies on a lightweight model with zero-shot generalization, substantially reducing training overhead compared to prior baselines. Across multiple benchmarks, we achieve faster wall-clock synthesis times while exceeding state-of-the-art methods in terms of success rate for complex circuits.

quant-pharXiv:2602.15146Feb 18, 20260 citations

Enhanced Reinforcement Learning-driven Process Design via Quantum Machine Learning

Austin Braniff, Fengqi You, Yuhe Tian

In this work, we introduce a quantum-enhanced reinforcement learning (RL) framework for process design synthesis. RL-driven methods for generating process designs have gained momentum due to their ability to intelligently identify optimal configurations without requiring pre-defined superstructures or flowsheet configurations. This eliminates reliance on prior expert knowledge, offering a comprehensive and robust design strategy. However, navigating the vast combinatorial design space poses computational challenges. To address this, a novel approach integrating RL with quantum machine learning (QML) is proposed. QML leverages theoretical advantages over classical methods to accelerate searches in large spaces. Built upon our prior work, the approach begins with a maximum set of available unit operations, represented in a flowsheet structure using an input-output stream matrix as RL observations. A Deep Q-Network (DQN) algorithm trains a parameterized quantum circuit (PQC) in place of a classical neural network (NN). The design structures generated by the RL agent are optimized using the IDAES Process Systems Engineering Framework, with the optimization objectives used as rewards to RL (e.g., cost, productivity). This quantum-enhanced algorithm is performed on a hydrodealkylation process case study, showcasing its efficiency and improved performance in navigating complex design spaces.

Systems and Control TransactionsJun 30, 20252 citations

Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware

Akash Kundu, Leopoldo Sarra

Quantum computing offers exciting opportunities for simulating complex quantum systems and optimizing large-scale combinatorial problems, but its practical use is limited by device noise and constrained connectivity. Designing quantum circuits, which are fundamental to quantum algorithms, is therefore a central challenge in current quantum hardware. Existing reinforcement learning-based methods for circuit design lose accuracy when restricted to hardware-native gates and device-level compilation. Here, we introduce gadget reinforcement learning (GRL) that combines learning with program synthesis to automatically construct composite gates that expand the action space while respecting hardware constraints. We show that this approach improves accuracy, hardware compatibility, and scalability for transverse-field Ising and quantum chemistry problems, reaching systems of up to ten qubits within realistic computational budgets. This framework demonstrates how learned, reusable circuit building blocks can guide the co-design of algorithms and hardware for quantum processors. Designing quantum circuits is challenging due to the exponential growth of the state space. The authors introduce gadget reinforcement learning, which combines reinforcement learning with program synthesis to enhance quantum circuit exploration, demonstrating improved accuracy and scalability in complex tasks, thus bridging algorithmic design and practical implementation for hardware-specific optimizations.

arXiv:2411.00230Communications PhysicsJan 3, 20264 citations

TATVA: A Reinforcement Learning Framework for Quantum Circuit Synthesis

Om Bhamare, Aryan Jadhav, Manas Shinde +1 more

One of the most important initial steps in quantum computing is high-fidelity quantum state preparation, because errors in it can affect the final computational result. Therefore, one of the major challenges is to design a quantum circuit that can achieve the desired target state accurately with a smaller number of gates. Traditionally, circuits are designed using predefined rules and mathematical decompositions, but this makes it difficult to design circuits for different and complex target states. This paper presents TATVA, a reinforcement learning (RL) system for synthesizing quantum circuits. TATVA views circuit synthesis as a sequential decision problem in which the agents select one gate at a time and calculate the fidelity after each applied gate, with the gate producing the highest fidelity selected for the circuit. The system introduces a parallel architecture that uses two RL agents, Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), along with Qiskit's statevector simulator. The system can synthesize up to 5 qubits while achieving a fidelity of 0.999999 and producing compact circuits. After achieving the desired fidelity, it performs a post-hoc optimization step that reduces the circuit depth while preserving the achieved fidelity. The circuit is finalized using fidelity, success rate, gate count, and circuit depth. Target states that are not used during training are used to test whether the system has learned effectively and can generalize beyond the training states. This approach aims to enable automatic quantum circuit synthesis with high fidelity using reinforcement learning.

quant-pharXiv:2610.00945Oct 2, 20260 citations

Generalizing Reinforcement Learning-Based Quantum Circuit Synthesis across Multiple Topologies

Giovanni Acampora, Allegra Cuzzocrea, Marco Lapegna +2 more

The optimization of the limited resources of current quantum devices hinges on the implementation of quantum-compiling techniques. The process of transforming high-level quantum algorithms into executable quantum circuits has a significant impact on the performance and fidelity of computations. One of the most critical aspects of the compiling stack is the synthesis of quantum circuits, which aims to minimize the depth and number of gates, as well as the error rate. Nevertheless, the optimal synthesis algorithms are extremely computationally expensive. This is why synthesis algorithms based on reinforcement learning (RL) have recently been introduced. However, such algorithms are constrained by the topology of quantum processors, necessitating retraining of the RL agent for different topologies. To achieve a balance between adaptability, optimality, and computational cost in RL-based quantum circuit synthesis techniques, this work proposes an RL-based quantum circuit synthesis algorithm of Clifford operators that addresses different topologies simultaneously. Experimental tests demonstrate that our synthesizer outperforms state-of-the-art greedy techniques while being significantly faster than boolean satisfiability solvers (SAT).

Mar 17, 20253 citations