Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2512.08023

F2: Offline Reinforcement Learning for Hamiltonian Simulation via Free-Fermionic Subroutine Compilation

Ethan Decker, Christopher Watson, Junyu Zhou, Yuhao Liu, Chenxu Liu, Ang Li, Gushu Li, Samuel Stein

quant-phcond-mat.mtrl-sciSubmitted Dec 8, 2025• Announced Dec 10, 2025• 0 citations
v1 · Dec 8, 2025v2 · Dec 10, 2025
arXivHTMLPDF

Abstract

Compiling shallow and accurate quantum circuits for Hamiltonian simulation remains challenging due to hardware constraints and the combinatorial complexity of minimizing gate count and circuit depth. Existing optimization method pipelines rely on hand-engineered classical heuristics, which cannot learn input-dependent structure and therefore miss substantial opportunities for circuit reduction. We introduce \textbf{F2}, an offline reinforcement learning framework that exploits free-fermionic structure to efficiently compile Trotter-based Hamiltonian simulation circuits. F2 provides (i) a reinforcement-learning environment over classically simulatable free-fermionic subroutines, (ii) architectural and objective-level inductive biases that stabilize long-horizon value learning, and (iii) a reversible synthetic-trajectory generation mechanism that consistently yields abundant, guaranteed-successful offline data. Across benchmarks spanning lattice models, protein fragments, and crystalline materials (12-222 qubits), F2 reduces gate count by 47\% and depth by 38\% on average relative to strong baselines (Qiskit, Cirq/OpenFermion) while maintaining average errors of $10^{-7}$. These results show that aligning deep reinforcement learning with the algebraic structure of quantum dynamics enables substantial improvements in circuit synthesis, suggesting a promising direction for scalable, learning-based quantum compilation

Related papers

Adaptive Quantum Circuit Compilation with Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.

•Sep 5, 2026•0 citations

Rethinking Quantum Circuit Optimization as a Reinforcement Learning Problem

Arjun Mehta, Sven Richter

Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.

American Journal Of Big Data•Apr 1, 2026•0 citations

Using Topology-Aware Reinforcement Learning to Synthesize Quantum Linear Reversible Circuits

Giovanni Acampora, Allegra Cuzzocrea, Marco Lapegna +2 more

In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.

International Joint Conference on Neural Networks (IJCNN)•Jun 30, 2025•0 citations

Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis

Lukas Theißinger, Thore Gerlach, David Berghaus +1 more

Quantum unitary synthesis addresses the problem of translating abstract quantum algorithms into sequences of hardware-executable quantum gates. Solving this task exactly is infeasible in general due to the exponential growth of the underlying combinatorial search space. Existing approaches suffer from misaligned optimization objectives, substantial training costs and limited generalization across different qubit counts. We mitigate these limitations by using supervised learning to approximate the minimum description length of residual unitaries and combining this estimate with stochastic beam search to identify near optimal gate sequences. Our method relies on a lightweight model with zero-shot generalization, substantially reducing training overhead compared to prior baselines. Across multiple benchmarks, we achieve faster wall-clock synthesis times while exceeding state-of-the-art methods in terms of success rate for complex circuits.

quant-pharXiv:2602.15146•Feb 18, 2026•0 citations

Structure-Aware Quantum Circuit Partitioning via Reinforcement Learning for Efficient Re-Synthesis

Mohammad Walid Charrwi, C. Rasmussen, Ed Younis +2 more

The advancement of quantum computing into the utility scale requires compilation frameworks that can effectively manage the discrepancy between high-level algorithmic intent and the low-level physical constraints of contemporary hardware. Quantum circuit partitioning is a pivotal stage in this compilation pipeline, particularly when leveraging high-performance synthesis tools that are computationally bounded by the number of qubits. Existing partitioning approaches, such as ScanPartitioner [19] and QuickPartitioner [21], while effective, do not leverage structural patterns in circuits, limiting their ability to make globally informed local partitioning decisions. To address this gap, we propose a novel structural-aware quantum circuit partitioning method using a reinforcement learning (RL) framework that harnesses global circuit knowledge to guide local partitioning decisions, enabling more optimization opportunities at the sub-circuit level. Experimental results on benchmark circuits transpiled to satisfy IBM quantum hardware constraints show that our approach reduces the native two-qubit gate count compared to existing quantum circuit partitioners (ScanPartitioner and QuickPartitioner) with an average two-qubit gate reduction of 18.55% over baselines. This research establishes a scalable and robust methodology for partitioning quantum circuits, bridging the gap between exact and approximate synthesis in the Noisy Intermediate-Scale Quantum (NISQ) era and beyond.

ACM Great Lakes Symposium on VLSI (GLSVLSI)•Jun 18, 2026•0 citations

Adaptive Quantum Circuit Compilation via Reinforcement Learning

Jincheng Zhang

Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.

•Aug 28, 2026•0 citations
Compiling shallow and accurate quantum circuits for Hamiltonian simulation remains challenging due to hardware constraints and the combinatorial complexity of minimizing gate count and circuit depth. Existing optimization method pipelines rely on hand-engineered classical heuristics, which cannot learn input-dependent structure and therefore miss substantial opportunities for circuit reduction.
We introduce \textbf{F2}, an offline reinforcement learning framework that exploits free-fermionic structure to efficiently compile Trotter-based Hamiltonian simulation circuits. F2 provides (i) a reinforcement-learning environment over classically simulatable free-fermionic subroutines, (ii) architectural and objective-level inductive biases that stabilize long-horizon value learning, and (iii) a reversible synthetic-trajectory generation mechanism that consistently yields abundant, guaranteed-successful offline data.
Across benchmarks spanning lattice models, protein fragments, and crystalline materials (12-222 qubits), F2 reduces gate count by 47% and depth by 38% on average relative to strong baselines (Qiskit, Cirq/OpenFermion) while maintaining average errors of 10−710^{-7}10−7. These results show that aligning deep reinforcement learning with the algebraic structure of quantum dynamics enables substantial improvements in circuit synthesis, suggesting a promising direction for scalable, learning-based quantum compilation
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.
In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.
Quantum unitary synthesis addresses the problem of translating abstract quantum algorithms into sequences of hardware-executable quantum gates. Solving this task exactly is infeasible in general due to the exponential growth of the underlying combinatorial search space. Existing approaches suffer from misaligned optimization objectives, substantial training costs and limited generalization across different qubit counts. We mitigate these limitations by using supervised learning to approximate the minimum description length of residual unitaries and combining this estimate with stochastic beam search to identify near optimal gate sequences. Our method relies on a lightweight model with zero-shot generalization, substantially reducing training overhead compared to prior baselines. Across multiple benchmarks, we achieve faster wall-clock synthesis times while exceeding state-of-the-art methods in terms of success rate for complex circuits.
The advancement of quantum computing into the utility scale requires compilation frameworks that can effectively manage the discrepancy between high-level algorithmic intent and the low-level physical constraints of contemporary hardware. Quantum circuit partitioning is a pivotal stage in this compilation pipeline, particularly when leveraging high-performance synthesis tools that are computationally bounded by the number of qubits. Existing partitioning approaches, such as ScanPartitioner [19] and QuickPartitioner [21], while effective, do not leverage structural patterns in circuits, limiting their ability to make globally informed local partitioning decisions. To address this gap, we propose a novel structural-aware quantum circuit partitioning method using a reinforcement learning (RL) framework that harnesses global circuit knowledge to guide local partitioning decisions, enabling more optimization opportunities at the sub-circuit level. Experimental results on benchmark circuits transpiled to satisfy IBM quantum hardware constraints show that our approach reduces the native two-qubit gate count compared to existing quantum circuit partitioners (ScanPartitioner and QuickPartitioner) with an average two-qubit gate reduction of 18.55% over baselines. This research establishes a scalable and robust methodology for partitioning quantum circuits, bridging the gap between exact and approximate synthesis in the Noisy Intermediate-Scale Quantum (NISQ) era and beyond.
Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.