arXiv:2204.09052

Optimizing Tensor Network Contraction Using Reinforcement Learning

Eli A. Meirom, Haggai Maron, Shie Mannor, Gal Chechik

Published Apr 18, 2022• 5 citations

Abstract

Quantum Computing (QC) stands to revolutionize computing, but is currently still limited. To develop and test quantum algorithms today, quantum circuits are often simulated on classical computers. Simulating a complex quantum circuit requires computing the contraction of a large network of tensors. The order (path) of contraction can have a drastic effect on the computing cost, but finding an efficient order is a challenging combinatorial optimization problem. We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges and obtain significant improvements over state-of-the-art techniques in three varieties of circuits, including the largest scale networks used in contemporary QC.

DOI: 10.48550/arxiv.2204.09052

Related papers

Reinforcement Learning for Efficient Quantum Circuit Equivalence Checking via Tensor Network Contraction

Suhaib Al-Rousan, Kim Guldstrand Larsen, Christian Schilling +1 more

Verifying whether two quantum circuits realize the same unitary is vital for optimization and compilation. However, approaches based on tensor network contraction often suffer from exponential blow-ups, not least since finding an optimal contraction order is NP-hard. We address this by employing tensor decision diagrams and formulating contraction as a sequential decision-making task. A reinforcement learning agent, trained via proximal policy optimization with a graph neural network policy, learns contraction orders to minimize both floating-point operations and peak memory. Our evaluation shows that the learned policy consistently outperforms heuristic baselines in running time while maintaining competitive memory usage, highlighting the potential of AI for scalable quantum circuit verification.

Jan 1, 20250 citations

Graph Neural Network Autoencoders for Efficient Quantum Circuit Optimisation

Ioana Moflic, Vikas Garg, Alexandru Paler

Reinforcement learning (RL) is a promising method for quantum circuit optimisation. However, the state space that has to be explored by an RL agent is extremely large when considering all the possibilities in which a quantum circuit can be transformed through local rewrite operations. This state space explosion slows down the learning of RL-based optimisation strategies. We present for the first time how to use graph neural network (GNN) autoencoders for the optimisation of quantum circuits. We construct directed acyclic graphs from the quantum circuits, encode the graphs and use the encodings to represent RL states. We illustrate our proof of concept implementation on Bernstein-Vazirani circuits and, from preliminary results, we conclude that our autoencoder approach: a) maintains the optimality of the original RL method; b) reduces by 20 \% the size of the table that encodes the learned optimisation strategy. Our method is the first realistic first step towards very large scale RL quantum circuit optimisation.

arXiv:2303.03280Mar 6, 20232 citations

Reinforcement Learning Based Quantum Circuit Optimization via ZX-Calculus

Jordi Riu, Jan Nogué, Gerard Vilaplana +2 more

We propose a novel Reinforcement Learning (RL) method for optimizing quantum circuits using graph-theoretic simplification rules of ZX-diagrams. The agent, trained using the Proximal Policy Optimization (PPO) algorithm, employs Graph Neural Networks to approximate the policy and value functions. We demonstrate the capacity of our approach by comparing it against the best performing ZX-Calculus-based algorithm for the problem in hand. After training on small Clifford+T circuits of 5-qubits and few tenths of gates, the agent consistently improves the state-of-the-art for this type of circuits, for at least up to 80-qubit and 2100 gates, whilst remaining competitive in terms of computational performance. Additionally, we illustrate the versatility of the agent by incorporating additional optimization routines on the workflow during training, improving the two-qubit gate count state-of-the-art on multiple structured quantum circuits for relevant applications of much larger dimension and different gate distributions than the circuits the agent trains on. This conveys the potential of tailoring the reward function to the specific characteristics of each application and hardware backend. Our approach is a valuable tool for the implementation of quantum algorithms in the near-term intermediate-scale range (NISQ).

arXiv:2312.11597QuantumMay 28, 202515 citations

Massive Parallel Simulation for Tensor Network Contraction Ordering Solving

Wei Xu, Weiqin Tong, Xiao-Yang Liu

Tensor network contraction is a powerful mathematical tool for dealing with complex systems and high-dimensional data, which is widely used in classical simulations of molecules and quantum circuits. To improve the performance of simulations, existing methods are implemented by searching the optimal contraction orderings for tensor networks. In particular, some reinforcement learning (RL) methods show a greater potential for solving. However, with the increasing scale of simulation, the number of tensors in the corresponding tensor network is huge, the existing RL methods suffer from sampling bottlenecks and slow solving speed, which make it difficult to efficiently solve for high-quality contraction orderings. In this paper, we design a massive parallel simulation environment for tensor network contraction ordering solving on a single GPU. We experimented with multiple numbers of parallel environments on the tensor-train network, and achieved the highest speedup when the number of environments was 26. We also have experimented on large-scale tensor-train networks and tree tensor networks, and compared to Kahypar and RL-Baseline, our method achieves average speedup of 8.96× and 7.48× on tensor-train networks, and 10.03× and 7.48× on tree tensor networks.

International Conference on Neural Networks, Information and Communication Engineering (NNICE)Jan 19, 20240 citations

Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networks

Alexander Mattick, Maniraman Periyasamy, Christian Ufrecht +4 more

Quantum computing is currently strongly limited by the impact of noise, in particular introduced by the application of two-qubit gates. For this reason, reducing the number of two-qubit gates is of paramount importance on noisy intermediate-scale quantum hardware. To advance towards more reliable quantum computing, we introduce a framework based on ZX calculus, graph-neural networks and reinforcement learning for quantum circuit optimization. By combining reinforcement learning and tree search, our method addresses the challenge of selecting optimal sequences of ZX calculus rewrite rules. Instead of relying on existing heuristic rules for minimizing circuits, our method trains a novel reinforcement learning policy that directly operates on ZX-graphs, therefore allowing us to search through the space of all possible circuit transformations to find a circuit significantly minimizing the number of CNOT gates. This way we can scale beyond hard-coded rules towards discovering arbitrary optimization rules. We demonstrate our method's competetiveness with state-of-the-art circuit optimizers and generalization capabilities on large sets of diverse random circuits.

arXiv:2504.03429Apr 4, 20250 citations

Adaptive Quantum Circuit Compilation with Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.

Sep 5, 20260 citations