Abstract Quantum compiling aims to construct a quantum circuitVby quantum gates drawn from a native gate alphabet, which is functionally equivalent to the target unitaryU. It is a crucial stage for the running of quantum algorithms on noisy intermediate-scale quantum (NISQ) devices. However, the space for structure exploration of quantum circuit is enormous, resulting in the requirement of human expertise, hundreds of experimentations or modifications from existing quantum circuits. In this paper, we propose a variational quantum compiling (VQC) algorithm based on reinforcement learning, in order to automatically design the structure of quantum circuit for VQC with no human intervention. An agent is trained to sequentially select quantum gates from the native gate alphabet and the qubits they act on by doubleQ-learning withϵ-greedy exploration strategy and experience replay. At first, the agent randomly explores a number of quantum circuits with different structures, and then iteratively discovers structures with higher performance on the learning task. Simulation results show that the proposed method can make exact compilations with less quantum gates compared to previous VQC algorithms. It can reduce the errors of quantum algorithms due to decoherence process and gate noise in NISQ devices, and enable quantum algorithms especially for complex algorithms to be executed within coherence time.
Abstract Quantum compiling aims to construct a quantum circuitVby quantum gates drawn from a native gate alphabet, which is functionally equivalent to the target unitaryU. It is a crucial stage for the running of quantum algorithms on noisy intermediate-scale quantum (NISQ) devices. However, the space for structure exploration of quantum circuit is enormous, resulting in the requirement of human expertise, hundreds of experimentations or modifications from existing quantum circuits. In this paper, we propose a variational quantum compiling (VQC) algorithm based on reinforcement learning, in order to automatically design the structure of quantum circuit for VQC with no human intervention. An agent is trained to sequentially select quantum gates from the native gate alphabet and the qubits they act on by doubleQ-learning withϵ-greedy exploration strategy and experience replay. At first, the agent randomly explores a number of quantum circuits with different structures, and then iteratively discovers structures with higher performance on the learning task. Simulation results show that the proposed method can make exact compilations with less quantum gates compared to previous VQC algorithms. It can reduce the errors of quantum algorithms due to decoherence process and gate noise in NISQ devices, and enable quantum algorithms especially for complex algorithms to be executed within coherence time.
With the evolution of technology, the need to solve complex computational problems like machine learning and deep learning has shot up. However, even the most powerful classical supercomputers find it difficult to execute these tasks. Advancements in quantum computing are leading researchers and tech-giants who strive for better quantum circuits to do machine learning tasks. Current works on Quantum Machine Learning (QML) ensure less memory consumption and reduced model parameters. However, it is strenuous to simulate the classical deep learning approach on existing quantum computing devices due to the inflexibility of Deep quantum circuits. Consequently, designing viable quantum algorithms for QML for noisy intermediate-scale quantum (NISQ) devices is essential. The proposed work aims to explore Variational Quantum Circuits (VQC) for Deep Q network-based Reinforcement Learning by remodeling the target network and experience replay into a representation of VQC. In addition, to the reduction in model parameters, quantum information encoding schemes are used to achieve better results than classical neural networks. VQCs are employed for policyselection and decision-making reinforcement learning by approximating the deep Q-value function with target network and experience replay.
Abstract Effectively implementing quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we evaluate a reinforcement learning (RL)-based quantum compiler on a superconducting processor. Our experiments reveal that for two-qubit circuits, the RL-based compiler surpasses conventional methods, demonstrating its ability to discover hardware-amenable circuits with near-optimal lengths. However, for three-qubit circuits, the RL-based compiler does not achieve unity theoretical fidelity. To address this limitation, we integrate a variational strategy with the RL-based compiler, highlighting their complementary strengths. Systematic experiments show that this variational RL-based compiler consistently identifies near-optimal circuits, even under stringent hardware constraints, outperforming conventional techniques. Furthermore, we analyze the impact of decoherence and gate errors, providing critical insights into the practical performance of RL-based compilers on quantum hardware. These findings exemplify the codesign of the software with hardware for efficient quantum compilation, offering valuable insights for the advancement of RL-based compilers.
The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate scale quantum (NISQ) devices. This work explores variational quantum circuits for deep reinforcement learning. Specifically, we reshape classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits. Moreover, we use a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks. To the best of our knowledge, this work is the first proof-of-principle demonstration of variational quantum circuits to approximate the deep Q-value function for decision-making and policy-selection reinforcement learning with experience replay and target network. Besides, our variational quantum circuits can be deployed in many near-term NISQ machines.
Variational quantum circuits (VQCs) constitute a central building block of near-term quantum machine learning (QML), yet the principled design of expressive and trainable architectures remains a major open challenge. The VQC design space grows combinatorially with the number of qubits, layers, entanglement structures, and gate parameterizations, rendering manual circuit construction inefficient and often suboptimal. We introduce an autonomous agent-based framework for VQC architecture search that integrates high-level reasoning with a quantum simulation environment. The agent proposes candidate circuit architectures, evaluates them through fully automated training and validation pipelines, and iteratively improves its design strategy via performance-driven feedback. Empirically, we show that the agent autonomously evolves circuit architectures from simple initial ansätze toward increasingly expressive designs, progressively trying to improve task performance. This demonstrates that agentic AI can effectively navigate and refine the VQC design landscape with minimal human intervention, providing a scalable methodology for automated quantum model development in the Noisy Intermediate-Scale Quantum (NISQ) regime.
Variational quantum compiling (VQC) algorithms aim to approximate deep quantum circuits with shallow parameterized ansatzes, making them more suitable for NISQ hardware. In this article a variant of VQC named the recursive variational quantum compiling (RVQC) algorithm is proposed. Existing VQC algorithms typically require coherently executing the full circuit during compilation. Under the influence of noise, sufficiently deep target circuits make compiling unfeasible using ordinary VQC. Since the compiling is often accomplished using a gradient-based quantum-classical approach, the quantum noise manifest as a noisy gradient during optimization, making convergence hard to obtain. On the other hand, RVQC can compile a circuit by first dividing it into N shorter sub-circuits, then evaluate one sub-circuit at a time. As a result, the circuit depth required to implement RVQC is not dependent on the depth of the target circuit, but on the depth of the sub-circuits. Choosing a high enough N thus ensures sufficiently shallow sub-circuit which can be successfully compiled individually. We show mathematical evidence of this property. RVQC was compared with VQC on a noise model of the IBM Santiago device with the goal of compiling several randomly generated five-qubit circuits of approximately depth 1000. It was shown that VQC was not able to converge within 500 iterations of optimization. On the other hand, RVQC was able to converge to a fidelity of 0.90±0.05 within a total of 500 iterations when splitting the target circuits into N=5 parts. We argue that this comes as a result of the mitigation of noise-induced barren plateaus.
A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an ε-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.