Quantum computers may solve some computing tasks more efficiently than classical computers, but to do so requires the design of appropriate quantum circuits. However, it is hard to design even simple quantum programs. Recent results have shown that deep learning augmented search has the potential to discover good circuits for a problem, but it is not yet sufficiently understood how well deep learning is able to model such a complex domain: the search space and output space grow exponentially. This work explores the ability of neural networks to encode quantum circuits for tasks that require knowledge about the unitary representation of the circuit. To this end, we trained neural networks to directly learn to predict the unitary of a circuit and applied reinforcement learning to train neural networks to solve circuit based quantum state preparation. This work finds that encoding quantum circuits is quite difficult especially with regards to the amount of entanglement applied in both application domains. The use of intermediate states and structure in quantum circuits combined with a reasonable inductive bias, where applicable, can alleviate these problems to some extent. The use of intermediate states also greatly improves the results of Deep Reinforcement Learning for quantum state preparation. Based on these results, we discuss the next set of challenges to address if we are to design neural network-based approaches for the automatic generation of quantum circuits. Add sentence on how it improves things in the end.
Quantum computers may solve some computing tasks more efficiently than classical computers, but to do so requires the design of appropriate quantum circuits. However, it is hard to design even simple quantum programs. Recent results have shown that deep learning augmented search has the potential to discover good circuits for a problem, but it is not yet sufficiently understood how well deep learning is able to model such a complex domain: the search space and output space grow exponentially. This work explores the ability of neural networks to encode quantum circuits for tasks that require knowledge about the unitary representation of the circuit. To this end, we trained neural networks to directly learn to predict the unitary of a circuit and applied reinforcement learning to train neural networks to solve circuit based quantum state preparation. This work finds that encoding quantum circuits is quite difficult especially with regards to the amount of entanglement applied in both application domains. The use of intermediate states and structure in quantum circuits combined with a reasonable inductive bias, where applicable, can alleviate these problems to some extent. The use of intermediate states also greatly improves the results of Deep Reinforcement Learning for quantum state preparation. Based on these results, we discuss the next set of challenges to address if we are to design neural network-based approaches for the automatic generation of quantum circuits. Add sentence on how it improves things in the end.
There is increasing interest in the area of quantum computing, but using and developing new quantum algorithms, often represented as quantum circuits, is arduous and requires designers with a wide area of experience. The exploration of techniques for the automatic design of quantum algorithms, i.e., quantum architecture search, is therefore of high importance. This work explores the use of neural networks for quantum architecture search, since neural networks have proven quite successful in various complex search domains, and current approaches in quantum architecture search make very restricted use of them. There are many different domains in which quantum architecture search can be applied. They can be used to compile unitaries into quantum circuits, develop ansätze for variational quantum circuits, or develop new quantum algorithms by applying it to input output pairs of one task one wishes to solve. The focus in this thesis is on the task of quantum state preparation, a very important task in the field of quantum computing, as it is a subroutine in a large number of quantum algorithms like quantum machine learning and Hamiltonian simulation. This thesis approaches the task of quantum architecture search first by establishing the importance of the structure of quantum circuits. This is done with a large-scale study of the influence different circuit structure parameters, such as qubits and depth, have on the performance on variational quantum machine learning. During this research, four guidelines for designing circuits for variational quantum machine learning were established. DeepQPrep, a neural network-augmented deductive search for the automatic design of quantum circuits for quantum state preparation, was proposed. It is able to create non-parameterized quantum circuits to solve 76.9% of a restrictive set of quantum states with quantum circuits that were much shallower than the ones found in the ground-truth data. The model was trained on a dataset created in this thesis as none were available at the time. During the exploration of extensions of DeepQPrep to enable it to solve more arbitrary quantum state preparation problems without the need for labeled data, different challenges became apparent. A study on the difficulty of training neural networks to create parameterized quantum circuits using reinforcement learning found that training is hampered by the barren plateau problem of quantum computing. It shows that the variance of the gradient of the quantum circuit output with respect to its parameters goes exponentially to zero. A study showed that this problem makes the application of neural networks to encode quantum circuits highly challenging when no intermediate quantum state information is used. This culminated in DeepQPrepGrad, which combines parameterized circuits, the inductive search of DeepQPrep, and the reinforcement learning-based approach with more compact representations. By combining these parts with the gradients of quantum circuits, an approach that is able to create parameterized quantum circuits for arbitrary quantum states was obtained. The approach was evaluated on 20000 arbitrary 3-qubit quantum states without the need for any labeled data, resulting in a mean minimal similarity of 0.97 alleviated the found challenges to a large degree. This thesis explored key challenges faced when training neural networks to encode quantum circuits. These challenges are of paramount importance as they potentially affect a wide range of tasks in the field of quantum computing to which neural networks could be applied. Lastly, DeepQPrepGrad provides a significant step forward for the creation of quantum circuits using neural networks. DeepQPrepGrad makes great progress in solving the challenges highlighted above and has the potential to be applicable in a wide range of quantum architecture search tasks, such as variational eigensolver and variational quantum machine learning.
The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate scale quantum (NISQ) devices. This work explores variational quantum circuits for deep reinforcement learning. Specifically, we reshape classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits. Moreover, we use a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks. To the best of our knowledge, this work is the first proof-of-principle demonstration of variational quantum circuits to approximate the deep Q-value function for decision-making and policy-selection reinforcement learning with experience replay and target network. Besides, our variational quantum circuits can be deployed in many near-term NISQ machines.
There is an increasing interest in the area of quantum computing but developing quantum algorithms is difficult. Neural Network augmented search algorithms have proven quite successful for general search problems (like program generation) but current approaches to quantum program generation make very restricted use of them. In this paper we present DeepQPrep, a Neural Network based approach to generate quantum circuits for state preparation; a common yet expensive task needed in many applications of quantum computing. We illustrate that Neural Network augmented search algorithms have significant promise for automated quantum program generation; DeepQPrep generated programs were able to solve 99% and 76.9% of 20000 previously unseen state prepartion tasks in a supervised machine learning context within two different application scenarios. The circuits produced by DeepQPrep are also shallower (on average) than their ground truth counterparts. We also compare DeepQPrep to IBM Qiskit’s approach to state preparation and illustrate that even when constrained, DeepQPrep generates significantly shallower circuits despite Qiskit solving more of the state preparation tasks. Based on our results, we argue that neural network augmented search approaches exhibit significant promise for generalised approaches to quantum program induction warranting further study in more complex scenarios.
With the evolution of technology, the need to solve complex computational problems like machine learning and deep learning has shot up. However, even the most powerful classical supercomputers find it difficult to execute these tasks. Advancements in quantum computing are leading researchers and tech-giants who strive for better quantum circuits to do machine learning tasks. Current works on Quantum Machine Learning (QML) ensure less memory consumption and reduced model parameters. However, it is strenuous to simulate the classical deep learning approach on existing quantum computing devices due to the inflexibility of Deep quantum circuits. Consequently, designing viable quantum algorithms for QML for noisy intermediate-scale quantum (NISQ) devices is essential. The proposed work aims to explore Variational Quantum Circuits (VQC) for Deep Q network-based Reinforcement Learning by remodeling the target network and experience replay into a representation of VQC. In addition, to the reduction in model parameters, quantum information encoding schemes are used to achieve better results than classical neural networks. VQCs are employed for policyselection and decision-making reinforcement learning by approximating the deep Q-value function with target network and experience replay.
Quantum computing promises significant computational advantages over classical methods in certain applications but is currently too unstable for executing complex tasks. High-fidelity quantum circuits are critical to overcoming this challenge, yet they are computationally challenging to design. On current noisy intermediate-scale quantum (NISQ) hardware, shallower circuits with fewer two-qubit gates generally run with higher fidelity. Increased fidelity can be sought by sequentially rewriting the circuit, which involves a vast search space. This costly search space exploration could be circumvented by using reinforcement learning (RL) to teach an agent to rewrite the circuit in a sequential decision-making process. This thesis aims to implement a Deep Q-Network (DQN) algorithm to optimize BernsteinVazirani (BVZ) circuits. The DQN algorithm interacts with a Cirq-based environment. The environment provides valid rewrite actions to the DQN algorithm, which gains a scalar reward emphasizing depth and gate reduction. In the thesis we compare two different state encodings as input for the neural network: a compact state summarizing vector representation with three features, and an image-like encoding that maps circuit state topology with tokens through a fixed embedding. Both input encodings use the same action rewrite set and reward design. The network is implemented in PyTorch and trained with Adam optimizer, and runs on GPU via CUDA. Empirically, the agent shows promising initial optimizing performance. On several BVZ circuits with varying qubit count and topology, the agent drives the depth toward the theoretical optimum of three. The simplistic vector encoding already reaches reasonable circuit depth plateaus, with the image-like encoding together with hyperparameter tuning improving reliability, showing avenues for continued improvement. We conclude that DQN is a viable foundation for automated circuit optimization on BVZ circuits. To achieve reliable circuit optimization at scale, future research should investigate even richer encoding methods and reward function shaping. Finally, the results support applying the DQN to be applied to circuit families beyond BVZ.
The development of quantum computational techniques has advanced greatly in recent years, parallel to the advancements in techniques for deep reinforcement learning. This work explores the potential for quantum computing to facilitate reinforcement learning problems. Quantum computing approaches offer important potential improvements in time and space complexity over traditional algorithms because of its ability to exploit the quantum phenomena of superposition and entanglement. Specifically, we investigate the use of quantum variational circuits, a form of quantum machine learning. We present our techniques for encoding classical data for a quantum variational circuit, we further explore pure and hybrid quantum algorithms for DQN and Double DQN. Our results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space. These comparison are conducted with two OpenAI Gym environments: CartPole and Blackjack, The success of this work is indicative of a strong future relationship between quantum machine learning and deep reinforcement learning.