In this work, we first consider standard quantum control problems for superconducting transmon qubits. Such quantum systems have been studied extensively both from an analytical and a numerical perspective [MSG&al11; TMW16]. Analytical pulses usually show dependencies from system parameters that are not easily found in numerical solutions. Therefore, our approach is then to employ neural networks to learn the functional dependence of optimal quantum control solutions from system parameters. We show that we can optimize such solutions analytically for single and two-qubit gates using either very few parameter samples or large numbers of pulse frequencies for large parameter ranges. Afterward, we move to a higher level of optimization. We consider variational gates in trapped-ion quantum computers [MMN&al16]. In this setting, we employ a hybrid scheme that uses both reinforcement learning and continuous optimization to optimize both circuit structure and variational angles concurrently [BDS&al18; SEL&al22]. We show that our reinforcement learning algorithm assisted by a continuous optimizer can construct effective solutions to the gate synthesis problem that matches and surpasses standard circuit compilers. Next, we consider variational optimization of non-linear maps [HCS&al23] acting on quantum states such as entanglement purification protocols. An entanglement purification protocol acts as a highly non-linear map, but it still outputs a faithful representation of a quantum system. Optimizing it for a specific family of states requires reducing the number of operations needed to retrieve the original state. This optimization has the potential to make purification protocols easier to implement on a quantum device. In fact, the exponential scaling of the purification protocol with the number of states [DB07] implies that such protocols may not be applied directly without careful engineering. We also show that the performance of standard purification protocols for arbitrary two-qubit input states leads to poor output values of the concurrence. Our optimized protocols prove instead able to increase the value of the concurrence above the maximum limit of traditional analytical protocols. We also show how the twirling operation becomes an obstacle to the performance of the protocol itself when considering random two-qubit states, although it is a useful tool in the design of entanglement purification protocols. Finally, we study parameter sampling in quantum circuits, focusing in particular on the LCU methods. Such problems are particularly interesting for meta-variational settings [CKA21] where we compute the average over observables evaluated at different points in the parameter space. They are also relevant for optimal quantum control algorithms because the computation of the fidelity with respect to a target operation is the basis of most optimal quantum control routines. In conclusion, we analyze several optimization problems that are relevant for quantum science and technology. We show that machine learning-assisted solutions can be applied successfully to engineer optimal quantum control pulses and compilation strategies based on variational angles. Also in the context of entanglement purification, we show how our optimized protocols can surpass current strategies for multiparametric families of states.
In this work, we first consider standard quantum control problems for superconducting transmon qubits. Such quantum systems have been studied extensively both from an analytical and a numerical perspective [MSG&al11; TMW16]. Analytical pulses usually show dependencies from system parameters that are not easily found in numerical solutions. Therefore, our approach is then to employ neural networks to learn the functional dependence of optimal quantum control solutions from system parameters. We show that we can optimize such solutions analytically for single and two-qubit gates using either very few parameter samples or large numbers of pulse frequencies for large parameter ranges. Afterward, we move to a higher level of optimization. We consider variational gates in trapped-ion quantum computers [MMN&al16]. In this setting, we employ a hybrid scheme that uses both reinforcement learning and continuous optimization to optimize both circuit structure and variational angles concurrently [BDS&al18; SEL&al22]. We show that our reinforcement learning algorithm assisted by a continuous optimizer can construct effective solutions to the gate synthesis problem that matches and surpasses standard circuit compilers. Next, we consider variational optimization of non-linear maps [HCS&al23] acting on quantum states such as entanglement purification protocols. An entanglement purification protocol acts as a highly non-linear map, but it still outputs a faithful representation of a quantum system. Optimizing it for a specific family of states requires reducing the number of operations needed to retrieve the original state. This optimization has the potential to make purification protocols easier to implement on a quantum device. In fact, the exponential scaling of the purification protocol with the number of states [DB07] implies that such protocols may not be applied directly without careful engineering. We also show that the performance of standard purification protocols for arbitrary two-qubit input states leads to poor output values of the concurrence. Our optimized protocols prove instead able to increase the value of the concurrence above the maximum limit of traditional analytical protocols. We also show how the twirling operation becomes an obstacle to the performance of the protocol itself when considering random two-qubit states, although it is a useful tool in the design of entanglement purification protocols. Finally, we study parameter sampling in quantum circuits, focusing in particular on the LCU methods. Such problems are particularly interesting for meta-variational settings [CKA21] where we compute the average over observables evaluated at different points in the parameter space. They are also relevant for optimal quantum control algorithms because the computation of the fidelity with respect to a target operation is the basis of most optimal quantum control routines. In conclusion, we analyze several optimization problems that are relevant for quantum science and technology. We show that machine learning-assisted solutions can be applied successfully to engineer optimal quantum control pulses and compilation strategies based on variational angles. Also in the context of entanglement purification, we show how our optimized protocols can surpass current strategies for multiparametric families of states.
Control synthesis for continuously-parameterized families of quantum gates can enable critical advantages for mid-sized quantum computing applications in advance of fault-tolerance. We combine quantum optimal control with physics-informed machine learning to efficiently synthesize control surfaces that interpolate among continuously-parameterized gate families. Using optimal control as an active learning strategy to guide pretraining, we bootstrap a physics-informed neural network to achieve rapid convergence to nonlinear control surfaces sufficient for our desired gates. We find our approach is critical for enabling an expressiveness beyond linear interpolation, which is important in cases of hard quantum control. We show in simulation that by adapting our pretraining to use a few reference pulse calibrations, we can apply transfer learning to quickly calibrate our learned control surfaces when devices fluctuate over time. We demonstrate synthesis for one and two qubit gates with one or two parameters, focusing on gate families for variational quantum algorithm (VQA) ansatz. By avoiding the inefficient decomposition of VQA ansatz into basis gate sets, continuous gate families are a potential method to improve the noise robustness of VQAs in the near term. Our framework shows how accessible optimal control tools can be combined with simple machine learning to enable practitioners to achieve 3x speedups for their algorithms by going beyond the standard gate sets.
Abstract The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.
Control synthesis for continuously-parameterized families of quantum gates can enable critical advantages for mid-sized quantum computing applications in advance of fault-tolerance. We combine quantum optimal control with physics-informed machine learning to efficiently synthesize control surfaces that interpolate among continuously-parameterized gate families. Using optimal control as an active learning strategy to guide pretraining, we bootstrap a physics-informed neural network to achieve rapid convergence to nonlinear control surfaces sufficient for our desired gates. We find our approach is critical for enabling an expressiveness beyond linear interpolation, which is important in cases of hard quantum control. We show in simulation that by adapting our pretraining to use a few reference pulse calibrations, we can apply transfer learning to quickly calibrate our learned control surfaces when devices fluctuate over time. We demonstrate synthesis for one and two qubit gates with one or two parameters, focusing on gate families for variational quantum algorithm (VQA) ansatz. By avoiding the inefficient decomposition of VQA ansatz into basis gate sets, continuous gate families are a potential method to improve the noise robustness of VQAs in the near term. Our framework shows how accessible optimal control tools can be combined with simple machine learning to enable practitioners to achieve 3x speedups for their algorithms by going beyond the standard gate sets.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.
The convergence of machine learning (ML) and quantum computing is quickly becoming a game-changing new frontier in computational science. The most intriguing area of overlap may be quantum optimization, which seeks to reduce the exponentially increasing complexity and computational cost of ML algorithms. As they grow larger and more complicated, traditional optimization methods, particularly in high-dimensional and non-convex space, will inevitably hit performance limits. Quantum optimization is the whole new paradigm that leverages quantum mechanical effects like superposition, entanglement, and tunnelling to explore solution spaces more efficiently. The chapter outlines the theory, practice, and potential of hybrid quantum-classical optimization algorithms for machine learning. The most brilliant research direction in the current NISQ (noisy intermediate-scale quantum) era is mixed systems leveraging classical computing capability augmented by quantum capacity. In such designs, computationally intensive subroutines like optimization for global optima or escape from local minima are left to quantum processors and data preprocessing, iterative refinement, and model testing are performed by classical processors. We start by defining the fundamental principles of quantum computing and acquainting readers with the fundamentals like qubits, quantum gates, and quantum circuits. These are the building blocks for quantum algorithms, which hopefully can be utilized for optimization, for example, the QAOA (quantum approximate optimization algorithm), the VQE (variational quantum Eigensolver), and quantum annealing. They all build to optimize the problems in some way differently: QAOA through parameterized quantum circuits, VQE through approximations of ground states of the Hamiltonians, and quantum annealing through quantum evolution to low-energy solutions. We then form a loop between these quantum algorithms and regular machine learning tasks. Optimization impacts virtually all ML phases, from training neural networks, where one reduces loss functions; feature selection, where one aims to discover the most relevant subset of input variables; and hyperparameter optimization, which often involves expensive search procedures over extremely large combinatorial spaces. Quantum optimization techniques can therefore, in principle, accelerate such procedures through computational complexity reduction and enhancing solutions over challenging landscapes. Realistic simulations and case studies emphasize the degree to which hybrid quantum-classical algorithms have been employed in solving ML problems, that is, QSVMs, quantum-enhanced clustering, and quantum reinforcement learning. Despite their experimental nature, these applications already show encouraging results in terms of improved performance metrics as well as reduction in time-to-solution compared to classical-only solutions. Notwithstanding that, this new field is not bereft of real challenges. Quantum hardware suffers from hard limits on qubit fidelity, decoherence, and scalability in the current times. Furthermore, quantum circuit integration into broad ML pipelines requires novel architectures and software platforms. As a counter to those issues, this chapter surveys recent advances in quantum programming languages, simulators, and hybrid platforms like PennyLane, Qiskit, and TensorFlow Quantum. We conclude by looking to the future possibility that quantum optimization will revolutionize the machine learning landscape. As hardware improves and algorithmic complexity increases, the possibility of quantum-powered learning systems solving hard problems becomes increasingly accessible. From speeding up deep network training to enabling new types of unsupervised learning, quantum optimization holds the potential to recast computational tractability in machine learning. This chapter is an all-encompassing introduction and state-of-the-art work for students, researchers, and practitioners interested in using quantum optimization for practical ML applications. It sets the foundation for understanding how to pursue hybrid approaches (how they may be applied today in the real world and bracing oneself for disruption in the future).