Distributed hybrid quantum-classical performance prediction for hyperparameter optimization
Eric Wulff, Juan Pablo García Amboage, Marcel Aach, Þorsteinn Elí Gíslason, Thorsteinn Kristinn Ingolfsson, Tomas Kristinn Ingolfsson, Edoardo Pasetto, Amer Delilbasic, Morris Riedel, Rakesh Sarma, Maria Girone, Andreas Lintermann
Abstract Hyperparameter Optimization of neural networks is a computationally expensive procedure, which requires a large number of different model configurations to be trained. To reduce such costs, this work presents a distributed, hybrid workflow, that runs the training of the neural networks on multiple Graphics Processing Units (GPUs) on a classical supercomputer, while predicting the configurations' performance with Quantum Support Vector Regression on a Quantum Annealer (QA). The workflow is shown to run on up to 50 GPUs and a QA at the same time, completely automating the communication between the classical and the quantum system. The approach is evaluated extensively on several benchmarking datasets from the Computer Vision, High Energy Physics, and Natural Language Processing domains. Results show that resource savings of up to approximately 9% can be achieved while obtaining similar, and in some cases even better, accuracy, highlighting the potential of hybrid quantum-classical machine learning algorithms. The workflow code is made available open-source to foster adoption in the community.
Abstract Hyperparameter Optimization of neural networks is a computationally expensive procedure, which requires a large number of different model configurations to be trained. To reduce such costs, this work presents a distributed, hybrid workflow, that runs the training of the neural networks on multiple Graphics Processing Units (GPUs) on a classical supercomputer, while predicting the configurations' performance with Quantum Support Vector Regression on a Quantum Annealer (QA). The workflow is shown to run on up to 50 GPUs and a QA at the same time, completely automating the communication between the classical and the quantum system. The approach is evaluated extensively on several benchmarking datasets from the Computer Vision, High Energy Physics, and Natural Language Processing domains. Results show that resource savings of up to approximately 9% can be achieved while obtaining similar, and in some cases even better, accuracy, highlighting the potential of hybrid quantum-classical machine learning algorithms. The workflow code is made available open-source to foster adoption in the community.
The recent rise in the number of data-intensive applications has revealed some inherent limitations of traditional machine learning, such as high training costs, slow convergence rates, and the uncapable of scalability of traditional machine learning in high-dimensional spaces. Bottlenecks are caused by the iterative optimization procedure, kernel evaluations, and similarity computations that are not efficiently supported by traditional hardware. Although recent advances in quantum computing, such as the Variational Quantum Algorithm, quantum kernels, and parameterized quantum circuits, hold promise for acceleration, the application of quantum speedup is presently hindered by noisy intermediate-scale quantum computing. This research provides a hybrid quantum-classical acceleration method that integrates quantum methods with selected computationally intensive components of classical machine learning systems rather than attempting to entirely replace them. Carried out a comprehensive analysis of the performance of well-known algorithms like Support Vector Machines, k-means clustering, and gradient-based neural networks in classical and quantum-assisted hybrid scenarios. The quantum resources are greatly beneficial for particular sub-tasks such as optimization, approximation of similarity, and exploration of the feature space, with the remaining learning process performed classically. Our analysis reveals that hybrid methods can offer a tangible acceleration and improved convergence behaviour in particular scenarios without sacrificing accuracy. Moreover, the analysis investigates the trade-offs associated with the utilization of quantum resources, robustness to noise, and orchestration costs.
The recent rise in the number of data-intensive applications has revealed some inherent limitations of traditional machine learning, such as high training costs, slow convergence rates, and the uncapable of scalability of traditional machine learning in high-dimensional spaces. Bottlenecks are caused by the iterative optimization procedure, kernel evaluations, and similarity computations that are not efficiently supported by traditional hardware. Although recent advances in quantum computing, such as the Variational Quantum Algorithm, quantum kernels, and parameterized quantum circuits, hold promise for acceleration, the application of quantum speedup is presently hindered by noisy intermediate-scale quantum computing. This research provides a hybrid quantum-classical acceleration method that integrates quantum methods with selected computationally intensive components of classical machine learning systems rather than attempting to entirely replace them. Carried out a comprehensive analysis of the performance of well-known algorithms like Support Vector Machines, k-means clustering, and gradient-based neural networks in classical and quantum-assisted hybrid scenarios. The quantum resources are greatly beneficial for particular sub-tasks such as optimization, approximation of similarity, and exploration of the feature space, with the remaining learning process performed classically. Our analysis reveals that hybrid methods can offer a tangible acceleration and improved convergence behaviour in particular scenarios without sacrificing accuracy. Moreover, the analysis investigates the trade-offs associated with the utilization of quantum resources, robustness to noise, and orchestration costs.
In recent years, the field of machine learning has witnessed a paradigm shift with the emergence of hybrid quantum-classical neural networks. These networks combine the power of classical neural networks with the computational advantages offered by quantum variational circuits. This study presents a comprehensive survey of hybrid quantum-classical machine learning algorithms, focusing on their architecture, performance, and hyperparameter optimization techniques. Introducing the concept of hybrid quantum-classical machine learning and its significance in contemporary image classification tasks. It explores the theoretical foundations of quantum variational circuits and their integration with classical neural network architectures. Special attention is given to prominent algorithms such as Quantum Convolutional Neural Networks, Quanvolutional Neural Networks, and Quantum ResNet, each offering unique advantages in image classification tasks. A major aspect covered in this study is the hyperparameter optimization techniques employed in hybrid quantum-classical machine learning. The chapter discusses quantum-inspired optimization methods tailored for efficiently tuning the hyperparameters of complex neural network models. Through comparative analysis, the effectiveness of these optimization techniques is demonstrated in reducing computational overhead and improving model performance. Moreover, the study also delves into experimental results obtained from benchmarking hybrid quantum-classical models against traditional classical approaches. It highlights the accuracy variations observed across different quantum architectures and explores the impact of architectural permutations on model convergence. Insights gained from these experiments provide valuable guidance for researchers and practitioners seeking to leverage hybrid quantum-classical neural networks in real-world applications.
As machine learning applications scale, optimization challenges in training deep models have intensified. Traditional gradient-based methods can struggle with high-dimensional, multimodal loss landscapes. This study proposes a quantum-inspired optimization model designed for large-scale ML workloads deployed in cloud environments. The framework incorporates simulated quantum tunneling, stochastic amplitude updates and adaptive annealing schedules to escape local minima more efficiently. It integrates seamlessly with distributed training systems such as Kubernetes and PyTorch Distributed. Experiments were conducted on vision and language benchmarks, including ImageNet and WikiText-103. Results show a 9 percent improvement in convergence speed and a 6.8 percent increase in final model accuracy compared with AdamW and Ranger optimizers. The method also demonstrated resilience to noisy gradients and unstable cloud GPU environments. Cost analysis highlights that fewer training epochs reduce compute expenses by up to 14 percent. These findings suggest that quantum-inspired optimization can enhance ML performance even without quantum hardware. Future work aims to integrate hybrid GPU-QPU cloud architectures.
Quantum machine learning is one of the fields where quantum computers are expected to bring advantages over classical methods. However, the limited size of current computers restricts the exploitation of the full potential of quantum machine learning methods. Additionally, different computing paradigms, both quantum and classical, each have their own strengths and weaknesses. Obtaining optimal results with algorithms thus requires algorithms to be tweaked to the underlying computational paradigm, and the tasks to be optimally distributed over the available computational resources. In this work, we explore the potential gains from combining different computing paradigms to solve the complex task of data classification for three different datasets. We use a gate-based quantum model to implement a quantum kernel and implement a complex feature map. Next, we formulate a quadratic unconstrained optimisation problem to be solved on quantum annealing hardware. We then evaluate the losses on classical hardware and reconfigure the model parameters accordingly. We tested this so-called triple-hybrid quantum support vector machine on various data sets, and find that it achieves higher precision than other support vector machines (both quantum and classical) on complex quantum data, whereas it achieves varying performance on simple classical data using limited training. For the complex data sets, the triple-hybrid version converges faster, requiring fewer circuit evaluations.
Use of AI/ML is associated with more and more computing power, which often necessitates the application to guzzle energy and take time to train. The paper presents a Quantum-Enhanced High-Performance Computing (Q-HPC) system that will combine traditional HPC units with quantum-assisted optimization to enhance the model training in addition to the predictive accuracy and energy efficiency. The framework that could be used to work with such large volumes of data is multi- GPU/CPU parallelization, and the optimization of parameters and hyperparameters can be implemented with the help of quantum-inspired algorithms. This will lead to a hybrid computation balancing dynamically the classical and quantum computations. Q-HPC was experimented with multiple AI/ML model types, such as convolutional networks, transformer models, graph neural networks and reinforcement learning agents, in which cases it was observed to traverse to the solution faster, more accurately and used less energy than the traditional HPC. It can also be said that the framework is dynamically adaptable and sustainable, i.e. it could be deployed to a massive diversifying range of tasks of AI/ML. The suggested model is a combination of the performance and scalability of the classical HPC with the optimization performance of quantum computing in order to create a new and valuable approach to the next generation AI. It deals with the issues of performance and environmental concerns of high-performance computing.