As large language models (LLMs) demonstrate outstanding performance across various tasks, attention-driven models have profoundly transformed the field of machine learning. Since attention computations account for the primary computational overhead in both model inference and training, efficiently computing attention matrices has become one of the core challenges in accelerating large language models. It is well-known that quantum machines possess computational advantages over classical machines, and the role of quantum computing in LLMs remains largely unexplored. In this work, we focus on leveraging the Grover search algorithm to efficiently compute a sparse attention matrix. Through comparisons with classical algorithms, we demonstrate that our method achieves quantum acceleration in polynomial time. Additionally, we observe that the generated quantum attention matrices naturally exhibit low-rank structures, providing further theoretical support for efficient modeling. Moreover, within the specific context of attention matrix computation, we conduct a systematic and detailed analysis of the error and time complexity of the proposed algorithm.
As large language models (LLMs) demonstrate outstanding performance across various tasks, attention-driven models have profoundly transformed the field of machine learning. Since attention computations account for the primary computational overhead in both model inference and training, efficiently computing attention matrices has become one of the core challenges in accelerating large language models. It is well-known that quantum machines possess computational advantages over classical machines, and the role of quantum computing in LLMs remains largely unexplored. In this work, we focus on leveraging the Grover search algorithm to efficiently compute a sparse attention matrix. Through comparisons with classical algorithms, we demonstrate that our method achieves quantum acceleration in polynomial time. Additionally, we observe that the generated quantum attention matrices naturally exhibit low-rank structures, providing further theoretical support for efficient modeling. Moreover, within the specific context of attention matrix computation, we conduct a systematic and detailed analysis of the error and time complexity of the proposed algorithm.
Large Language Models (LLMs) such as GPT and LLaMA have transformed artificial intelligence, but their rapid growth has brought major challenges in computation, energy use, and scalability. Quantum computing (QC) offers a fundamentally different way to process information, with the potential to accelerate key operations in training, inference, optimization, and representation learning. This survey provides a structured and comprehensive review of how QC can support the future development of LLMs. It examines quantum algorithms, hybrid quantum-classical methods, quantum neural networks, quantum embeddings, and security applications. It also evaluates their feasibility in both the current noisy intermediate-scale quantum (NISQ) era and the future fault-tolerant era. The survey highlights current progress, identifies major technical barriers, and outlines practical research directions needed to build scalable, efficient, and secure quantum-enhanced language models.
Quantum Machine Learning (QML) is a rapidly developing field that explores the potential of quantum computers to enhance machine learning algorithms. A critical challenge in QML is the design of efficient and effective attention mechanisms, which are fundamental to many deep learning architectures, particularly in natural language processing and computer vision. This paper investigates the application of adaptive attention mechanisms within the context of QML, specifically focusing on the design of a novel self-adaptive attention mechanism leveraging quantum properties. We propose a framework that dynamically adjusts the attention weights based on the quantum state, allowing for a more nuanced and efficient modeling process. The core claim is that this design will result in improved model performance and enhanced learning efficiency compared to traditional attention mechanisms, utilizing the inherent advantages of quantum computing. This work presents a theoretical analysis and preliminary implementation of this novel approach.
Abstract Quantum computing is an exciting non-Von Neu-mann paradigm, offering provable speedups over classical computing for specific problems. However, the practical limits of classical simulatability for quantum circuits remain unclear, especially with current noisy quantum devices. In this work, we explore the potential of leveraging Large Language Models (LLMs) to simulate the output of a quantum Turing machine using Grover’s quantum circuits, known to provide quadratic speedups over classical counterparts. To this end, we developed GroverGPT, a specialized model based on LLaMA’s 8-billion-parameter architecture, trained on over 15 trillion tokens. Unlike brute-force state-vector simulations, which demand substantial computational resources, GroverGPT employs pattern recognition to approximate quantum search algorithms without explicitly representing quantum states. Analyzing 97K quantum search instances, GroverGPT consistently outperformed OpenAI’s GPT-4o (45% accuracy), achieving nearly 100% accuracy on 6-and 10-qubit datasets when trained on 4-qubit or larger datasets. It also demonstrated strong generalization, surpassing 95% accuracy for systems with over 20 qubits when trained on 3-to 6-qubit data. Analysis indicates GroverGPT captures quantum features of Grover’s search rather than classical patterns, supported by novel prompting strategies to enhance performance. Although accuracy declines with increasing system size, these findings offer insights into the practical boundaries of classical simulatability. This work suggests task-specific LLMs can surpass general-purpose models like GPT-4o in quantum algorithm learning and serve as powerful tools for advancing quantum research.
Quantum computation offers new directions in computer science that go beyond classical systems' state-of-the-art abilities. The combination of quantum technologies with Large Language Models (LLMs) provides promising avenues for the creation of more efficient and powerful NLP architectures. In this research, we examine quantum-enhanced transformers and further develop the Self-Attention Sequential Quantum Transformer Channel (SASQuatch) framework to enhance inference time and perplexity. The models proposed are tested on sentence completion tasks and compared against a baseline transformer, showing significant gains in efficiency and predictive power. We further compare the method with other optimization methods, such as the Quantum Approximate Optimization Algorithm (QAOA), thus highlighting the necessity of further investigation of hybrid quantum architectures in improving the performance of large-scale language models.
Significant computational challenges exist in the training of large language models (LLMs), specifically for the efficient convergence. This paper discusses a hybrid quantum-classical framework designed to address these challenges. With the integration of quantum computing principles, like the superposition, entanglement, and tunneling, into the classical deep learning methods, an approach is studied to accelerate convergence, enhance optimization efficiency, and improve model generalization. A quantum feature mapping is used to project classical data into high-dimensional Hilbert spaces, enhancing data representation and revealing more complex feature patterns in the data. The quantum-assisted optimization algorithms, like Quantum Approximate Optimization Algorithm (QAOA) and Variational Quantum Eigensolver (VQE), can efficiently find optimal solutions within complex, non-convex loss landscapes and mitigate the issues of local minima encountered by classical methods. Furthermore, the quantum-accelerated matrix operations, with the use of Harrow-Hassidim-Lloyd (HHL) algorithm or Quantum Fourier Transform (QFT) etc., can provide computational speed-ups for the LLM training. Quantum measurement introduces inherent randomness, similar to the dropout in deep neural networks, where certain neurons are randomly deactivated during training. In the noisy intermediate-scale quantum (NISQ) era, our quantum-classical framework and conceptual analyses demonstrate the feasibility and potential advantages. This work lays the foundation for future research in this area.
Powerful generative artificial intelligence from large language models (LLMs) harnesses extensive computational resources for inference. In this work, we investigate the transformer architecture, a key component of these models, under the lens of fault-tolerant quantum computing. We develop quantum subroutines to construct the building blocks in the transformer, including the self-attention, residual connection with layer normalization, and feed-forward network. As an important subroutine, we show how to efficiently implement the Hadamard product and element-wise functions of matrices on quantum computers. Our algorithm prepares an amplitude encoding of the transformer output, which can be measured for prediction or use in the next layer. We find that the matrix norm of the input sequence plays a dominant role in the quantum complexity. With numerical experiments on open-source LLMs, including for bio-informatics applications, we demonstrate the potential of a quantum speedup for transformer inference in practical regimes.