Transformers excel in various domains but often demand massive computational resources. Quantum Neural Networks (QNNs) combine classical and quantum computing to mitigate these costs. However, QNNs typically fix the number of qubits, forcing complete retraining whenever the qubit count changes and leading to substantial inefficiency. We propose an Adaptive Quantum Transformer (AQT) to resolve this limitation by introducing an adapter module — originally designed for large language models (LLMs) — into a quantum vision Transformer. This module reuses parameters when the qubit count expands or contracts, preserving learned states. We validate AQT in an autoscaling cloud environment, demonstrating dynamic qubit scaling without sacrificing accuracy or retraining from scratch. Results on vision tasks reveal that AQT achieves higher accuracy than a conventional quantum vision Transformer while converging faster and requiring fewer computational resources. Notably, AQT provides over triple the efficiency in training time and better stability under fluctuating qubit availability, suggesting a promising avenue for practical, cost-effective quantum–classical hybrid intelligence.
Transformers excel in various domains but often demand massive computational resources. Quantum Neural Networks (QNNs) combine classical and quantum computing to mitigate these costs. However, QNNs typically fix the number of qubits, forcing complete retraining whenever the qubit count changes and leading to substantial inefficiency. We propose an Adaptive Quantum Transformer (AQT) to resolve this limitation by introducing an adapter module — originally designed for large language models (LLMs) — into a quantum vision Transformer. This module reuses parameters when the qubit count expands or contracts, preserving learned states. We validate AQT in an autoscaling cloud environment, demonstrating dynamic qubit scaling without sacrificing accuracy or retraining from scratch. Results on vision tasks reveal that AQT achieves higher accuracy than a conventional quantum vision Transformer while converging faster and requiring fewer computational resources. Notably, AQT provides over triple the efficiency in training time and better stability under fluctuating qubit availability, suggesting a promising avenue for practical, cost-effective quantum–classical hybrid intelligence.
Quantum machine learning (QML) is a promising application of quantum computing, yet when and how practical learning models can achieve quantum advantage remains poorly understood. Even a polynomial improvement in the resources needed to attain a given predictive performance would be significant. In this work, we investigate empirical scaling laws of quantum vision transformers (QViTs), which connect QML to the transformer architecture underlying modern large language models. We ask whether QViTs exhibit scaling behavior analogous to classical models and how this behavior changes when qubit number is included alongside dataset size and model parameters. The validation losses in our numerical image-classification experiments are well described by a joint shifted power-law model in qubit number, transformer depth, and training dataset size. Expressing this fit in terms of the classical encoder parameter budget identifies a regime in which increasing the number of qubits can compensate for fewer classical encoder parameters at comparable predicted performance. This finding is consistent with earlier observations of parameter efficiency in quantum learning models and suggests a direction for exploring possible resource savings in QML.
In this work, quantum transformers are designed and analysed in detail by extending the state-of-the-art classical transformer neural network architectures known to be very performant in natural language processing and image analysis. Building upon the previous work, which uses parametrised quantum circuits for data loading and orthogonal neural layers, we introduce three types of quantum transformers for training and inference, including a quantum transformer based on compound matrices, which guarantees a theoretical advantage of the quantum attention mechanism compared to their classical counterpart both in terms of asymptotic run time and the number of model parameters. These quantum architectures can be built using shallow quantum circuits and produce qualitatively different classification models. The three proposed quantum attention layers vary on the spectrum between closely following the classical transformers and exhibiting more quantum characteristics. As building blocks of the quantum transformer, we propose a novel method for loading a matrix as quantum states as well as two new trainable quantum orthogonal layers adaptable to different levels of connectivity and quality of quantum computers. We performed extensive simulations of the quantum transformers on standard medical image datasets that showed competitively, and at times better performance compared to the classical benchmarks, including the best-in-class classical vision transformers. The quantum transformers we trained on these small-scale datasets require fewer parameters compared to standard classical benchmarks. Finally, we implemented our quantum transformers on superconducting quantum computers and obtained encouraging results for up to six qubit experiments.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%, and to 16.84% for Gemma-2, with a marginal performance improvement of 0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
The remarkable success of Transformer architectures in Large Language Models (LLMs) has revolutionized natural language processing, yet the transition to quantum computing for next-generation language models remains an open challenge. While quantum computing promises exponential advantages, a fundamental gap exists between classical deep learning and quantum computing paradigms, particularly given the severe constraints of Noisy Intermediate-Scale Quantum (NISQ) devices, including barren plateaus, limited qubit coherence, and circuit depth restrictions. We present Adaptive Quantum-Classical Fusion (AQCF), the first framework to bridge classical and quantum computing for language models by reimagining Transformer architectures through quantum-classical co-design. Our key insight is that effective bridging requires dynamic adaptation rather than static translation—the framework analyzes input complexity in real-time to orchestrate seamless transitions between classical and quantum processing. AQCF introduces entropy-driven adaptive circuits that circumvent barren plateaus, quantum memory banks that unify classical attention with quantum state-based similarity retrieval, and intelligent fusion controllers that ensure each computational paradigm handles tasks where it naturally excels. This bridging architecture maintains full compatibility with existing classical Transformers while progressively incorporating quantum advantages as they become accessible. Experiments on sentiment analysis demonstrate that AQCF achieves competitive performance while significantly improving quantum resource efficiency, operating successfully within typical NISQ constraints. By establishing a seamless integration pathway from today's classical LLMs to tomorrow's quantum-enhanced models, our framework provides both immediate practical value on current quantum hardware and a clear evolution path toward full Quantum LLMs as technology matures.
Image classification plays a vital role in large-scale data analysis, especially in object recognition tasks using advanced Deep Learning (DL) frameworks. However, the growing complexity and computational demands of modern DL models have introduced challenges related to scalability and efficiency. Quantum Computing (QC) has emerged as a promising alternative, capable of addressing these limitations by leveraging the principles of Quantum Machine Learning (QML). However, many existing QML models require a large number of qubits, which poses limitations within the current Noisy Intermediate-Scale Quantum (NISQ) era. This work introduces a novel Adaptive Quantum Convolutional Neural Network (AQCNN) designed for efficient and scalable image classification within the constraints of NISQ devices. Addressing the limitations of existing QML approaches, particularly the high qubit requirements, AQCNN incorporates a resource-efficient quantum convolutional layer that performs localized quantum filtering using parameterized quantum circuits. A classical preprocessing layer encodes input features to reduce qubit load, followed by quantum embedding and hybrid quantum-classical layers that optimize feature extraction and classification performance. The model leverages an adaptive quantum convolution strategy, minimizing quantum gate depth and circuit complexity while preserving critical spatial hierarchies in image data. Evaluated on benchmark datasets, AQCNN achieved 95.88% accuracy on MNIST and 95.68% on FMNIST, outperforming comparable QML architectures. Additionally, the model supports scalable execution through parallel quantum circuit arrays, enabling practical deployment on current quantum hardware. This architecture demonstrates a significant advance in quantum-assisted image classification, balancing performance with qubit and gate efficiency. The integration of adaptive quantum convolution and hybrid processing not only enhances classification accuracy but also provides a viable path forward for deploying QML solutions under realistic hardware constraints.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.