Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the $\mathcal{O}(N^2D)$ attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3\% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.
Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the O(N2D) attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.
Abstract We introduce Quantum Kernel Transformer (QKT), a hybrid quantumclassical architecture that inserts a trainable parameterized quantum circuit between a classical backbone and an attention-based classifier head. Backbone features are projected to a small qubit input, encoded by a quantum feature map, transformed by a shallow variational ansatz, and decoded by attention over the resulting basis-state probability distribution. This keeps highdimensional representation learning on the classical side while using the quantum neural network as a non-linear embedding whose qubit count is independent of the raw input dimension. We evaluate QKT on five classification benchmarks spanning text (AG News), tabular biomedical data (Breast Cancer Wisconsin), bioacoustic spectrograms (BirdCLEF), images (CIFAR-10), and high-energy physics (SUSY). Against classical, classical-attention, and linear hybrid-QNN baselines, QKT is competitive on AG News and BCW, obtains the highest mean accuracy on BirdCLEF, and reduces the severe instability observed for the linear hybrid-QNN baseline on SUSY. It underperforms classical baselines on CIFAR-10, identifying an important failure mode for image-domain features. The evidence does not establish a universal quantum advantage; instead, it identifies where a trainable quantum-kernel bottleneck with distribution attention is robust under a reproducible workstation-scale protocol. Code, configuration files, generated tables, figures, and LaTeX sources are released to support independent reproduction.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
This working paper introduces the Quantum Interference Transformer (QIT), a sequence model in which token relationships are resolved by constructive and destructive interference in a parameterised quantum circuit, rather than by classical dot-product attention. We present QIT-0, a minimal PennyLane/PyTorch prototype. Tokens are encoded as rotation angles, cross-token correlations are seeded by a ring CNOT ladder, a learned strongly-entangling unitary acts as the attention operator, and Pauli-Z expectation values are decoded by a linear layer. On a 4-bit parity memorisation benchmark (16 inputs; 5 random seeds), QIT-0 (78 parameters) reaches 100% accuracy in 6.2 ± 1.7 epochs. Classical baselines with more parameters converge more slowly or fail within the epoch budget. Additional experiments include ablations, a first-token negative control, and related structured tasks. This is an independent technical report, not a peer-reviewed publication. We do not claim general quantum advantage for machine learning. Code: https://github.com/MihailStancescu/qit
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
The integration of quantum computing principles with classical deep learning architectures has emerged as a promising direction for advancing image recognition systems beyond the limitations of purely classical convolutional neural networks (CNNs). This study proposes and analyzes a hybrid quantum–classical deep learning framework based on quanvolutional neural network (QNN) principles, where quantum circuits are embedded within convolutional feature extraction layers to enhance representation capability. The research investigates how quantum-enhanced filters, particularly quanvolutional layers, contribute to improved feature diversity, non-linear transformation capacity, and classification accuracy in image recognition tasks. Building upon established convolutional architectures and quantum machine learning theories, the proposed framework integrates quantum random circuit encoding with classical deep neural networks to evaluate performance gains in comparison to conventional CNN models. The study further incorporates theoretical insights from hybrid quantum-classical learning systems, emphasizing the role of parameterized quantum circuits in feature mapping and optimization stability. The methodology is grounded in NISQ-era quantum computing constraints and leverages hybrid optimization strategies to ensure computational feasibility. Experimental evaluation demonstrates that quanvolution-based models can provide improved robustness in feature extraction, particularly in high-dimensional image spaces, while maintaining competitive computational efficiency. However, scalability limitations and noise sensitivity in quantum circuits remain key challenges. Overall, this work contributes to the growing field of quantum-enhanced machine learning by providing a structured performance analysis of hybrid quanvolutional architectures and highlighting their potential role in next-generation intelligent visual recognition systems.
The recent exploding growth in size of state-of-the-art machine learning models highlights a well-known issue where exponential parameter growth, which has grown to trillions as in the case of the Generative Pre-trained Transformer (GPT), leads to training time and memory requirements which limit their advancement in the near term. The predominant models use the so-called transformer network and have a large field of applicability, including predicting text and images, classification, and even predicting solutions to the dynamics of physical systems. Here we present a variational quantum circuit architecture named Self-Attention Sequential Quantum Transformer Channel (SASQuaTCh), which builds networks of qubits that perform analogous operations of the transformer network, namely the keystone self-attention operation, and leads to an exponential improvement in parameter complexity and run-time complexity over its classical counterpart. Our approach leverages recent insights from kernel-based operator learning in the context of predicting spatiotemporal systems to represent deep layers of a vision transformer network using simple gate operations and a set of multi-dimensional quantum Fourier transforms. To validate our approach, we consider image classification tasks in simulation and with hardware, where with only 9 qubits and a handful of parameters we are able to simultaneously embed and classify a grayscale image of handwritten digits with high accuracy.