Progress in the realisation of reliable large-scale quantum computers has motivated research into the design of quantum machine learning models. We present Quixer: a novel quantum transformer model which utilises the Linear Combination of Unitaries and Quantum Singular Value Transform primitives as building blocks. Quixer operates by preparing a superposition of tokens and applying a trainable non-linear transformation to this mix. We present the first results for a quantum transformer model applied to a practical language modelling task, obtaining results competitive with an equivalent classical baseline. In addition, we include resource estimates for evaluating the model on quantum hardware, and provide an open-source implementation for classical simulation. We conclude by highlighting the generality of Quixer, showing that its parameterised components can be substituted with fixed structures to yield new classes of quantum transformers.
Progress in the realisation of reliable large-scale quantum computers has motivated research into the design of quantum machine learning models. We present Quixer: a novel quantum transformer model which utilises the Linear Combination of Unitaries and Quantum Singular Value Transform primitives as building blocks. Quixer operates by preparing a superposition of tokens and applying a trainable non-linear transformation to this mix. We present the first results for a quantum transformer model applied to a practical language modelling task, obtaining results competitive with an equivalent classical baseline. In addition, we include resource estimates for evaluating the model on quantum hardware, and provide an open-source implementation for classical simulation. We conclude by highlighting the generality of Quixer, showing that its parameterised components can be substituted with fixed structures to yield new classes of quantum transformers.
Recent advancements in the viability of large-scale quantum computers have significantly stimulated research into quantum machine learning models and their architecture. Specifically, quantum generative artificial intelligence may be a field that can benefit a lot from the expected power of future quantum processing hardware. Recently, Quantinuum’s research team presented Quixer: a new quantum transformer model based on using the Linear Combination of Unitaries and Quantum Singular Value Transform primitives to mimic a classical transformer’s attention mechanism. They presented the results obtained in a practical language modelling task, which were not far from those of an equivalent classical baseline. In this work, we go trough a deep explanation of the state of the art of quantum generative AI. Additionally, we expand the experiment in Quantinuum’s paper by implementing a wide variety of different hyperparameter configurations and examining new metrics and their results. We conclude by remarking the potential that Quantum GenAI has but also the large research effort that is needed to achieve that potential.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
We propose a compact NISQ-compatible quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-token interface of a classical transformer, but replaces attention and feed-forward sublayers with variational quantum encoder blocks, connector circuits, decoder blocks and a direct two-qubit measurement readout. Token contexts are angle-encoded into small quantum registers, processed by parallel variational heads and encoder integration circuits and conditioned through decoder ancillae to produce a distribution over a four-token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical transformer baseline. The quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the quantum models are sensitive to initialization. The contribution is therefore not a claim of quantum advantage, but a concrete architecture and evaluation of transformer-inspired QNLP sequence modelling under near-term quantum constraints.
Quantum Transformers integrate the representational power of classical Transformers with the computational advantages of quantum computing. Since 2022, research in this area has rapidly expanded, giving rise to diverse technical paradigms and early applications. To address the growing need for consolidation, this paper presents the first comprehensive, systematic, and in-depth survey of quantum Transformer models. First, we delineate the research scope, focusing on improving Transformer parts with quantum methods, and introduce foundational concepts in classical Transformers and quantum machine learning. Then we organize existing studies into two main paradigms: PQC-based and QLA-based, with PQC-based paradigm further divided into QKV-only Quantum Mapping, Quantum Pairwise Attention, Quantum Holistic Attention. and Quantum-Assisted Optimization, analyzing their core mechanisms and architectural traits. We also summarize empirical results that demonstrate preliminary quantum advantages, especially on small-scale tasks or resource-constrained settings. Following this, we examine key technical challenges, such as complexity-resource trade-offs, scalability and generalization limitations, and trainability issues including barren plateaus, and provide potential solutions, including quantumizing classical transformer variants with lower complexity, hybrid designs, and improved optimization strategies. Finally, we propose several promising future directions, e.g., scaling quantum modules into large architectures, applying quantum Transformers to domains with inherently quantum data (e.g., physics, chemistry), and developing theory-driven designs grounded in quantum information science. This survey will help researchers and practitioners quickly grasp the overall landscape of current quantum Transformer research and promote future developments in this emerging field.
Variational quantum learning is traditionally constrained to unitary dynamics, often treating quantum channels as detrimental noise. In this work, we reformulate the quantum channels as trainable computational primitives and establish a non-unitary quantum machine learning framework grounded in open-system dynamics. We demonstrate that the outputs of channel-enhanced quantum models form a structured superposition of multiple functional components. Each component is governed by an effective observable whose spectrum can be adaptively modulated during training, a significant departure from the spectral invariance in unitary transformations. Moreover, the proposed framework generalizes conventional unitary quantum models by retaining them as a special case while introducing additional non-unitary degrees of freedom. Furthermore, we reveal that trainable quantum channels enrich the optimization geometry through ensemble-averaged gradient and additional optimization directions induced by the Kraus operators. Extensive experiments on classification tasks using trainable amplitude-damping and phase-damping channels confirm enhanced optimization dynamics and predictive performance. In addition, we experimentally validate the proposed framework through hardware inference using ten-qubit quantum models implemented on the superconducting quantum processor, confirming its practical feasibility and hardware compatibility. Our work provides a principled approach for leveraging quantum channels as trainable resources and advances the design of high-performance quantum learning architectures.
Discrete circular convolution over Z/NZ is a linear operator and can be implemented on quantum hardware within the linear-combination-of-unitaries (LCU) framework. In this work, we make this connection explicit through an asymmetric-LCU formulation: circular convolution is the postselected block of a circuit whose controlled-shift unitary is modular addition on computational-basis states. The asymmetry is essential: fixing the postselection state to the uniform state ∣u⟩ while supplying the kernel state ∣b⟩ as the input ancilla naturally preserves the complex coefficients bi within the block, whereas a symmetric overlap would yield ∣bi∣2 weights and erase their phases. Accordingly, when ∣a⟩ and ∣b⟩ are supplied by upstream quantum routines, the convolution subroutine requires only the fixed uncompute PREPu†, completely avoiding the need for a kernel-dependent inverse preparation PREPb†. We then introduce a reversal matrix Jn=X⊗n and define reflected shifts Li,n=Li,nJn. This symmetrization yields a recursive operator algebra for convolution that is natively compatible with LCU/block-encoding workflows. The resulting symmetrized operator differs from circular convolution only by one known input-side Jn layer. Crucially, for real-valued kernels, the resulting operator Hn(b)=∑ibiLi,n is Hermitian, providing a direct Hermitian interface for quantum singular value transformation (QSVT) and related spectral transformations. Based on this framework, we present a transparent recursive construction, paired with an exactly equivalent optimized bitwise compilation of the same SELECT block. Finally, we evaluate implementation trade-offs and resource scaling under explicit cost-model conventions.