Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Conference on Electronic Information Engineering and Computer Technology (EIECT)

Structured Quantum Neural Networks Inspired by Multi-Head Attention Mechanisms

Chufan Lyu

Published Oct 24, 2025• 0 citations
Publisher
DOI
10.1109/eiect68017.2025.11331713

Related papers

Quantum-Enhanced Multi-Head Attention for Hybrid Classical-Quantum Neural Networks

Aayush Gauba

•Jan 1, 2025•0 citations

A quantum neural network with built-in self-attention mechanism

Shangshang Shi, Zhimin Wang, J. X. Li +6 more

Neurocomputing•Jan 26, 2026•0 citations

A Quantum Neural Network with Built-In Self-Attention Mechanism

Shangshang Shi, Zhimin Wang, Jiaxin Li +6 more

•Jan 1, 2025•0 citations

AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling

Siddhant Dutta, Nouhaila Innan, Sadok Ben Yahia +1 more

The growing computational demands of artificial intelligence (AI) in addressing climate change raise significant concerns about inefficiencies and environmental impact, as highlighted by the Jevons paradox. We propose an attention-enhanced quantum physics-informed neural networks model (AQ-PINNs) to tackle these challenges. This approach integrates quantum computing techniques into physics-informed neural networks (PINNs) for climate modeling, aiming to enhance predictive accuracy in fluid dynamics governed by the Navier-Stokes equations while reducing the computational burden and carbon footprint. By harnessing variational quantum multi-head self-attention mechanisms, our AQ-PINNs achieve a 51.51% reduction in model parameters compared to classical multi-head self-attention methods while maintaining comparable convergence and loss. It also employs quantum tensor networks to enhance representational capacity, which can lead to more efficient gradient computations and reduced susceptibility to barren plateaus. Our AQ-PINNs represent a crucial step towards more sustainable and effective climate modeling solutions.

arXiv:2409.01626•Sep 3, 2024•2 citations

QAMA: Scalable Quantum Annealing Multi-Head Attention Operator for Deep Learning

Du, Peng, Jinjing Shi, Wenxuan Wang +3 more

Attention mechanisms underpin modern deep learning, while the quadratic time and space complexity limit scalability for long sequences. To address this, Quantum Annealing Multi-Head Attention (QAMA) is proposed, a novel drop-in operator that reformulates attention as an energy-based Hamiltonian optimization problem. In this framework, token interactions are encoded into binary quadratic terms, and quantum annealing is employed to search for low-energy configurations that correspond to effective attention patterns. Unlike classical sparse or approximate attention methods that rely on hand-crafted heuristics, QAMA allows sparsity structures to emerge naturally from the optimization process. Theoretically, computational complexity is analysed through single-spin flip dynamics, providing time to solution runtime bounds that depend on the spectral properties of the annealing Hamiltonian. Empirically, evaluation on both natural language and vision benchmarks shows that, across tasks, accuracy deviates by at most 2.7 points from standard multi-head attention, while requiring only linear qubits in sequence length. Visualizations further reveal that the Hamiltonian penalty terms induce meaningful and interpretable sparsity across heads. Finally, deployment on a coherent Ising machine validates the feasibility of running QAMA on real quantum hardware, showing tangible inference-time reductions compared with classical implementations. These results highlight QAMA as a pioneering and scalable step toward integrating quantum optimization devices into deep neural architectures, providing a seamlessly integrable and hardware-compatible alternative to conventional attention mechanisms. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

arXiv:2504.11083•Apr 15, 2025•0 citations

Quantum Neural Networks for a Supply Chain Logistics Application

Randall Correll, Sean J. Weinberg, Fabio Sanches +2 more

Abstract Problem instances of a size suitable for practical applications are not likely to be addressed during the noisy intermediate‐scale quantum (NISQ) period with (almost) pure quantum algorithms. Hybrid classical‐quantum algorithms have potential, however, to achieve good performance on much larger problem instances. One such hybrid algorithm on a problem of substantial importance: vehicle routing for supply chain logistics with multiple trucks and complex demand structure is investigated. Reinforcement learning with neural networks with embedded quantum circuits is used. In such neural networks, projecting high‐dimensional feature vectors down to smaller vectors is necessary to accommodate restrictions on the number of qubits of NISQ hardware. However, a multi‐head attention mechanism is used where, even in classical machine learning, such projections are natural and desirable. Data from the truck routing logistics of a company in the automotive sector is considered, and the methodology is applied by decomposing into small teams of trucks and results are found comparable to human truck assignment.

arXiv:2212.00576•Advanced Quantum Technologies•May 18, 2023•20 citations
The growing computational demands of artificial intelligence (AI) in addressing climate change raise significant concerns about inefficiencies and environmental impact, as highlighted by the Jevons paradox. We propose an attention-enhanced quantum physics-informed neural networks model (AQ-PINNs) to tackle these challenges. This approach integrates quantum computing techniques into physics-informed neural networks (PINNs) for climate modeling, aiming to enhance predictive accuracy in fluid dynamics governed by the Navier-Stokes equations while reducing the computational burden and carbon footprint. By harnessing variational quantum multi-head self-attention mechanisms, our AQ-PINNs achieve a 51.51% reduction in model parameters compared to classical multi-head self-attention methods while maintaining comparable convergence and loss. It also employs quantum tensor networks to enhance representational capacity, which can lead to more efficient gradient computations and reduced susceptibility to barren plateaus. Our AQ-PINNs represent a crucial step towards more sustainable and effective climate modeling solutions.
Attention mechanisms underpin modern deep learning, while the quadratic time and space complexity limit scalability for long sequences. To address this, Quantum Annealing Multi-Head Attention (QAMA) is proposed, a novel drop-in operator that reformulates attention as an energy-based Hamiltonian optimization problem. In this framework, token interactions are encoded into binary quadratic terms, and quantum annealing is employed to search for low-energy configurations that correspond to effective attention patterns. Unlike classical sparse or approximate attention methods that rely on hand-crafted heuristics, QAMA allows sparsity structures to emerge naturally from the optimization process. Theoretically, computational complexity is analysed through single-spin flip dynamics, providing time to solution runtime bounds that depend on the spectral properties of the annealing Hamiltonian. Empirically, evaluation on both natural language and vision benchmarks shows that, across tasks, accuracy deviates by at most 2.7 points from standard multi-head attention, while requiring only linear qubits in sequence length. Visualizations further reveal that the Hamiltonian penalty terms induce meaningful and interpretable sparsity across heads. Finally, deployment on a coherent Ising machine validates the feasibility of running QAMA on real quantum hardware, showing tangible inference-time reductions compared with classical implementations. These results highlight QAMA as a pioneering and scalable step toward integrating quantum optimization devices into deep neural architectures, providing a seamlessly integrable and hardware-compatible alternative to conventional attention mechanisms. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
Abstract Problem instances of a size suitable for practical applications are not likely to be addressed during the noisy intermediate‐scale quantum (NISQ) period with (almost) pure quantum algorithms. Hybrid classical‐quantum algorithms have potential, however, to achieve good performance on much larger problem instances. One such hybrid algorithm on a problem of substantial importance: vehicle routing for supply chain logistics with multiple trucks and complex demand structure is investigated. Reinforcement learning with neural networks with embedded quantum circuits is used. In such neural networks, projecting high‐dimensional feature vectors down to smaller vectors is necessary to accommodate restrictions on the number of qubits of NISQ hardware. However, a multi‐head attention mechanism is used where, even in classical machine learning, such projections are natural and desirable. Data from the truck routing logistics of a company in the automotive sector is considered, and the methodology is applied by decomposing into small teams of trucks and results are found comparable to human truck assignment.