Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.
Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.
The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).
The integration of quantum computing into classical machine learning architectures has emerged as a promising approach to enhance model efficiency and computational capacity. In this work, we introduce the Quantum Kernel-Based Long Short-Term Memory (QK-LSTM) network, which utilizes quantum kernel functions within the classical LSTM framework to capture complex, non-linear patterns in sequential data. By embedding input data into a high-dimensional quantum feature space, the QK-LSTM model reduces the reliance on large parameter sets, achieving effective compression while maintaining accuracy in sequence modeling tasks. This quantum-enhanced architecture demonstrates efficient convergence, robust loss minimization, and model compactness, making it suitable for deployment in edge computing environments and resource-limited quantum devices (especially in the NISQ era). Benchmark comparisons reveal that QK-LSTM achieves performance on par with classical LSTM models, yet with fewer parameters, underscoring its potential to advance quantum machine learning applications in natural language processing and other domains requiring efficient temporal data processing.
QCAAPatchTF is a quantum attention network integrated with an advanced patch-based transformer, designed for multivariate time series forecasting, classification, and anomaly detection. Leveraging quantum superpositions, entanglement, and variational quantum eigensolver principles, the model introduces a quantum-classical hybrid self-attention mechanism to capture multivariate correlations across time points. For multivariate long-term time series, the quantum self-attention mechanism can reduce computational complexity while maintaining temporal relationships. It then applies the quantum-classical hybrid self-attention mechanism alongside a feed-forward network in the encoder stage of the advanced patch-based transformer. While the feed-forward network learns nonlinear representations for each variable frame, the quantum self-attention mechanism processes individual series to enhance multivariate relationships. The advanced patch-based transformer computes the optimized patch length by dividing the sequence length into a fixed number of patches instead of using an arbitrary set of values. The stride is then set to half of the patch length to ensure efficient overlapping representations while maintaining temporal continuity. QCAAPatchTF achieves state-of-the-art performance in both long-term and short-term forecasting, classification, and anomaly detection tasks, demonstrating state-of-the-art accuracy and efficiency on complex real-world datasets.
Time-series forecasting is vital across sectors such as finance, energy, and environmental science, where accurate future predictions drive informed decision-making. Traditional statistical methods and deep learning models like LSTM and Transformers, while effective, often struggle with high-dimensional data and long-term dependencies. In this paper, we propose a hybrid quantum-classical model that utilizes Parameterized Quantum Circuits (PQCs) to capture complex temporal patterns in multivariate time-series data. Our architecture integrates a classical encoder, a quantum variational processor, and a classical decoder, trained end-to-end using gradient-based optimization. Experiments conducted on benchmark datasets—electricity load, stock prices, and weather forecasting—demonstrate that the hybrid model consistently outperforms classical baselines in terms of Mean Squared Error (MSE), Mean Absolute Error (MAE), and R² score. Robustness analysis confirms the model’s resilience to moderate quantum noise, while an ablation study highlights the performance gain from quantum components. This work showcases the potential of hybrid quantum-classical approaches on Noisy Intermediate-Scale Quantum (NISQ) devices and lays a foundation for scalable, real-world quantum machine learning applications.
Abstract In this paper, we address the challenge of multivariate time-series forecasting through quantum-enhanced machine learning architectures. We propose adaptation strategies that extend variational quantum circuit models (VQC), traditionally constrained to univariate data, toward the multivariate setting, exploring both purely quantum and hybrid quantum-classical formulations. First, we extend and benchmark several VQC-based and hybrid models to systematically evaluate their ability to capture cross-variable dependencies. Building on these foundations, we introduce the iQTransformer, a novel quantum transformer architecture that integrates a quantum self-attention mechanism within the iTransformer framework, enabling a quantum-native representation of inter-variable relationships. Finally, we present a comprehensive empirical evaluation on both synthetic and real-world datasets, demonstrating that quantum-enhanced models can achieve competitive or superior forecasting accuracy with fewer trainable parameters and faster convergence than state-of-the-art classical and quantum baselines. These findings highlight the potential of quantum-enhanced architectures as efficient and scalable tools for advancing multivariate time-series forecasting.
This study explores quantum and classical hybrid architectures for financial time-series fore casting, focusing on Quantum Long Short-Term Memory (QLSTM) networks and Quantum Reservoir Computing (QRC), using univariate and multivariate lag structures on real financial data. We assess how lag embeddings affect predictive accuracy and robustness. Data are en coded into quantum states via amplitude encoding, enabling efficient representation of normalized lagged observations under realistic qubit constraints. The recurrent dynamics of QLSTM and the reservoir of QRC are implemented as parameterized quantum circuits, while classical optimizers train the readout and, where applicable, variational circuit parameters. We benchmark quantum models against classical LSTM and reservoir computing using common error like metrics. Our results show that, with suitable lag selection and amplitude encoding, quantum-enhanced archi tectures match classical baselines in univariate settings and can modestly outperform them in multivariate regimes with correlated inputs, where expressive encodings are most beneficial.