Q-OmniNet: Quantum-Inspired Hybrid TensorNetworks with Quantum Kernel Estimation for Polysemy-Aware Multimodal Affective Analysis of Code-Mixed Hindi–English Text
Abstract The rapid growth of Hinglish communication onsocial media has introduced complex linguistic, cultural, andmultimodal patterns that conventional affective computing sys-tems struggle to handle. While deep learning and transformer-based models perform well on monolingual and grammaticallyconsistent text, they are limited in code-mixed Hinglish settings.These models rely on deterministic, single-meaning embeddingsthat inadequately represent polysemy, fail to capture context-dependent polarity shifts across Hindi–English transitions, andtreat multimodal signals—such as text, emojis, and visuals—asweakly coupled rather than jointly dependent. Consequently,emotional nuance, irony, and ambiguity common in real-worldHinglish interactions are often misinterpreted.To overcome these limitations, this work proposes Q-OmniNet,a quantum-inspired hybrid tensor network that models affectivemeaning as contextual superposition and captures non-separable(entangled) relationships across modalities. The framework inte-grates (i) a tensor-network-based contextual encoder to simulatemultimodal entanglement, (ii) complex-valued embeddings to en-code semantic magnitude and contextual phase, and (iii) quantumkernel estimation to model higher-order affective similarity viaquantum-state overlap. Together, these components provide aricher representation of Hinglish emotion dynamics than classicalor transformer-based approaches.Experimental results on real-world multimodal Hinglishdatasets show substantial improvements. Q-OmniNet achievesaccuracies of 0.89 for emotion classification, 0.93 for sentimentanalysis, and 0.97 for sarcasm detection, outperforming tradi-tional baselines by 15–25% and transformer models by 8–14%.The quantum-hybrid variant demonstrates superior robustnessunder noisy and ambiguous conditions, achieving an averageaccuracy of 0.9275 compared to 0.7617 for conventional models.These findings highlight the effectiveness of quantum-inspiredarchitectures for context-sensitive, multilingual, and multimodalaffective understanding.
Abstract The rapid growth of Hinglish communication onsocial media has introduced complex linguistic, cultural, andmultimodal patterns that conventional affective computing sys-tems struggle to handle. While deep learning and transformer-based models perform well on monolingual and grammaticallyconsistent text, they are limited in code-mixed Hinglish settings.These models rely on deterministic, single-meaning embeddingsthat inadequately represent polysemy, fail to capture context-dependent polarity shifts across Hindi–English transitions, andtreat multimodal signals—such as text, emojis, and visuals—asweakly coupled rather than jointly dependent. Consequently,emotional nuance, irony, and ambiguity common in real-worldHinglish interactions are often misinterpreted.To overcome these limitations, this work proposes Q-OmniNet,a quantum-inspired hybrid tensor network that models affectivemeaning as contextual superposition and captures non-separable(entangled) relationships across modalities. The framework inte-grates (i) a tensor-network-based contextual encoder to simulatemultimodal entanglement, (ii) complex-valued embeddings to en-code semantic magnitude and contextual phase, and (iii) quantumkernel estimation to model higher-order affective similarity viaquantum-state overlap. Together, these components provide aricher representation of Hinglish emotion dynamics than classicalor transformer-based approaches.Experimental results on real-world multimodal Hinglishdatasets show substantial improvements. Q-OmniNet achievesaccuracies of 0.89 for emotion classification, 0.93 for sentimentanalysis, and 0.97 for sarcasm detection, outperforming tradi-tional baselines by 15–25% and transformer models by 8–14%.The quantum-hybrid variant demonstrates superior robustnessunder noisy and ambiguous conditions, achieving an averageaccuracy of 0.9275 compared to 0.7617 for conventional models.These findings highlight the effectiveness of quantum-inspiredarchitectures for context-sensitive, multilingual, and multimodalaffective understanding.
Sarcasm detection in unimodal or multimodal setting is a very complex task. Sarcasm, emotion, and sentiment are related to each other, and hence any multitask model could be an effective way to leverage the interdependence among these tasks. In order to better represent these clandestine associations, we avoid solely relying on traditional machine learning methods to encode the relationships between the modalities. In this article, we propose a hybrid quantum model that banks upon the low computational complexity and robust representational power of a variational quantum circuit (VQC) and the tried and tested dense neural network to tackle sentiment, emotion, and sarcasm classification simultaneously. We empirically establish that the quantum properties like superposition, entanglement, and interference will better capture and replicate not only the cross-modal interactions between text, acoustics, and visuals but also the correlations between the three responses. We consider the extended MUStARD dataset to evaluate our proposed hybrid model. The results show that our proposed hybrid quantum framework yields more promising results for the primary task of sarcasm detection with the help of the two secondary classification tasks,viz.sentiment and emotion.
Sentiment, emotion, and sarcasm analysis in multimodal dialogues is crucial for understanding the underlying intentions and attitudes expressed by individuals. Traditional methods often struggle to capture the full intensity of these polarities, leading to less accurate results. To address this limitation, we propose a quantum-inspired approach leveraging Quantum Neural Networks (QNNs) for enhanced classification and intensity analysis. A key component of our method is the Variational Quantum Eigensolver (VQE), a hybrid quantum-classical algorithm that optimizes the parameters of the QNN by minimizing the eigenvalues of a Hamiltonian system. This optimization enables the network to learn complex relationships in multimodal data more effectively. Our approach surpasses state-of-the-art methods, achieving up to 7.5 % higher accuracy and 6.8 % greater precision. Experiments on benchmark datasets such as MUStARD, Memotion, CMU-MOSEI, and MELD demonstrate its effectiveness, with an F1-score of 87.3 % on CMU-MOSEI. This method is particularly beneficial in domains like social media, customer support, and entertainment, where both verbal and non-verbal cues play a critical role in accurate sentiment analysis .
Sentiment analysis (SA) is a critical component of Natural Language Processing (NLP), particularly for automatic text classification. However, previous approaches have shortcomings in capturing nuances such as negation, word pairings, and contextual understanding, particularly in languages like Hindi. To solve these issues, this research offers a novel Quantum Neural Network (QNN) technique designed specifically for sentiment analysis in Hindi text data, which employs quantum computing concepts to capture linguistic nuances and context better. This study uses a Quantum Variational Auto Encoder to encode classical data into a quantum form, capturing diverse sentiments like sarcasm and colloquial expressions. A Tuned Quantum Convolutional Neural Network architecture is introduced to capture complex linguistic syntax. A novel Sequential-based hyperband optimization technique is used to enhance model performance. The hybrid approach significantly improves accuracy and efficiency in handling quantum data, contributing to the advancement of SA, particularly in Hindi Movie Reviews. The findings demonstrate that the proposed strategy performs best with accuracy 97.64 %, precision 85.93%, recall 99.17%, F-1 score 92.20%, than other accepted strategies.
A long-standing goal in the field of Artificial Intelligence (AI) is to develop systems that can perceive and understand human multimodal language. This requires both the consideration of context in the form of surrounding utterances in a conversation, i.e., context modelling, as well as the impact of different modalities (e.g., linguistic, visual acoustic), i.e., multimodal fusion. In the last few years, significant strides have been made towards the interpretation of human language due to simultaneous advancement in deep learning, data gathering and computing infrastructure. AI models have been investigated to either model interactions across distinct modalities, i.e., linguistic, visual and acoustic, or model interactions across parties in a conversation, achieving unprecedented levels of performance. However, AI models are often designed with only performance as their design target, leaving aside other essential factors such as transparency, interpretability, and how humans understand and reason about cognitive states. In line with this observation, in this dissertation, we develop quantum probabilistic neural models and techniques that allow us to capture rational and irrational cognitive biases, without requiring a priori understanding and identification of them. First, we present a comprehensive empirical comparison of state-of-the-art (SOTA) modality fusion strategies for video sentiment analysis. The findings provide us helpful insights into the development of more effective modality fusion models incorporating quantum-inspired components. Second, we introduce an end-to-end complex-valued neural model for video sentiment analysis, simulating quantum procedural steps, outside of physics, into the neural network modelling paradigm. Third, we investigate non-classical correlations across different modalities. In particular, we describe a methodology to model interactions between image and text for an information retrieval scenario. The results provide us with theoretical and empirical insights to develop a transparent end-to-end probabilistic neural model for video emotion detection in conversations, capturing non-classical correlations across distinct modalities. Fourth, we introduce a theoretical framework to model user's cognitive states underlying their multimodal decision perspectives, and propose a methodology to capture interference of modalities in decision making. Overall, we show that our models advance the SOTA on various affective analysis tasks, achieve high transparency due to the mapping to quantum physics meanings, and improve post-hoc interpretability, unearthing useful and explainable knowledge about cross-modal interactions.
Speech emotion recognition (SER) remains fragile in real-world conditions because emotional cues are subtle, speaker-dependent, and easily confounded by recording variability, while high-performing deep models typically rely on large and carefully curated training sets. Quantum machine learning offers an alternative way to introduce nonlinear correlation modeling with compact modules, yet existing quantum SER studies remain limited, and the impact of circuit structure is not well understood. This paper presents HQTN-SER, a hybrid quantum-classical framework that investigates how quantum tensor network connectivity can support SER under small-qubit settings. HQTN-SER introduces (i) an MPS-inspired quantum tensor network module that enforces structured interactions to model correlations in speech representations with a small number of trainable parameters, and (ii) a fusion strategy that combines quantum measurement features with a learned classical latent embedding for end-to-end emotion classification. We evaluate HQTN-SER on three public benchmarks (RAVDESS, SAVEE, and MDER) under a unified preprocessing and training protocol. The proposed model achieves test accuracies of 80.09\% on RAVDESS, 78.12\% on SAVEE, and 73.51\% on MDER, with corresponding macro-F1 scores of 0.8012, 0.7826, and 0.7351, respectively. The model also exhibits stable convergence while using only three or four qubits, suggesting that MPS-inspired circuit structure can be an effective and hardware-aware design choice for quantum-assisted SER. These results provide a compact experimental baseline for studying structured quantum modules in affective computing.
Quantum computing is typically associated with domains such as cybersecurity, drug discovery, finance, and large-scale algorithmic optimization. While these applications drive technological progress, they remain largely disconnected from the emotional and psychological realities that shape everyday human experience. To enhance personal relevance, quantum systems can instead be applied to support daily emotional well-being, guiding individuals through complex affective states and fostering more harmonious, enriched lives. This study proposes an AI framework that leverages quantum optimization to interpret and classify emotional expressions from text, expanding computational capacity to understand aspects of human consciousness and contextual affect in synchrony with lived experience. By embedding quantum-enhanced affective computation into daily interactions, the model moves beyond predictive analytics to form a dynamic interface with emotional cognition, with potential applications in counselling, self-awareness, and therapeutic support across clinical and organizational settings. Using the HappyDB dataset, containing over 100,000 self-reported happy moments annotated with emotional categories, we demonstrate that conventional AI approaches struggle to capture subtle context and interdependent emotional cues. Our method addresses this gap by integrating classical neural architectures with a quantum variational layer, enabling synchronous evaluation of complex semantic and affective relationships in text. The hybrid model transforms text into semantic embeddings, processes them through sequential LSTM layers, and optimizes latent emotional representations via quantum-inspired techniques. This allows the system to emulate human-like context awareness and achieve enhanced emotion classification beyond the capabilities of classical methods. By applying quantum computing to daily-life emotional cognition at scale, this work reframes quantum algorithms as tools for enriching human experience. Beyond mental health assessment and digital therapeutics, it establishes a new frontier for human-centered quantum intelligence, demonstrating that complex psychological and semantic interactions can be optimized to advance AI-assisted well-being.