Quantum-enhanced machine learning, encompassing both quantum algorithms and quantum-inspired classical methods such as tensor networks, offers promising tools for extracting structure from complex, high-dimensional data. In this work, we study the training dynamics of Matrix Product State (MPS) classifiers applied to three-class problems, using both fashion MNIST and hyperspectral satellite imagery as representative datasets. We investigate the phenomenon of grokking, where generalization emerges suddenly after memorization, by tracking entanglement entropy, local magnetization, and model performance across training sweeps. Additionally, we employ information-theory tools to gain deeper insights: transfer entropy is used to reveal causal dependencies between label-specific quantum masks, while O-information captures the shift from synergistic to redundant correlations among class outputs. Our results show that grokking in the fashion MNIST task coincides with a sharp entanglement transition and a peak in redundant information, whereas the overfitted hyperspectral model retains synergistic, disordered behavior. These findings highlight the relevance of high-order information dynamics in quantum-inspired learning and emphasize the distinct learning behaviors that emerge in multi-class classification, offering a principled framework to interpret generalization in quantum machine learning architectures.
Quantum-enhanced machine learning, encompassing both quantum algorithms and quantum-inspired classical methods such as tensor networks, offers promising tools for extracting structure from complex, high-dimensional data. In this work, we study the training dynamics of Matrix Product State (MPS) classifiers applied to three-class problems, using both fashion MNIST and hyperspectral satellite imagery as representative datasets. We investigate the phenomenon of grokking, where generalization emerges suddenly after memorization, by tracking entanglement entropy, local magnetization, and model performance across training sweeps. Additionally, we employ information-theory tools to gain deeper insights: transfer entropy is used to reveal causal dependencies between label-specific quantum masks, while O-information captures the shift from synergistic to redundant correlations among class outputs. Our results show that grokking in the fashion MNIST task coincides with a sharp entanglement transition and a peak in redundant information, whereas the overfitted hyperspectral model retains synergistic, disordered behavior. These findings highlight the relevance of high-order information dynamics in quantum-inspired learning and emphasize the distinct learning behaviors that emerge in multi-class classification, offering a principled framework to interpret generalization in quantum machine learning architectures.
Grokking is a intriguing phenomenon in machine learning where a neural network, after many training iterations with negligible improvement in generalization, suddenly achieves high accuracy on unseen data. By working in the quantum-inspired machine learning framework based on tensor networks, we numerically prove that grokking phenomenon can be related to an entanglement dynamical transition in the underlying quantum many-body systems, consisting in a one-dimensional lattice with each site hosting a qubit. Two datasets are considered as use case scenarios, namely fashion MNIST and gene expression communities of hepatocellular carcinoma. In both cases, we train Matrix Product State (MPS) to perform binary classification tasks, and we analyse the learning dynamics. We exploit measurement of qubits magnetization and correlation functions in the MPS network as a tool to identify meaningful and relevant gene subcommunities, verified by means of enrichment procedures.
A quantum-inspired tensor network framework for wildfire susceptibility classification in the Gargano region is introduced, leveraging AlphaEarth embeddings and Matrix Product State models. The approach combines scalable geospatial representations with an interpretable quantum mask, enabling both binary and multiclass classification of wildfire susceptibility. Beyond predictive performance, the study reveals a pronounced grokking transition in the binary case and provides a detailed analysis of inter-class confusion in the multiclass setting. By introducing level-resolved mixedness diagnostics based on reduced density matrices, we show that the MPS classifier naturally encodes a hierarchy of class distinguishability, with non-adjacent categories becoming more separable than neighboring ones. These results demonstrate that tensor network models not only achieve competitive classification accuracy but also offer a physically grounded framework to quantify and interpret class separability in complex environmental datasets.
The remarkable generalization capabilities of overparameterized neural networks remain one of the most profound mysteries in modern machine learning. Despite having sufficient capacity to simply memorize training data, these networks consistently achieve superior performance on unseen test examples. Traditional analyses focusing on weight distributions, loss landscapes, and optimization dynamics have provided partial insights but fail to capture the complex correlational structure that emerges between neurons during learning. This paper introduces a comprehensive quantum information framework for analyzing neural network representations, treating hidden layer activations as quantum states and characterizing them through density matrices, von Neumann entropy, purity, and related quantum information metrics. Through systematic experiments on synthetic classification tasks and algorithmic reasoning problems spanning over 3000 cumulative training epochs and 5000 Wang-Landau sampling iterations, we demonstrate that these quantum metrics reveal structure completely invisible to classical analysis. Our results show that stochastic gradient descent systematically reduces von Neumann entropy while increasing purity as networks learn, transitioning from highly mixed quantum states with entropy approximately 1.42 and purity approximately 0.29 to purer configurations with entropy approximately 1.19 and purity approximately 0.37 that achieve superior test accuracy of 99.5%. Using Wang-Landau sampling to explore the equilibrium entropy landscape, we discover that neural networks naturally favor high-entropy states with entropy approximately 1.42 and moderate accuracy of 62.0%, and that optimization actively purifies the quantum state to achieve task performance. On modular arithmetic tasks exhibiting the grokking phenomenon, we observe that von Neumann entropy increases dramatically from 1.55 to 3.29, representing a 112% increase, during an extended 2000-epoch memorization phase before any improvement in generalization, while purity decreases correspondingly from 0.444 to 0.042, a 90.5% decrease, approaching the maximally mixed limit of 0.01. This entropy accumulation phase appears necessary for eventual generalization, with the network building quantum correlations at a rate of approximately 0.00097 nats per epoch during the plateau. Weight entropy, in contrast, shows minimal change of only 11% increase over the same period. Correlation analysis reveals that von Neumann entropy and purity are almost perfectly anticorrelated with a correlation coefficient of 1 -0.99, while weight entropy shows weak correlation with quantum metrics with absolute correlation coefficients less than 0.3. The effective rank of the density matrix stabilizes near 2.7 to 3.0 for the 6-neuron classifier, indicating that the network effectively uses approximately three independent activation modes. These findings establish quantum information metrics as powerful tools for understanding neural network learning dynamics, provide the first quantitative evidence for the entropy accumulation interpretation of grokking, and suggest that neural networks undergo a quantum-inspired phase transition during learning. We discuss implications for generalization theory, optimization algorithm design, and the statistical physics of deep learning, and outline a comprehensive research agenda for extending these methods to larger architectures and more complex tasks.
Abstract A quantum-inspired tensor network framework for wildfire susceptibility classification in the Gargano region is introduced, leveraging AlphaEarth embeddings and Matrix Product State models. The approach combines scalable geospatial representations with an interpretable quantum mask, enabling both binary and multiclass classification of wildfire susceptibility. Beyond predictive performance, the study reveals a pronounced grokking transition in the binary case and provides a detailed analysis of inter-class confusion in the multiclass setting. By introducing level-resolved mixedness diagnostics based on reduced density matrices, we show that the MPS classifier naturally encodes a hierarchy of class distinguishability, with non-adjacent categories becoming more separable than neighboring ones. These results demonstrate that tensor network models not only achieve competitive classification accuracy but also offer a physically grounded framework to quantify and interpret class separability in complex environmental datasets.
The remarkable generalization capabilities of overparameterized neural networks remain one of the most profound mysteries in modern machine learning. Despite having sufficient capacity to simply memorize training data, these networks consistently achieve superior performance on unseen test examples. Traditional analyses focusing on weight distributions, loss landscapes, and optimization dynamics have provided partial insights but fail to capture the complex correlational structure that emerges between neurons during learning. This paper introduces a comprehensive quantum information framework for analyzing neural network representations, treating hidden layer activations as quantum states and characterizing them through density matrices, von Neumann entropy, purity, and related quantum information metrics. Through systematic experiments on synthetic classification tasks and algorithmic reasoning problems spanning over 3000 cumulative training epochs and 5000 Wang-Landau sampling iterations, we demonstrate that these quantum metrics reveal structure completely invisible to classical analysis. Our results show that stochastic gradient descent systematically reduces von Neumann entropy while increasing purity as networks learn, transitioning from highly mixed quantum states with entropy approximately 1.42 and purity approximately 0.29 to purer configurations with entropy approximately 1.19 and purity approximately 0.37 that achieve superior test accuracy of 99.5%. Using Wang-Landau sampling to explore the equilibrium entropy landscape, we discover that neural networks naturally favor high-entropy states with entropy approximately 1.42 and moderate accuracy of 62.0%, and that optimization actively purifies the quantum state to achieve task performance. On modular arithmetic tasks exhibiting the grokking phenomenon, we observe that von Neumann entropy increases dramatically from 1.55 to 3.29, representing a 112% increase, during an extended 2000-epoch memorization phase before any improvement in generalization, while purity decreases correspondingly from 0.444 to 0.042, a 90.5% decrease, approaching the maximally mixed limit of 0.01. This entropy accumulation phase appears necessary for eventual generalization, with the network building quantum correlations at a rate of approximately 0.00097 nats per epoch during the plateau. Weight entropy, in contrast, shows minimal change of only 11% increase over the same period. Correlation analysis reveals that von Neumann entropy and purity are almost perfectly anticorrelated with a correlation coefficient of 1-0.99, while weight entropy shows weak correlation with quantum metrics with absolute correlation coefficients less than 0.3. The effective rank of the density matrix stabilizes near 2.7 to 3.0 for the 6-neuron classifier, indicating that the network effectively uses approximately three independent activation modes. These findings establish quantum information metrics as powerful tools for understanding neural network learning dynamics, provide the first quantitative evidence for the entropy accumulation interpretation of grokking, and suggest that neural networks undergo a quantum-inspired phase transition during learning. We discuss implications for generalization theory, optimization algorithm design, and the statistical physics of deep learning, and outline a comprehensive research agenda for extending these methods to larger architectures and more complex tasks.
We introduce a machine learning model, the q-CNN model, sharing key features with convolutional neural networks and admitting a tensor network description. As examples, we apply q-CNN to the MNIST and Fashion MNIST classification tasks. We explain how the network associates a quantum state to each classification label, and study the entanglement structure of these network states. In both our experiments on the MNIST and Fashion-MNIST datasets, we observe a distinct increase in both the left/right as well as the up/down bipartition entanglement entropy during training as the network learns the fine features of the data. More generally, we observe a universal negative correlation between the value of the entanglement entropy and the value of the cost function, suggesting that the network needs to learn the entanglement structure in order the perform the task accurately. This supports the possibility of exploiting the entanglement structure as a guide to design the machine learning algorithm suitable for given tasks.