arXiv:2211.07531

On the capacity of neural networks

Leonardo Cruciani

Published Nov 2, 2022• 0 citations

Abstract

The aim of this thesis is to compare the capacity of different models of neural networks. We start by analysing the problem solving capacity of a single perceptron using a simple combinatorial argument. After some observations on the storage capacity of a basic network, known as an associative memory, we introduce a powerful statistical mechanical approach to calculate its capacity in the training rule-dependent Hopfield model. With the aim of finding a more general definition that can be applied even to quantum neural nets, we then follow Gardner's work, which let us get rid of the dependency on the training rule, and comment the results obtained by Lewenstein et al. by applying Gardner's methods on a recently proposed quantum perceptron model.

DOI: 10.48550/arxiv.2211.07531

Related papers

Optimal storage capacity of quantum Hopfield neural networks

Lukas Bödeker, Eliana Fiorelli, Markus Müller

Quantum neural networks form one pillar of the emergent field of quantum machine learning. Here quantum generalizations of classical networks realizing associative memories---capable of retrieving patterns, or memories, from corrupted initial states---have been proposed. It is a challenging open problem to analyze quantum associative memories with an extensive number of patterns and to determine the maximal number of patterns the quantum networks can reliably store, i.e., their storage capacity. In this work, we propose and explore a general method for evaluating the maximal storage capacity of quantum neural network models. By generalizing what is known as Gardner's approach in the classical realm, we exploit the theory of classical spin glasses for deriving the optimal storage capacity of quantum networks with quenched pattern variables. As an example, we apply our method to an open-system quantum associative memory formed of interacting spin-1/2 particles realizing coupled artificial neurons. The system undergoes a Markovian time evolution resulting from a dissipative retrieval dynamics that competes with a coherent quantum dynamics. We map out the nonequilibrium phase diagram and study the effect of temperature and Hamiltonian dynamics on the storage capacity. Our method opens an avenue for a systematic characterization of the storage capacity of quantum associative memories.

arXiv:2210.07894Physical Review ResearchMay 1, 202310 citations

Storage properties of a quantum perceptron

Aikaterini Gratsea, Gratsea, Valentin Kasper +1 more

Driven by growing computational power and algorithmic developments, machine learning methods have become valuable tools for analyzing vast amounts of data. Simultaneously, the fast technological progress of quantum information processing suggests employing quantum hardware for machine learning purposes. Recent works discuss different architectures of quantum perceptrons, but the abilities of such quantum devices remain debated. Here, we investigate the storage capacity of a particular quantum perceptron architecture by using statistical mechanics techniques and connect our analysis to the theory of classical spin glasses. We focus on a specific quantum perceptron model and explore its storage properties in the limit of a large number of inputs. Finally, we comment on using statistical physics techniques for further studies of neural networks.

arXiv:2111.08414Bulletin of the American Physical SocietyNov 16, 20213 citations

Storage Capacity Evaluation of the Quantum Perceptron using the Replica Method

Mitsuru Urushibata, Masayuki Ohzeki

We investigate a quantum perceptron implemented on a quantum circuit using a repeat until method. We evaluate this from the perspective of capacity, one of the performance evaluation measures for perceptions. We assess a Gardner volume, defined as a volume of coefficients of the perceptron that can correctly classify given training examples using the replica method. The model is defined on the quantum circuit. Nevertheless, it is straightforward to assess the capacity using the replica method, which is a standard method in classical statistical mechanics. The reason why we can solve our model by the replica method is the repeat until method, in which we focus on the output of the measurements of the quantum circuit. We find that the capacity of a quantum perceptron is larger than that of a classical perceptron since the quantum one is simple but effectively falls into a highly nonlinear form of the activation function.

arXiv:2404.14785Apr 23, 20240 citations

Storage capacity and learning capability of quantum neural networks

Maciej Lewenstein, Aikaterini Gratsea, Andreu Riera-Campeny +3 more

Abstract We study the storage capacity of quantum neural networks (QNNs), described by completely positive trace preserving (CPTP) maps acting on an N-dimensional Hilbert space. We demonstrate that attractor QNNs can store in a non-trivial manner up to N linearly independent pure states. For n qubits, QNNs can reach an exponential storage capacity, O ( 2 n ) , clearly outperforming standard classical neural networks whose storage capacity scales linearly with the number of neurons n. We estimate, employing the Gardner program, the relative volume of CPTP maps with M ⩽ N stationary states and show that this volume decreases exponentially with M and shrinks to zero for M ⩾ N + 1. We generalize our results to QNNs storing mixed states as well as input–output relations for feed-forward QNNs. Our approach opens the path to relate storage properties of QNNs to the quantum features of the input–output states. This paper is dedicated to the memory of Peter Wittek.

arXiv:2011.06113Quantum Science and TechnologyJun 1, 202126 citations

Implementing associative memory with quantum neural networks

Jieyu Zhao

In this paper an attempt is made to implement associative memory with quantum neural networks. The quantum algorithms for the storage of pattern and the retrieval of information are presented. It is shown that the capacity and the processing speed of the quantum neural network increase exponentially compared to the conventional model. The neural computing method for the calculation of quantum entanglement is also discussed.

International Conference on Machine Learning and CyberneticsApr 12, 20052 citations

Comparing Classical Hopfield Networks and a Hybrid Quantum-Classical Associative Memory Model: Recall Accuracy, Noise Robustness, and Storage Capacity

Sanjay Shivaganesh

Associative memory, the ability to reconstruct complete information from partial or noisy cues, is a defining feature of human cognition and an enduring objective in artificial intelligence. Classical neural models such as Hopfield networks have long provided insight into distributed storage and retrieval mechanisms, yet they remain constrained by limited storage capacity and susceptibility to noise. Recent developments in quantum computing have motivated investigation of hybrid quantum-classical architectures as alternative approaches to memory representation and retrieval. This study examines how effectively hybrid quantum-classical models can reproduce associative memory functions compared to Hopfield networks, with a particular focus on recall accuracy under varying noise levels, storage capacity, and computational efficiency. Through small-scale experiments using binary patterns and hybrid quantum circuits, we evaluate potential quantum advantages and practical trade-offs. The findings contribute to ongoing discourse in quantum machine learning and cognitive modeling, underscoring both the promise and current limitations of quantum-inspired associative memory systems. Our guiding research question is: To what extent can hybrid quantum–classical models replicate associative memory functions more effectively than classical neural memory models, particularly in terms of recall accuracy under noise and storage capacity?

Sep 14, 20260 citations