Exploring the Interpretability of Quantum Neural Networks by Mutual Information

Xin Zhang, Xiaoyu Li, Zhaoyang Dong +1 more

Maciej Koch-Janusz, Zohar Ringel

Zhang, Xin, Yuexian Hou

Quantum neural networks (QNNs) is a parameterized quantum circuit model, which can be trained by gradient-based optimizer, can be used for supervised learning, regression tasks, combinatorial optimization, etc. Although many works have demonstrated that QNNs have better learnability, generalizability, etc. compared to classical neural networks. However, as with classical neural networks, we still can't explain their working mechanism well. In this paper, we reveal the training mechanism of QNNs by mutual information. Unlike traditional mutual information in neural networks, due to quantum computing remains information conserved, the mutual information is trivial of the input and output of U operator. In our work, in order to observe the change of mutual information during training, we divide the quantum circuit (U operator) into two subsystems, discard subsystem (D) and measurement subsystem (M) respectively. We calculate two mutual information, I(Di : Mo) and I(Mi : Mo) (i and o means input or output of the corresponding subsystem), and observe their behavior during training. As the epochs increases, I(Di : Mo) gradually increases, this may means some information of discard subsystem is continuously pushed into the measurement subsystem, the information should be label-related. What's more, I(Mi : Mo) exist two-phase behavior in training process, this consistent with the information bottleneck anticipation. The first phase, I(Mi : Mo) is increasing, this means the measurement subsystem perform feature fitting. The second phase, I(Mi : Mo) is decreasing, this may means the system is generalizing, the measurement subsystem discard label-irrelevant information into the discard subsystem as many as possible. Our work discussed the working mechanism of QNNs by mutual information, further, it can be used to analyze the accuracy and generalization of QNNs.

Wei Cui, Shilu Yan

Myeongjin Shin, Junseo Lee, Kabgyun Jeong

Abstract We propose a method of quantum machine learning called quantum mutual information neural estimation (QMINE) for estimating von Neumann entropy and quantum mutual information, which are fundamental properties in quantum information theory. The QMINE proposed here basically utilizes a technique of quantum neural networks (QNNs), to minimize a loss function that determines the von Neumann entropy, and thus quantum mutual information, which is believed more powerful to process quantum datasets than conventional neural networks due to quantum superposition and entanglement. To create a precise loss function, we propose a quantum Donsker-Varadhan representation (QDVR), which is a quantum analog of the classical Donsker-Varadhan representation. By exploiting a parameter shift rule on parameterized quantum circuits, we can efficiently implement and optimize the QNN and estimate the quantum entropies using the QMINE technique. Furthermore, numerical observations support our predictions of QDVR and demonstrate the good performance of QMINE.

Edward Gillman, Federico Carollo, Igor Lesanovsky

Central to the field of quantum machine learning is the design of quantum perceptrons and neural network architectures. A key question in this regard is the impact of quantum effects on the way such models process information. Here, we establish a connection between (1+1)D quantum cellular automata, which implement a discrete nonequilibrium quantum many-body dynamics through successive applications of local quantum gates, and quantum neural networks (QNNs), which process information by feeding it through perceptrons interconnecting adjacent layers. Exploiting this link, we construct a class of QNNs that are highly structured-aiding both interpretability and helping to avoid trainability issues in machine learning tasks-yet can be connected rigorously to continuous-time Lindblad dynamics. We further analyze the universal properties of an example case, identifying a change of critical behavior when quantum effects are varied, showing their potential impact on the collective dynamical behavior underlying information processing in large-scale QNNs.