arXiv:2412.17497 · Algorithms

Advantages of Density in Tensor Network Geometries for Gradient-Based Training

Sergi Masot-Llima, Artur García-Sáez

Published Jan 31, 2025• 0 citations

Abstract

Tensor networks are a very powerful data structure tool originating from simulations of quantum systems. In recent years, they have seen increased use in machine learning, mostly in trainings with gradient-based techniques, due to their flexibility and performance achieved by exploiting hardware acceleration. As ansatzes, tensor networks can be used with flexible geometries, and it is known that for highly regular ones, their dimensionality has a large impact on performance and representation power. For heterogeneous structures, however, these effects are not completely characterized. In this article, we train tensor networks with different geometries to encode a random quantum state, and see that densely connected structures achieve better infidelities than more sparse structures, with higher success rates and less time. Additionally, we give some general insight on how to improve the memory requirements of these sparse structures and the impact of such improvement on the trainings. Finally, as we use HPC resources for the calculations, we discuss the requirements for this approach and showcase performance improvements with GPU acceleration on a last-generation supercomputer.

DOI: 10.3390/a18020070

Related papers

Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning

Fergus Barratt, James Dborin, Wright, Lewis

Tensor networks have demonstrated significant value for machine learning in a myriad of different applications. However, optimizing tensor networks using standard gradient descent has proven to be difficult in practice. Tensor networks suffer from initialization problems resulting in exploding or vanishing gradients and require extensive hyperparameter tuning. Efforts to overcome these problems usually depend on specific network architectures, or ad hoc prescriptions. In this paper we address the problems of initialization and hyperparameter tuning, making it possible to train tensor networks using established machine learning techniques. We introduce a `copy node' method that successfully initializes arbitrary tensor networks, in addition to a gradient based regularization technique for bond dimensions. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models with far fewer parameters, while improving generalization performance.

arXiv:2203.03366Mar 3, 20222 citations

Tensor Networks: a quantum-information perspective on numerical renormalization groups

Pietro Silvi

Exact many-body quantum problems are known to be computationally hard due to the exponential scaling of the numerical resources required. Since the advent of the Density Matrix Renormalization Group, it became clear that a successful strategy to work around this obstacle was to develop numerical methods based on the well-known theoretical renormalization group. In recent years, it was realized that quantum states engineered via numerical renormalization allow a variational representation in terms of a tensor network picture. The discovery provided a further boost to the effectiveness of these techniques, not only due to the increased flexibility and manipulability, but also because tensor network states embed a direct interface to the entanglement they carry, so that one can directly address many-body quantum correlations within these variational ansatz states. This lead to the application of several numerical tools, originally developed in the field of quantum-information, to approach condensed matter problems.

arXiv:1205.4198May 18, 20121 citation

Entangling Solid Solutions: Machine Learning of Tensor Networks for Materials Property Prediction

David E. Sommer, Scott T. Dunham

Progress in the application of machine learning techniques to the prediction of solid-state and molecular materials properties has been greatly facilitated by the development state-of-the-art feature representations and novel deep learning architectures. A large class of atomic structure representations based on expansions of smoothed atomic densities have been shown to correspond to specific choices of basis sets in an abstract many-body Hilbert space. Concurrently, tensor network structures, conventionally the purview of quantum many-body physics and quantum information, have been successfully applied in supervised and unsupervised learning tasks in computer vision and natural language processing. In this work, we argue that architectures based on tensor networks are well-suited to machine learning on Hilbert-space representations of atomic structures. This is demonstrated on supervised learning tasks involving widely available datasets of density functional theory calculations of metal and semiconductor alloys. In particular, we show that certain standard tensor network topologies exhibit strong generalizability even on small training datasets while being parametrically efficient. We further relate this generalizability to the presence of complex entanglement in the trained tensor networks. We also discuss connections to learning with generalized structural kernels and related strategies for compressing large input feature spaces.

arXiv:2203.09613Mar 17, 20221 citation

Classical versus quantum: Comparing tensor-network-based quantum circuits on Large Hadron Collider data

Jack Y. Araz, Michael Spannowsky

Tensor networks (TN) are approximations of high-dimensional tensors designed to represent locally entangled quantum many-body systems efficiently. This paper provides a comprehensive comparison between classical TNs and TN-inspired quantum circuits in the context of machine learning on highly complex, simulated Large Hadron Collider data. We show that classical TNs require exponentially large bond dimensions and higher Hilbert-space mapping to perform comparably to their quantum counterparts. While such an expansion in the dimensionality allows better performance, we observe that, with increased dimensionality, classical TNs lead to a highly flat loss landscape, rendering the usage of gradient-based optimization methods highly challenging. Furthermore, by employing quantitative metrics, such as the Fisher information and effective dimensions, we show that classical TNs require a more extensive training sample to represent the data as efficiently as TN-inspired quantum circuits. We also engage with the idea of hybrid classical-quantum TNs and show possible architectures to employ a larger phase space from the data. We offer our results using three main TN Ans\"atze: Tree tensor networks, matrix product states, and multiscale entanglement renormalization Ans\"atze.

arXiv:2202.10471Physical Review ADec 19, 202232 citations

Application of quantum-inspired tensor networks to optimize federated learning systems

Amandeep Singh Bhatia, Mandeep Kaur Saggi, Sabre Kais

Abstract Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.

Quantum Machine IntelligenceJan 28, 202514 citations

Density peak clustering using tensor network

Xiao Shi, Yun Shang

Tensor networks, which have been traditionally used to simulate many-body physics, have recently gained significant attention in the field of machine learning due to their powerful representation capabilities. In this work, we propose a density-based clustering algorithm inspired by tensor networks. We encode classical data into tensor network states on an extended Hilbert space and train the tensor network states to capture the features of the clusters. Here, we define density and related concepts in terms of fidelity, rather than using a classical distance measure. We evaluate the performance of our algorithm on six synthetic data sets, four real world data sets, and three commonly used computer vision data sets. The results demonstrate that our method provides state-of-the-art performance on several synthetic data sets and real world data sets, even when the number of clusters is unknown. Additionally, our algorithm performs competitively with state-of-the-art algorithms on the MNIST, USPS, and Fashion-MNIST image data sets. These findings reveal the great potential of tensor networks for machine learning applications.

arXiv:2302.00192Feb 1, 20230 citations