Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2501.18691

Regularized second-order optimization of tensor-network Born machines

Matan Ben-Dov, Jing Chen

Published Jan 30, 2025• 0 citations
arXivHTMLPDF

Abstract

Tensor-network Born machines (TNBMs) are quantum-inspired generative models for learning data distributions. Using tensor-network contraction and optimization techniques, the model learns an efficient representation of the target distribution, capable of capturing complex correlations with a compact parameterization. Despite their promise, the optimization of TNBMs presents several challenges. A key bottleneck of TNBMs is the logarithmic nature of the loss function commonly used for this problem. The single-tensor logarithmic optimization problem cannot be solved analytically, necessitating an iterative approach that slows down convergence and increases the risk of getting trapped in one of many non-optimal local minima. In this paper, we present an improved second-order optimization technique for TNBM training, which significantly enhances convergence rates and the quality of the optimized model. Our method employs a modified Newton's method on the manifold of normalized states, incorporating regularization of the loss landscape to mitigate local minima issues. We demonstrate the effectiveness of our approach by training a one-dimensional matrix product state (MPS) on both discrete and continuous datasets, showcasing its advantages in terms of stability and efficiency, and demonstrating its potential as a robust and scalable approach for optimizing quantum-inspired generative models.

DOI
10.48550/arxiv.2501.18691

Related papers

Quantum-Inspired Optimization of Deep Learning Hyperparameters via Tensor Networks

Jincheng Zhang

Optimizing hyperparameters in deep learning models remains a significant bottleneck in the development and deployment of effective neural networks. Traditional methods, such as grid search and random search, often struggle with the exponentially growing search space, leading to substantial computational costs and limited exploration of potentially optimal configurations. This paper proposes a novel approach to deep learning hyperparameter optimization that leverages the principles of quantum-inspired optimization, specifically employing tensor network methods, notably Matrix Product States (MPS), to efficiently approximate the complex optimization landscape. Our method utilizes quantum-inspired sampling techniques to guide the exploration, effectively mimicking the probabilistic nature of quantum measurements. We demonstrate the feasibility and potential advantages of this approach through theoretical analysis and a conceptual framework. The core of our method lies in representing the hyperparameter space as a tensor network, allowing for efficient calculations of the objective function (e.g., validation loss) without exhaustively evaluating every possible combination. This significantly reduces the computational burden compared to conventional methods. The paper outlines a detailed algorithm and discusses key considerations for implementation, including tensor network construction, sampling strategies, and convergence criteria. We propose a framework for iteratively refining the tensor network representation based on observed validation performance, ultimately converging towards a high-performing hyperparameter configuration. The theoretical underpinning draws upon concepts from quantum mechanics, particularly the idea of measurement and probability distribution, to drive the optimization process. We believe this quantum-inspired approach offers a promising avenue for accelerating deep learning hyperparameter tuning and achieving state-of-the-art performance.

•Sep 2, 2026•0 citations

Initialization and training of matrix product state probabilistic models

Xun Tang, Yuehaw Khoo, Lexing Ying B

Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap, where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure -- one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth.

arXiv:2505.06419•May 9, 2025•0 citations

Quantum Tensor Network Learning with DMRG

Gustav J L Jäger, Martin B Plenio, Hans-Martin Rieser

Tensor Networks are a relatively new machine learning approach. The architectures proposed initially are inspired by approaches from quantum many-body physics simulations. One common layout is the matrix product state (MPS) also known as a tensor train optimized with gradient descent techniques. We introduce a global normalization condition, so that the MPS represents a quantum state. We investigate two optimization methods that find the locally optimal tensors and compare them regarding their effectiveness. One is based on gradient descent and the other on an adaptation of DMRG.

quant-pharXiv:2608.18901•European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)•Aug 20, 2026•1 citation

Sequential learning on a tensor network Born machine with trainable token embedding

Wanda Hou, Miao Li, Yi‐Zhuang You

Abstract Generative models aim to learn the probability distributions underlying data, enabling the generation of new, realistic samples. Quantum-inspired generative models, such as Born machines based on the matrix product state (MPS) framework, have demonstrated remarkable capabilities in unsupervised learning tasks. This study advances the Born machine paradigm by introducing trainable token embeddings through positive operator-valued measurements (POVMs), replacing the traditional approach of static tensor indices. Key technical innovations include encoding tokens as quantum measurement operators with trainable parameters and leveraging QR decomposition to adjust the physical dimensions of the MPS. This approach maximizes the utilization of operator space and enhances the model’s expressiveness. Empirical results on RNA data demonstrate that the proposed method significantly reduces negative log-likelihood compared to one-hot embeddings, with higher physical dimensions further enhancing single-site probabilities and multi-site correlations. The model also outperforms GPT-2 in single-site estimation and achieves competitive correlation modeling, showcasing the potential of trainable POVM embeddings for complex data correlations in quantum-inspired sequence modeling.

arXiv:2311.05050•Machine Learning Science and Technology•May 28, 2025•2 citations

Application of quantum-inspired tensor networks to optimize federated learning systems

Amandeep Singh Bhatia, Mandeep Kaur Saggi, Sabre Kais

Abstract Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.

Quantum Machine Intelligence•Jan 28, 2025•14 citations

Tensor Network Generator-Enhanced Optimization for Traveling Salesman Problem

Ryo Sakai, Chen-Yu Liu

We present an application of the tensor network generator-enhanced optimization (TN-GEO) framework to address the traveling salesman problem (TSP), a fundamental combinatorial optimization challenge. Our approach employs a tensor network Born machine based on automatically differentiable matrix product states (MPS) as the generative model, using the Born rule to define probability distributions over candidate solutions. Unlike approaches based on binary encoding, which require $N^2$ variables and penalty terms to enforce valid tour constraints, we adopt a permutation-based formulation with integer variables and use autoregressive sampling with masking to guarantee that every generated sample is a valid tour by construction. We also introduce a $k$-site MPS variant that learns distributions over $k$-grams (consecutive city subsequences) using a sliding window approach, enabling parameter-efficient modeling for larger instances. Experimental validation on TSPLIB benchmark instances with up to 52 cities demonstrates that TN-GEO can outperform classical heuristics including swap and 2-opt hill-climbing. The $k$-site variants, which put more focus on local correlations, show better results compared to the full-MPS case.

cs.LGCross-listedarXiv:2602.20175•Feb 16, 2026•0 citations
Tensor-network Born machines (TNBMs) are quantum-inspired generative models for learning data distributions. Using tensor-network contraction and optimization techniques, the model learns an efficient representation of the target distribution, capable of capturing complex correlations with a compact parameterization. Despite their promise, the optimization of TNBMs presents several challenges. A key bottleneck of TNBMs is the logarithmic nature of the loss function commonly used for this problem. The single-tensor logarithmic optimization problem cannot be solved analytically, necessitating an iterative approach that slows down convergence and increases the risk of getting trapped in one of many non-optimal local minima. In this paper, we present an improved second-order optimization technique for TNBM training, which significantly enhances convergence rates and the quality of the optimized model. Our method employs a modified Newton's method on the manifold of normalized states, incorporating regularization of the loss landscape to mitigate local minima issues. We demonstrate the effectiveness of our approach by training a one-dimensional matrix product state (MPS) on both discrete and continuous datasets, showcasing its advantages in terms of stability and efficiency, and demonstrating its potential as a robust and scalable approach for optimizing quantum-inspired generative models.
Optimizing hyperparameters in deep learning models remains a significant bottleneck in the development and deployment of effective neural networks. Traditional methods, such as grid search and random search, often struggle with the exponentially growing search space, leading to substantial computational costs and limited exploration of potentially optimal configurations. This paper proposes a novel approach to deep learning hyperparameter optimization that leverages the principles of quantum-inspired optimization, specifically employing tensor network methods, notably Matrix Product States (MPS), to efficiently approximate the complex optimization landscape. Our method utilizes quantum-inspired sampling techniques to guide the exploration, effectively mimicking the probabilistic nature of quantum measurements. We demonstrate the feasibility and potential advantages of this approach through theoretical analysis and a conceptual framework. The core of our method lies in representing the hyperparameter space as a tensor network, allowing for efficient calculations of the objective function (e.g., validation loss) without exhaustively evaluating every possible combination. This significantly reduces the computational burden compared to conventional methods. The paper outlines a detailed algorithm and discusses key considerations for implementation, including tensor network construction, sampling strategies, and convergence criteria. We propose a framework for iteratively refining the tensor network representation based on observed validation performance, ultimately converging towards a high-performing hyperparameter configuration. The theoretical underpinning draws upon concepts from quantum mechanics, particularly the idea of measurement and probability distribution, to drive the optimization process. We believe this quantum-inspired approach offers a promising avenue for accelerating deep learning hyperparameter tuning and achieving state-of-the-art performance.
Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap, where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure -- one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth.
Tensor Networks are a relatively new machine learning approach. The architectures proposed initially are inspired by approaches from quantum many-body physics simulations. One common layout is the matrix product state (MPS) also known as a tensor train optimized with gradient descent techniques. We introduce a global normalization condition, so that the MPS represents a quantum state. We investigate two optimization methods that find the locally optimal tensors and compare them regarding their effectiveness. One is based on gradient descent and the other on an adaptation of DMRG.
Abstract Generative models aim to learn the probability distributions underlying data, enabling the generation of new, realistic samples. Quantum-inspired generative models, such as Born machines based on the matrix product state (MPS) framework, have demonstrated remarkable capabilities in unsupervised learning tasks. This study advances the Born machine paradigm by introducing trainable token embeddings through positive operator-valued measurements (POVMs), replacing the traditional approach of static tensor indices. Key technical innovations include encoding tokens as quantum measurement operators with trainable parameters and leveraging QR decomposition to adjust the physical dimensions of the MPS. This approach maximizes the utilization of operator space and enhances the model’s expressiveness. Empirical results on RNA data demonstrate that the proposed method significantly reduces negative log-likelihood compared to one-hot embeddings, with higher physical dimensions further enhancing single-site probabilities and multi-site correlations. The model also outperforms GPT-2 in single-site estimation and achieves competitive correlation modeling, showcasing the potential of trainable POVM embeddings for complex data correlations in quantum-inspired sequence modeling.
Abstract Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.
We present an application of the tensor network generator-enhanced optimization (TN-GEO) framework to address the traveling salesman problem (TSP), a fundamental combinatorial optimization challenge. Our approach employs a tensor network Born machine based on automatically differentiable matrix product states (MPS) as the generative model, using the Born rule to define probability distributions over candidate solutions. Unlike approaches based on binary encoding, which require N2N^2N2 variables and penalty terms to enforce valid tour constraints, we adopt a permutation-based formulation with integer variables and use autoregressive sampling with masking to guarantee that every generated sample is a valid tour by construction. We also introduce a kkk-site MPS variant that learns distributions over kkk-grams (consecutive city subsequences) using a sliding window approach, enabling parameter-efficient modeling for larger instances. Experimental validation on TSPLIB benchmark instances with up to 52 cities demonstrates that TN-GEO can outperform classical heuristics including swap and 2-opt hill-climbing. The kkk-site variants, which put more focus on local correlations, show better results compared to the full-MPS case.