Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2504.12037 · Communications in Theoretical Physics

Integrating neural networks and tensor networks for computing free energy

Hanyan Cao, Yijia Wang, Feng Ping Pan, Pan Zhang

Published Mar 31, 2025• 0 citations
arXivHTMLPDF

Abstract

Abstract Computing free energy is a fundamental problem in statistical physics. Recently, two distinct methods have been developed and have demonstrated remarkable success: the tensor-network-based contraction method and the neural-network-based variational method. Tensor networks are accurate, but their application is often limited to low-dimensional systems due to the high computational complexity in high-dimensional systems. The neural network method applies to systems with general topology. However, as a variational method, it is not as accurate as tensor networks. In this work, we propose an integrated approach, tensor-network-based variational autoregressive networks (TNVAN), that leverages the strengths of both tensor networks and neural networks: combining the variational autoregressive neural network’s ability to compute an upper bound on free energy and perform unbiased sampling from the variational distribution with the tensor network’s power to accurately compute the partition function for small sub-systems, resulting in a robust method for precisely estimating free energy. To evaluate the proposed approach, we conducted numerical experiments on spin glass systems with various topologies, including two-dimensional lattices, fully connected graphs, and random graphs. Our numerical results demonstrate the superior accuracy of our method compared to existing approaches. In particular, it effectively handles systems with long-range interactions and leverages GPU efficiency without requiring singular value decomposition, indicating great potential in tackling statistical mechanics problems and simulating high-dimensional complex systems through both tensor networks and neural networks.

DOI
10.1088/1572-9494/adc6f8

Related papers

Neural-network based general method for statistical mechanics on sparse systems

Feng Pan, Haijun Zhou, Pan Zhang

We propose a general method for solving statistical mechanics problems defined on sparse graphs, such as random graphs, real-world networks, and low-dimensional lattices. Our approach extract a small feedback vertex set of the sparse graph, converting the sparse system to a strongly correlated system with many-body and dense interactions on the feedback set, then solve it using variational method based on neural networks to estimate free energy, observables, and generate unbiased samples via direct sampling. Extensive experiments show that our approach is more accurate than existing approaches for sparse spin glass systems. On random graphs and real-world networks, our approach significantly outperforms the standard methods for sparse systems such as belief-propagation; on structured sparse systems such as two-dimensional lattices our approach is significantly faster and more accurate than recently proposed variational autoregressive networks using convolution neural networks.

•Jun 26, 2019•0 citations

Variational Evolutionary Network for Statistical Physics Systems

Ren, Yixiong, Jianhui Zhou

Monte Carlo methods are widely used importance sampling techniques for studying complex physical systems. Integrating these methods with deep learning has significantly improved efficiency and accuracy in high-dimensional problems and complex system simulations. However, these neural network-enhanced Monte Carlo methods still face challenges such as slow sampling speeds, statistical bias, and inaccuracies in the ground state. To address these issues, we propose a variational evolutionary network, which utilizes neural networks for variational free energy and combines evolutionary algorithms for sampling. During the sampling process, we construct generation and selection operators to filter samples based on importance, thereby achieving efficient importance sampling. We demonstrate that this sampling method provides an upper bound on the ground-state energy, enhancing both sampling efficiency and ground-state accuracy. Moreover, we numerically examine our method in two-dimensional Ising model and Sherrington-Kirkpatrick Model for spin glass. Thus, our algorithm could offer improved accuracy in handling complex energy landscapes and significantly enhance computational efficiency.

arXiv:2412.16598•Dec 21, 2024•0 citations

Solving Statistical Mechanics Using Variational Autoregressive Networks

Dian Wu, Lei Wang, Pan Zhang

We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variational free energy, estimates physical quantities such as entropy, magnetizations and correlations, and generates uncorrelated samples all at once. Training of the network employs the policy gradient approach in reinforcement learning, which unbiasedly estimates the gradient of variational parameters. We apply our approach to several classic systems, including 2D Ising models, the Hopfield model, the Sherrington-Kirkpatrick model, and the inverse Ising model, for demonstrating its advantages over existing variational mean-field methods. Our approach sheds light on solving statistical physics problems using modern deep generative neural networks.

arXiv:1809.10606•Physical Review Letters•Feb 28, 2019•218 citations

Tensor networks for unsupervised machine learning

Jing Liu, Sujie Li, Jiang Zhang +1 more

Modeling the joint distribution of high-dimensional data is a central task in unsupervised machine learning. In recent years, many interests have been attracted to developing learning models based on tensor networks, which have the advantages of a principle understanding of the expressive power using entanglement properties, and as a bridge connecting classical computation and quantum computation. Despite the great potential, however, existing tensor network models for unsupervised machine learning only work as a proof of principle, as their performance is much worse than the standard models such as restricted Boltzmann machines and neural networks. In this Letter, we present autoregressive matrix product states (AMPS), a tensor network model combining matrix product states from quantum many-body physics and autoregressive modeling from machine learning. Our model enjoys the exact calculation of normalized probability and unbiased sampling. We demonstrate the performance of our model using two applications, generative modeling on synthetic and real-world data, and reinforcement learning in statistical physics. Using extensive numerical experiments, we show that the proposed model significantly outperforms the existing tensor network models and the restricted Boltzmann machines, and is competitive with state-of-the-art neural network models.

arXiv:2106.12974•Physical review. E•Jan 31, 2023•37 citations

Statistical mechanics in continuous space with tensor network methods

Gunhee Park, Tomislav Begušić, Si-Jing Du +2 more

Tensor network (TN) methods are well established for computing partition functions in statistical mechanics, though this use has traditionally been limited to lattice models. We extend the scope of TN methodology to interacting particle systems in continuous space. Through a real-space discretization combined with a cell-based coarse-graining scheme, we formulate an effective lattice model that explicitly preserves spatial locality. The partition function of this model is represented as a TN, and the thermodynamic quantities are computed via boundary contraction. We apply this framework to the two-dimensional hard-disk problem and demonstrate the strengths of the TN formulation compared to existing Monte Carlo simulations.

arXiv:2604.25060•Apr 27, 2026•0 citations

Deep Autoregressive Models for the Efficient Variational Simulation of Many-Body Quantum Systems

Or Sharir, Yoav Levine, Noam Wies +2 more

Artificial neural networks were recently shown to be an efficient representation of highly entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in variational Monte Carlo method, most notably the use of Markov-chain Monte Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized neural-network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.

arXiv:1902.04057•Physical Review Letters•Jan 16, 2020•257 citations
Abstract Computing free energy is a fundamental problem in statistical physics. Recently, two distinct methods have been developed and have demonstrated remarkable success: the tensor-network-based contraction method and the neural-network-based variational method. Tensor networks are accurate, but their application is often limited to low-dimensional systems due to the high computational complexity in high-dimensional systems. The neural network method applies to systems with general topology. However, as a variational method, it is not as accurate as tensor networks. In this work, we propose an integrated approach, tensor-network-based variational autoregressive networks (TNVAN), that leverages the strengths of both tensor networks and neural networks: combining the variational autoregressive neural network’s ability to compute an upper bound on free energy and perform unbiased sampling from the variational distribution with the tensor network’s power to accurately compute the partition function for small sub-systems, resulting in a robust method for precisely estimating free energy. To evaluate the proposed approach, we conducted numerical experiments on spin glass systems with various topologies, including two-dimensional lattices, fully connected graphs, and random graphs. Our numerical results demonstrate the superior accuracy of our method compared to existing approaches. In particular, it effectively handles systems with long-range interactions and leverages GPU efficiency without requiring singular value decomposition, indicating great potential in tackling statistical mechanics problems and simulating high-dimensional complex systems through both tensor networks and neural networks.
We propose a general method for solving statistical mechanics problems defined on sparse graphs, such as random graphs, real-world networks, and low-dimensional lattices. Our approach extract a small feedback vertex set of the sparse graph, converting the sparse system to a strongly correlated system with many-body and dense interactions on the feedback set, then solve it using variational method based on neural networks to estimate free energy, observables, and generate unbiased samples via direct sampling. Extensive experiments show that our approach is more accurate than existing approaches for sparse spin glass systems. On random graphs and real-world networks, our approach significantly outperforms the standard methods for sparse systems such as belief-propagation; on structured sparse systems such as two-dimensional lattices our approach is significantly faster and more accurate than recently proposed variational autoregressive networks using convolution neural networks.
Monte Carlo methods are widely used importance sampling techniques for studying complex physical systems. Integrating these methods with deep learning has significantly improved efficiency and accuracy in high-dimensional problems and complex system simulations. However, these neural network-enhanced Monte Carlo methods still face challenges such as slow sampling speeds, statistical bias, and inaccuracies in the ground state. To address these issues, we propose a variational evolutionary network, which utilizes neural networks for variational free energy and combines evolutionary algorithms for sampling. During the sampling process, we construct generation and selection operators to filter samples based on importance, thereby achieving efficient importance sampling. We demonstrate that this sampling method provides an upper bound on the ground-state energy, enhancing both sampling efficiency and ground-state accuracy. Moreover, we numerically examine our method in two-dimensional Ising model and Sherrington-Kirkpatrick Model for spin glass. Thus, our algorithm could offer improved accuracy in handling complex energy landscapes and significantly enhance computational efficiency.
We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variational free energy, estimates physical quantities such as entropy, magnetizations and correlations, and generates uncorrelated samples all at once. Training of the network employs the policy gradient approach in reinforcement learning, which unbiasedly estimates the gradient of variational parameters. We apply our approach to several classic systems, including 2D Ising models, the Hopfield model, the Sherrington-Kirkpatrick model, and the inverse Ising model, for demonstrating its advantages over existing variational mean-field methods. Our approach sheds light on solving statistical physics problems using modern deep generative neural networks.
Modeling the joint distribution of high-dimensional data is a central task in unsupervised machine learning. In recent years, many interests have been attracted to developing learning models based on tensor networks, which have the advantages of a principle understanding of the expressive power using entanglement properties, and as a bridge connecting classical computation and quantum computation. Despite the great potential, however, existing tensor network models for unsupervised machine learning only work as a proof of principle, as their performance is much worse than the standard models such as restricted Boltzmann machines and neural networks. In this Letter, we present autoregressive matrix product states (AMPS), a tensor network model combining matrix product states from quantum many-body physics and autoregressive modeling from machine learning. Our model enjoys the exact calculation of normalized probability and unbiased sampling. We demonstrate the performance of our model using two applications, generative modeling on synthetic and real-world data, and reinforcement learning in statistical physics. Using extensive numerical experiments, we show that the proposed model significantly outperforms the existing tensor network models and the restricted Boltzmann machines, and is competitive with state-of-the-art neural network models.
Tensor network (TN) methods are well established for computing partition functions in statistical mechanics, though this use has traditionally been limited to lattice models. We extend the scope of TN methodology to interacting particle systems in continuous space. Through a real-space discretization combined with a cell-based coarse-graining scheme, we formulate an effective lattice model that explicitly preserves spatial locality. The partition function of this model is represented as a TN, and the thermodynamic quantities are computed via boundary contraction. We apply this framework to the two-dimensional hard-disk problem and demonstrate the strengths of the TN formulation compared to existing Monte Carlo simulations.
Artificial neural networks were recently shown to be an efficient representation of highly entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in variational Monte Carlo method, most notably the use of Markov-chain Monte Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized neural-network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.