Inverse Ising inference allows pairwise interactions of complex binary systems to be reconstructed from empirical correlations. Typical estimators used for this inference, such as pseudo-likelihood maximization (PLM), are biased. Using the Sherrington-Kirkpatrick model as a benchmark, we show that these biases are large in critical regimes close to phase boundaries, and they may alter the qualitative interpretation of the inferred model. In particular, we show that the small-sample bias causes models inferred through PLM to appear closer to criticality than one would expect from the data. Data-driven methods to correct this bias are explored and applied to a functional magnetic resonance imaging data set from neuroscience. Our results indicate that additional care should be taken when attributing criticality to real-world data sets.
Abstract Using methods of statistical physics, we analyse the error of learning couplings in large Ising models from independent data (the inverse Ising problem). We concentrate on learning based on local cost functions, such as the pseudo-likelihood method for which the couplings are inferred independently for each spin. Assuming that the data are generated from a true Ising model, we compute the reconstruction error of the couplings using a combination of the replica method with the cavity approach for densely connected systems. We show that an explicit estimator based on a quadratic cost function achieves minimal reconstruction error, but requires the length of the true coupling vector as prior knowledge. A simple mean field estimator of the couplings which does not need such knowledge is asymptotically optimal, i.e. when the number of observations is much larger than the number of spins. Comparison of the theory with numerical simulations shows excellent agreement for data generated from two models with random couplings in the high temperature region: a model with independent couplings (Sherrington–Kirkpatrick model), and a model where the matrix of couplings has a Wishart distribution.
We present a systematic small-correlation expansion to solve the inverse Ising problem and find a set of couplings and fields corresponding to a given set of correlations and magnetizations. Couplings are calculated up to the third order in the correlations for generic magnetizations and to the seventh order in the case of zero magnetizations; in addition, we show how to sum some useful classes of diagrams exactly. The resulting expansion outperforms existing algorithms on the Sherrington–Kirkpatrick spin-glass model.
When independent learners are trained on data from the same disordered system, finite and noisy samples can lead them to different descriptions of the underlying physics. We quantify the reproducibility of such descriptions through consensus, the degree of agreement among independently trained models. In a teacher-student framework on the two-dimensional random-bond Ising model, an ensemble of students each infers a single uniform effective coupling from a finite set of equilibrium configurations drawn from one quenched bond realization. Aggregating the inferred couplings across students and realizations yields the consensus distribution, whose width quantifies the agreement among learners. The variance of this distribution separates exactly into a finite-sampling contribution, bounded below by inverse of Fisher information, and a quenched-disorder contribution. Sum of these two variance is minimized near critical region, so the consensus width has a minimum near the pseudocritical temperature that sharpens with increasing disorder. The two minima arise from different mechanisms: the sampling term inherits the critical peak of the Fisher information, while the disorder term is temperature-independent at leading order and acquires its critical minimum only at fourth order in the disorder strength, through the covariance between the linear and cubic responses to the bond disorder. Because the students fit a uniform coupling to a heterogeneous lattice, the inferred coupling also carries a misspecification bias that survives in the infinite-data limit. This bias displaces the minimum of the mean-squared error above the consensus minimum, so the temperature at which independent learners agree most closely is not the temperature at which their shared answer is most faithful.
Diverse social, engineered, and biological systems are made up of multiple interacting dynamical components; to work properly, certain components must be carefully regulated so that their dynamics avoid instability. However, many desirable properties, like optimized computation and control, emerge close to the tipping point between stability and instability, i.e., near criticality. Claims that such dynamical systems operate near criticality have been controversial because it is difficult to assess proximity to criticality when control parameters are unknown, hard to manipulate experimentally, and fluctuating in response to changing environmental or internal conditions. Moreover, observations of system dynamics are often limited to time series measurements of one or a few components. Here we develop a temporal renormalization group theory for defining criticality for univariate time series models and an information theoretic definition of proximity to criticality that is independent of model parameterization. We apply this theory to order-n autoregressive models resulting in a data-analytic strategy for time-resolved estimates of proximity to criticality based on measured univariate time series. After benchmarking our approach on diverse ground-truth cases, we apply it to recordings of spiking activity in the mammalian brain. Our estimate of proximity to criticality predicts behavior and suggests that brain dynamics shift closer to criticality during wakefulness and shift away during deep sleep.
Generative learning of a quantum measurement distribution from finite data requires choosing a model capacity that balances representation accuracy against estimation cost. We develop a bias-variance framework that decomposes the expected population Kullback-Leibler error at model capacity
s and sample size
N as
E(s,N)=B(s)+V(s,N), with representation bias
B and a variance term
V capturing the mean estimation error. In the well-sampled regime,
V(s,N)≃[NC(s)]−1, and exponential or algebraic laws for
B and
C yield four scaling templates for the optimal error, from inverse powers of
N to inverse powers of
lnN. For local autoregressive models, the context radius sets the model capacity, and
B is the summed conditional mutual information lost by truncating the context. For a smoothed histogram learner, we derive an exact finite-sample estimation cost that accounts for rarely observed contexts and recovers the parameter-count law in the well-sampled regime. For transverse-field Ising Born distributions, the independently evaluated bias and estimation cost predict both the optimal context radius and the optimal error. In the well-sampled regime, the estimation cost grows exponentially with context radius. Its competition with approximately exponential bias decay in the gapped regimes predicts power-law error decay in
N, while algebraic bias decay at criticality predicts asymptotic inverse-power decay in
lnN. Bulk-site results are consistent with the corresponding finite-sample predictions. These results connect the structure of quantum measurement distributions to the model capacity justified by finite data.