The low-temperature behavior of the infinite-range Ising spin-glass model with a Gaussian distribution of interactions is studied by evaluating the statistical mechanics exactly for small systems (up to
N=20 spins). Our results can be well understood by a simple picture of elementary cluster excitations from the ground state which overturn, on average, of order
N21 spins. The conventional definition of the spin-glass order parameter is shown to be unsatisfactory and a new definition is proposed. It is speculated that ensemble averages may give different results from time averages for $N\ensuremath{\rightarrow}\ensuremath{\infty}$ even in the infinite time limit.