We present a theory for quantum nondemolition (QND) measurements of an atomic ensemble in the presence of spontaneous emission. We derive the master equation that governs the evolution of the ground state of the atoms and the quantum state of light. Solving the master equation exactly without invoking the Holstein-Primakoff approximation and projecting out the quantum state of light, we derive a positive operator-valued measure that describes the QND measurement. We show that, at high spontaneous emission conditions, the QND measurement has a unique dominant state to which the measurement collapses. We additionally investigate the behavior of the QND measurement in the limiting case of strong atom-light interactions, where we show that the positive operator valued measure becomes a projection operator. We further analyze the effect of spontaneous emission noise on atomic state preparation. We find that it limits the width of the eigenvalue spectrum available to a quantum state in a linear superposition. This effect leads to state collapse on the dominant state. We generate various nonclassical states of the atom by tuning the atom-light interaction strength. We find that nonclassical states such as the Schr\"odinger-cat state, whose coherence spans the entire eigenvalue spectrum of the total spin operator
Jz for a given spin eigenvalue
J, lose their coherence because spontaneous emission limits the accessibility of states farther away from the dominant state.