We have investigated the detailed line shapes of modulation transfer spectroscopy (MTS) for the
Fg=2→Fe=1,2 , and 3 transitions of 85Rb due to neighboring hyperfine transitions. This analysis has been conducted under four different polarization configurations: linear parallel, linear perpendicular, circular parallel, and circular perpendicular configurations. To the best of our knowledge the effects of neighboring transitions have been applied to MTS for the first time. The dependencies in the asymmetric MTS spectra resulting from the neighboring effects on the
Fg=2→Fe=1 transition of 85Rb are accounted for by artificially adjusting the hyperfine energy separations in the excited state. We also investigate the detailed MTS line shapes for the
Fg=1→Fe=0,1,2 transitions of 87Rb, considering circular polarization configurations. The significant strength of the MTS spectra for the crossover lines involving the
Fe=0 and 1 states of 87Rb in the
σ+−σ− polarization configuration can be explained through a MTS signal mediated by spontaneous emission. Theoretical line shapes of these transitions, calculated using the density matrix equations, show good agreement with the observed spectra.