Charge state instabilities have been a bottleneck for the implementation of solid-state spin systems and pose a major challenge to the development of spin-based quantum technologies. Here we investigate the stabilization of negatively charged nitrogen-vacancy (NV
−) centers in phosphorus-doped diamond at liquid helium temperatures. Photoionization of phosphorous donors in conjunction with charge diffusion at the nanoscale enhances NV
0 to NV
− conversion and stabilizes the NV
− charge state without the need for an additional repump laser. The phosphorus-assisted stabilization is explored and confirmed both with experiments and our theoretical model. Stable photoluminescence-excitation spectra are obtained for NV
− centers created during the growth. The fluorescence is continuously recorded under resonant excitation to real-time monitor the charge state and the ionization and recombination rates are extracted from time traces. We find a linear laser power dependence of the recombination rate as opposed to the conventional quadratic dependence, which is attributed to the photo-ionization of phosphorus atoms.