Conventional phase-shifting interferometry-based (PSI-based) cryptosystem needs at least two-step phase-shifting. In this work, we propose a phase-shifting-free interferometric cryptosystem, which needs only one interferogram recording. Since the phase-shifting step is not required in the proposed cryptosystem, not only the low encryption speed which is a bottleneck problem of the conventional PSI-based one is solved, but also the setup of the cryptosystem is simplified. A series of simulation experimental results demonstrate the validity and robustness of the proposed cryptosystem.