We propose a scheme to achieve efficient frequency conversion for a single photon propagating in a 1D conventional waveguide by exploiting the quantum interference induced by the scale of a V-type giant atom (GA) characterized by the distance between the two coupling points as well as single-photon transition pathways originated from the coupling between the GA and the resonator. The presence of photons in the resonator triggers the frequency conversion of photons. The scattering spectra and the conversion contrast are studied in both the Markovian and the non-Markovian regimes. The disappearance of frequency conversion is rooted in the complete suppression of the emission from the excited state to either of lower states in the
n+1 subspace where
n is the photon number of the resonator, and the non-Markovicity-induced nonreciprocity is found under specific conditions. Altering the photon number
n induces the non-reciprocal transmission of single photons in the waveguide, hence, enhance the conversion probability.