We present a Heisenberg-Langevin formalism to study the effective dynamics of a superconducting qubit coupled to an open multimode resonator, without resorting to the rotating wave, two level, Born or Markov approximations. Our effective equations are derived by eliminating resonator degrees of freedom while encoding their effect in the Green's function of the electromagnetic background. We account for the openness of the resonator exactly by employing a spectral representation for the Green's function in terms of a set of non-Hermitian modes. A well-behaved time domain perturbation theory is derived to systematically account for the nonlinearity of weakly nonlinear qubits like transmon. We apply this method to the problem of spontaneous emission, capturing accurately the non-Markovian features of the qubit dynamics, valid for any qubit-resonator coupling strength. Any discrete-level quantum system coupled to the electromagnetic continuum is subject to radiative decay and renormalization of its energy levels. When inside a cavity, these quantities can be strongly modified with respect to vacuum. Generally, this modification can be captured by including only the closest resonant cavity mode. In circuit-QED architecture, with substantial coupling strengths, it is however found that such rates are strongly influenced by far off-resonant modes. A multimode calculation over the infinite set of cavity modes leads to divergences unless an artificial cutoff is imposed. Previous studies have not pointed out what the source of this divergence is. We show that unless the effect of
A2 is accounted for up to all orders exactly, any multimode calculations of circuit-QED quantities is bound to diverge. Subsequently, we present the calculation of finite radiative corrections to qubit properties that is free of an artificially introduced high frequency cut-off.