Symmetry transformations have proved useful in determining the algebraic structure and internal dynamical properties of physical systems. In the quantum Rabi model, invariance under parity symmetry transformation has been used to obtain exact solutions of the eigenvalue equation and very good approximations of the internal dynamics of the interacting atom-light system. In this article, two symmetry operators, characterized as "duality" symmetry operators, have been introduced which transform the quantum Rabi Hamiltonian into duality conjugates. The parity and duality symmetry operators constitute an algebraically closed set of symmetry transformation operators of the quantum Rabi model. The closed
SU(2) Lie algebra provides the standard eigenvalues and eigenstates of the parity symmetry operator. It is established that Jaynes-Cummings and anti-Jaynes-Cummings operators are duality symmetry conjugates. Symmetric or antisymmetric linear combinations of the Rabi Hamiltonian and a corresponding duality conjugate yield the familiar spin-dependent force driven bosonic , coupling-only or quantized light mode quadrature-driven fermionic Hamiltonian. It is established that the effective bosonic, fermionic and coupling-only Hamiltonians are exact, not approximate forms of the quantum Rabi Hamiltonian as they have generally been interpreted. The effective bosonic form generates the dynamics of the light mode driven by the atomic spin-dependent force, while the fermionic form generates the dynamics of the atomic spin driven by the quantized light mode quadrature-dependent force, thus providing a complete picture of the quantum Rabi dynamics.