Ultracold atoms uniformly filling an optical lattice can be treated like an artificial crystal. An implementation including the atomic occupation of a single excited atomic state can be represented by a two-component Bose-Hubbard model. Its phase diagram exhibits a quantum phase transition from a superfluid to a Mott insulator phase. The dynamics of electronic excitations governed by electrostatic dipole-dipole interactions in the ordered regime can be well described by wave-like collective excitations called excitons. Here we present an extensive study of such excitons for a wide range of geometries and dimensionality. Their lifetimes can vary over many orders of magnitude from metastable propagation to superradiant decay. Particularly strong effects occur in one dimensional atomic chains coupled to tapered optical fibers. For an optical lattice within a cavity the excitons are coupled to cavity photons and the resulting collective cavity QED model can be efficiently formulated in terms of polaritons. Their properties are explicitly calculated for different lattices and they constitute a non-destructive monitoring tool for important system properties. Even the formation of molecules in optical lattices manifests itself in modified polariton properties as e.g. an anisotropic optical spectrum. Partial dissipation of the exciton energy in the lattice leads to heating, which can be microscopically understood through a mechanism transferring atoms into higher Bloch bands via a resonant excitation transfer among neighboring lattice sites. The presence of lattice defects like vacancies in the Mott insulator induces a characteristic scattering of polaritons, which can be optically observed to monitor the lattice integrity. Our models can be applied to simulate and understand corresponding collective phenomena in solid crystals, where many effects are often masked by noise and disorder.