We study instabilities of single-species fermionic atoms in the
p-orbital bands in two-dimensional optical lattices at noninteger filling against interactions. Charge density wave and orbital density wave orders with stripe or checkerboard patterns are found for attractive and repulsive interactions, respectively. The superfluid phase, usually expected of attractively interacting fermions, is strongly suppressed. We also use field theory to analyze the possible phase transitions from orbital stripe order to liquid-crystal phases and obtain the phase diagram. The condition of nearly-perfect Fermi-surface nesting, which is key to the above results, is shown robustly independent of fermion fillings in such
p-orbital systems, and the $(2{k}_{F},\ifmmode\pm\else\textpm\fi{}2{k}_{F})$ momentum of density wave oscillation is highly tunable. Such remarkable features show the promise of making those exotic orbital phases, which are of broad interest in condensed-matter physics, experimentally realizable with optical lattice gases.
We study two-band effects on ultracold fermionic atoms in optical lattices by means of dynamical mean-field theory. We find that at half filling an atomic-density-wave (ADW) state emerges owing to the two-band effects in the attractive interaction region, while an antiferromagnetic state appears in the repulsive interaction region. As the orbital splitting is increased, quantum phase transitions from the ADW state to the superfluid state and from the antiferromagnetic state to the metallic state occur in the corresponding regions. By systematically changing the orbital splitting and the interaction, we obtain the phase diagram at half filling. The results are discussed using the effective boson model derived for strong attractive interaction.
In this thesis, we discuss ultracold quantum gases both in continuum and optical lattices. For the continuum Fermi gases in BCS-BEC crossover, we present an effective field theory study on the recently discovered puzzling damping phenomena on the BCS side of the crossover. We find that in contrast to the previous proposed pair-breaking mechanism of damping, the damping process is due to the interaction between superfluid phonons and thermally excited fermionic quasi particles. Results from our effective field theory are compared quantitatively with experiments, showing a good agreement. For the ultracold fermionic atoms in optical lattices, we propose two novel quantum phases. Firstly, we show that a novel superconducting pairing occurs for spin-imbalanced Fermi gases with the spin up and down Fermi levels lying within the px- and s- orbital bands of a quasi-one-dimensional optical lattice. The pairs condense at a finite momentum equal to the sum of the two Fermi momenta of spin up and down fermions, and form a p-orbital condensate. The phase diagram shows that the p-orbital pair condensate occurs in a wide range of fillings. Secondly, we study instabilities of single-species fermionic atoms in the p-orbital bands in two-dimensional square optical lattices. From the nearly-perfect nesting Fermi surfaces, charge density wave and orbital density wave orderings with stripe or checkerboard patterns are found for attractive and repulsive interactions, respectively. The superconducting phase, usually expected of attractively interacting fermions, is strongly suppressed. We also use field theory to analyze the possible liquid crystal phases in our system. For bosons, we study ultracold bosonic atoms loaded in a one-dimensional optical lattice of two-fold p-orbital degeneracy at each site, and find an anti-ferro-orbital, a homogeneous px Mott insulator phase and two kinds of superfluid phases distinguished by the orbital ordering.