We explore ordering tendencies in weakly coupled Hubbard and t-J models on the two-dimensional square lattice for fermions with N flavors. Such systems may be realized using ultracold atoms with several hyperfine states in optical lattices. The analysis is based on a functional renormalization group approach. In the repulsive Hubbard model for N > 6 and half band filling staggered flux order is found as the dominant instability, in agreement with the large-N limit. In the t-J model without occupancy constraint this type of ground state is suggested already for N > 2. In the attractive case and odd N a full Fermi surface coexists with a superfluid in the ground state. We describe how these unconventional paired states can be identified in the density response of the fermions.
We investigate the fermionic SU(N) Hubbard model on the two-dimensional square lattice for weak to moderate interactions using renormalization group and mean-field methods. For the repulsive case U>0 at half filling and small N the dominant tendency is towards breaking of the SU(N) symmetry. For N>6 staggered flux order takes over as the dominant instability, in agreement with the large-N limit. Away from half filling for N=3 two flavors remain half filled by cannibalizing the third flavor. For U<0 and odd N a full Fermi surface coexists with a superconductor. These results may be relevant to future experiments with cold fermionic atoms in optical lattices.
We study fermionic superfluidity in an ultracold Bose-Fermi mixture loaded into a square optical lattice subjected to a staggered flux. While the bosons form a Bose-Einstein condensate at very low temperature and weak interaction, the interacting fermions experience an additional long-ranged attractive interaction mediated by phonons in the bosonic condensate. This leads us to consider a generalized Hubbard model with on-site and nearest-neighbor attractive interactions, which give rise to two competing pairing channels. We use the Bardeen-Cooper-Schrieffer theory to determine the regimes where distinct fermionic superfluids are stabilized and find that the nonlocal pairing channel favors a superfluid state which breaks both the gauge and the lattice symmetries, similar to unconventional superconductivity occurring in some strongly correlated systems. Furthermore, the particular structure of the single-particle spectrum leads to unexpected consequences, for example, a dome-shaped superfluid region in the temperature versus filing fraction phase diagram, with a normal phase that contains much richer physics than a Fermi liquid. Notably, the relevant temperature regime and coupling strength are readily accessible in state of the art experiments with ultracold trapped atoms.
The progress of ultracold atoms renders numerous possibilities to investigate exotic magnetism and superfluidity, which are rarely observed in solid state systems. In this thesis, we will introduce two novel physical descriptions: "frustrated Cooper pairing" and "large-hyperfine spin physics". Geometric frustration in quantum magnetism refers to which magnetic interactions on different bonds cannot be simultaneously minimized, and usually Cooper pairing favors uniform phases among different lattice sites. Here, we introduce "frustration" in Cooper pairing in a fermionic p-orbital model. By mean- field calculations, we show that the system exhibits behavior analogous to frustrated magnetism, and an unconventional supersolid state with the f-wave symmetry. Next, we introduce large-spin physics. In usual condensed matter systems, large spin is not intriguing because large values of spin suppress quantum fluctuations. In contrast, in ultracold fermion systems, large-hyperfine spin enhances quantum fluctuations and brings exotic quantum magnetism. Here the simplest large-spin fermionic system, a spin-3/2 exchange system is proposed, which can be characterized by an Sp(4)/SO(5) symmetry. In one dimension, the ground states exhibit either a dimerized state with a finite spin gap or a gapless spin liquid state by means of the density matrix renormalization group method. In the latter case, the spin-spin correlation functions are identified to have 4-site periodicities, which behaves similarly to the SU(4) chain. In two dimension, we infer that there exist three competing phases: Neel ordering, columnar dimerization and 2 x 2 plaquette formation, in the thermodynamic limit by exact diagonalization calculation on small sizes. Finally we perform the projector Quantum Monte Carlo method to study another large-spin system: the half-filled SU(N) Hubbard model. We show that at half-filling there is no sign problem such that our simulations are accurate. By finite size scaling, it is clearly found that the magnetic Neel ordering can exist not only for N = 2 but also in the N = 4 case at strong interactions. For N ̲> 6 or N = 4 at small U, the numerical results do not have any prominent signal that the long-range ordering exists in the thermodynamic limit. Due to strong finite size effects and finite numerical accuracy, however, we are unable to make any conclusion to identify the physics in the regimes
In this thesis, we discuss ultracold quantum gases both in continuum and optical lattices. For the continuum Fermi gases in BCS-BEC crossover, we present an effective field theory study on the recently discovered puzzling damping phenomena on the BCS side of the crossover. We find that in contrast to the previous proposed pair-breaking mechanism of damping, the damping process is due to the interaction between superfluid phonons and thermally excited fermionic quasi particles. Results from our effective field theory are compared quantitatively with experiments, showing a good agreement. For the ultracold fermionic atoms in optical lattices, we propose two novel quantum phases. Firstly, we show that a novel superconducting pairing occurs for spin-imbalanced Fermi gases with the spin up and down Fermi levels lying within the px- and s- orbital bands of a quasi-one-dimensional optical lattice. The pairs condense at a finite momentum equal to the sum of the two Fermi momenta of spin up and down fermions, and form a p-orbital condensate. The phase diagram shows that the p-orbital pair condensate occurs in a wide range of fillings. Secondly, we study instabilities of single-species fermionic atoms in the p-orbital bands in two-dimensional square optical lattices. From the nearly-perfect nesting Fermi surfaces, charge density wave and orbital density wave orderings with stripe or checkerboard patterns are found for attractive and repulsive interactions, respectively. The superconducting phase, usually expected of attractively interacting fermions, is strongly suppressed. We also use field theory to analyze the possible liquid crystal phases in our system. For bosons, we study ultracold bosonic atoms loaded in a one-dimensional optical lattice of two-fold p-orbital degeneracy at each site, and find an anti-ferro-orbital, a homogeneous px Mott insulator phase and two kinds of superfluid phases distinguished by the orbital ordering.