The progress of ultracold atoms renders numerous possibilities to investigate exotic magnetism and superfluidity, which are rarely observed in solid state systems. In this thesis, we will introduce two novel physical descriptions: "frustrated Cooper pairing" and "large-hyperfine spin physics". Geometric frustration in quantum magnetism refers to which magnetic interactions on different bonds cannot be simultaneously minimized, and usually Cooper pairing favors uniform phases among different lattice sites. Here, we introduce "frustration" in Cooper pairing in a fermionic p-orbital model. By mean- field calculations, we show that the system exhibits behavior analogous to frustrated magnetism, and an unconventional supersolid state with the f-wave symmetry. Next, we introduce large-spin physics. In usual condensed matter systems, large spin is not intriguing because large values of spin suppress quantum fluctuations. In contrast, in ultracold fermion systems, large-hyperfine spin enhances quantum fluctuations and brings exotic quantum magnetism. Here the simplest large-spin fermionic system, a spin-3/2 exchange system is proposed, which can be characterized by an Sp(4)/SO(5) symmetry. In one dimension, the ground states exhibit either a dimerized state with a finite spin gap or a gapless spin liquid state by means of the density matrix renormalization group method. In the latter case, the spin-spin correlation functions are identified to have 4-site periodicities, which behaves similarly to the SU(4) chain. In two dimension, we infer that there exist three competing phases: Neel ordering, columnar dimerization and 2 x 2 plaquette formation, in the thermodynamic limit by exact diagonalization calculation on small sizes. Finally we perform the projector Quantum Monte Carlo method to study another large-spin system: the half-filled SU(N) Hubbard model. We show that at half-filling there is no sign problem such that our simulations are accurate. By finite size scaling, it is clearly found that the magnetic Neel ordering can exist not only for N = 2 but also in the N = 4 case at strong interactions. For N ̲> 6 or N = 4 at small U, the numerical results do not have any prominent signal that the long-range ordering exists in the thermodynamic limit. Due to strong finite size effects and finite numerical accuracy, however, we are unable to make any conclusion to identify the physics in the regimes
In this thesis, we investigate strongly correlated phenomena in quantum spin systems. In the first part of this work, we study geometrically frustrated antiferromagnets (AFMs). Generalizing the SU(2) Heisenberg Hamiltonian to Sp(N) symmetry, we obtain, in the large-N limit, the mean-field phase diagrams for the planar pyrochlore and cubic AFMs. We then use gauge theories to consider fluctuation effects about their respective mean-field configurations. We find, in addition to conventional Neel states, a plethora of novel magnetically disordered phases: two kinds of spin liquids, Z2 in 2+1D and U(1)in 3+1D, and several valence bond solids such as two and three-dimensional plaquette and columnar singlet states. We use the same approach to study the diamond lattice AFM which possesses extended classical ground state degeneracy. We demonstrate that quantum and entropic fluctuations lift this degeneracy in different ways. In the second part of the thesis, we study ultracold spinor atoms confined in optical lattices. We first demonstrate the feasibility of experimental realization of rotor models using ultracold spin-one Bose atoms in a spin-dependent and disordered optical lattice. We show that the ground state of such disordered rotor models with quadrupolar interactions can exhibit biaxial nematic ordering in the disorder-averaged sense, and suggest an imaging experiment to detect the biaxial nematicity in such systems. Finally, using variational wavefunction methods, we study the Mott phases and superfluid-insulator transition of spin-three bosons in an optical lattice with an anisotropic two dimensional optical trap. We chart out the phase diagrams for Mott states with n = 1 and n = 2 atoms per lattice site. We show that the long-range dipolar interaction stabilizes a state characterized by antiferromagnetic chains made of ferromagnetically aligned spins. We also obtain the mean-field phase boundary for the superfluid-insulator transition, and show that inside the superfluid phase and near the superfluid-insulator phase boundary, the system undergoes a first order antiferromagnetic-ferromagnetic spin ordering transition.
Geometric frustration in quantum magnetism refers to the fact that magnetic interactions on different bonds cannot be simultaneously minimized. The usual Cooper pairing systems favor uniform spatial distributions of pairing phases among different lattice sites without frustration. In contrast, we propose ``frustrated Cooper pairing'' in non-bipartite lattices which leads to supersolid states of Cooper pairs. Not only the amplitudes of the pairing order parameter but also its signs vary from site to site. This exotic pairing state naturally occurs in the
p-orbital bands in optical lattices with ultracold spinless fermions. In the triangular lattice, it exhibits an unconventional supersolid state with the
f-wave symmetry.
We numerically study the quantum magnetism of ultracold alkali and alkaline-earth fermion systems with large hyperfine spin
F=23, which are characterized by a generic
Sp(N) symmetry with
N=4. The methods of exact diagonalization (ED) and density matrix renormalization group are employed for the large size one-dimensional (1D) systems, and the ED method is applied to a two-dimensional (2D) square lattice on small sizes. We focus on the magnetic exchange models in the Mott-insulating state at quarter-filling. Both 1D and 2D systems exhibit rich phase diagrams depending on the ratio between the spin exchanges
J0 and
J2 in the bond spin singlet and quintet channels, respectively. In one dimension, the ground states exhibit a long-range-ordered dimerization with a finite spin gap at
J0/J2>1 and a gapless spin-liquid state at ${J}_{0}/{J}_{2}\ensuremath{\leqslant}1$, respectively. In the former and latter cases, the correlation functions exhibit the two-site and four-site periodicities, respectively. In two-dimensions, various spin-correlation functions are calculated up to the size of $4\ifmmode\times\else\texttimes\fi{}4$. The N\'eel-type spin correlation dominates at large values of
J0/J2, while a $2\ifmmode\times\else\texttimes\fi{}2$ plaquette correlation is prominent at small values of this ratio. Between them, a columnar spin-Peierls dimerization correlation peaks. We infer the competition among the plaquette ordering, the dimer ordering, and the N\'eel ordering in the 2D system.
Optical traps and lattices provide a new opportunity to study strongly correlated high spin systems with cold atoms. In this article, we review the recent progress on the hidden symmetry properties in the simplest high spin fermionic systems with hyperfine spin F=3/2, which may be realized with atoms of 132 Cs , 9 Be , 135 Ba , 137 Ba , and 201 Hg . A generic SO(5) or isomorphically, Sp(4) symmetry is proved in such systems with the s-wave scattering interactions in optical traps, or with the on-site Hubbard interactions in optical lattices. Various important features from this high symmetry are studied in the Fermi liquid theory, the mean field phase diagram, and the sign problem in quantum Monte-Carlo simulations. In the s-wave quintet Cooper pairing phase, the half-quantum vortex exhibits the global analogue of the Alice string and non-Abelian Cheshire charge properties in gauge theories. The existence of the quartetting phase, a four-fermion counterpart of the Cooper pairing phase, and its competition with other orders are studied in one-dimensional spin-3/2 systems. We also show that counter-intuitively quantum fluctuations in spin-3/2 magnetic systems are even stronger than those in spin-1/2 systems.
In the past few decades, tremendous efforts have been made toward understanding the exotic physics emerging from competition between various ordering tendencies in strongly correlated systems. Employing state-of-the-art quantum Monte Carlo simulation, we investigate an interacting SU(N) fermionic model with varying interaction strength and value of N, and we unveil the ground-state phase diagram of the model exhibiting a plethora of exotic phases. For small values of N-namely, N=2, 3-the ground state is an antiferromagnetic (AFM) phase, whereas in the large-N limit, a staggered valence bond solid (VBS) order is dominant. For intermediate values of N such as N=4, 5, remarkably, our study reveals that distinct VBS orders appear in the weak and strong coupling regimes. More fantastically, the competition between staggered and columnar VBS ordering tendencies gives rise to a Mott insulating phase without spontaneous symmetry breaking (SSB), existing in a large interacting parameter regime, which is consistent with a gapped quantum spin liquid. Our study not only provides a platform to investigate the fundamental physics of quantum many-body systems-it also offers a novel route toward searching for exotic states of matter such as quantum spin liquid in realistic quantum materials.
We investigate the ground-state properties of two strongly-correlated systems: the two-dimensional
t−J model that can be used to study the high-temperature superconducting phase in the doped antiferromagnet, and the frustrated N\'eel antiferromagnet described by the
J1−J2 Heisenberg model on the square lattice, which is widely considered as the prototype model for spin frustration. In this thesis, we apply to these two systems state-of-the-art quantum Monte Carlo techniques, including the variational and the Green's function Monte Carlo with the fixed-node approximation. Few Lanczos steps are used to systematically improve the accuracy of the trial wave functions. By introducing a suitable regularization scheme for the variational calculations with few Lanczos steps, stable and controllable simulations can be performed up to very large cluster sizes with very good accuracy. In the two-dimensional
t−J model at
J/t=0.4, we show that the accuracy of the Gutzwiller-projected variational state (containing
dx2−y2 pairing) can be improved much by few Lanczos steps; in addition, the fixed-node Monte Carlo with these systematically improvable trial wave functions gives results that are comparable with the best accurate DMRG ones. Our main outcome is that the ground state is homogeneous and no evidence of stripes is detected around the doping
δ=1/8. Indeed, our best approximation to the ground state does not show any tendency towards charge inhomogeneity. Furthermore, our results show that a uniform state containing superconductivity and antiferromagnetism is stabilized at low hole doping, i.e.,
δ≲0.1. In the
J1−J2 Heisenberg model on the square lattice, we use the projected mean-field state that is built from Abrikosov fermions having a
Z2 gauge structure and four Dirac points in the spinon spectrum. No spin or dimer order is found in the strongly frustrated regime and our calculations imply that a spin liquid phase may faithfully represent the exact ground state around
J2/J1=0.5. The few Lanczos step technique is used to systematically improve the accuracy of the variational states both for the ground state and for few relevant low-energy excitations. This procedure allows us to estimate, in a valuable and systemic way, the spin gaps within thermodynamical limit and to show a solid evidence of an unconventional gapless excitation spectrum in the strongly frustrated regime, i.e.,
J2/J1≃0.5. In particular, we found gapless triplet excitations at momenta
(π,0) and
(0,π), which are compatible with the presence of four Dirac points at momenta
(±2π,±2π) in the spinon spectrum.