Three-dimensional (3D) variants of the Kitaev model can harbor gapless spin liquids with a Majorana Fermi surface on certain tricoordinated lattice structures such as the recently introduced hyperoctagon lattice. Here, we investigate Fermi surface instabilities arising from additional spin exchange terms (such as a Heisenberg coupling) which introduce interactions between the emergent Majorana fermion degrees of freedom. We show that independent of the sign and structure of the interactions, the Majorana surface is always unstable. Generically, the system spontaneously doubles its unit cell at exponentially small temperatures and forms a spin liquid with line nodes. Depending on the microscopics, further symmetries of the system can be broken at this transition. These spin-Peierls instabilities of a 3D spin liquid are closely related to BCS instabilities of fermions.
Motivated by the recent synthesis of $\ensuremath{\beta}$-Li
2IrO
3---a spin-orbit entangled
j=1/2 Mott insulator with a three-dimensional lattice structure of the Ir
4+ ions---we consider generalizations of the Kitaev model believed to capture some of the microscopic interactions between the iridium moments on various trivalent lattice structures in three spatial dimensions. Of particular interest is the so-called hyperoctagon lattice---the premedial lattice of the hyperkagome lattice, for which the ground state is a gapless quantum spin liquid where the gapless Majorana modes form an extended two-dimensional Majorana Fermi surface. We demonstrate that this Majorana Fermi surface is inherently protected by lattice symmetries and discuss possible instabilities. We thus provide the first example of an analytically tractable microscopic model of interacting SU(2) spin-
21 degrees of freedom in three spatial dimensions that harbors a spin liquid with a two-dimensional spinon Fermi surface.
The excitation spectrum of Kitaev quantum spin liquids consists of itinerant Majorana fermions, which can be gapless or gapped, and vortices of a
Z2 gauge field, known as visons, which are \new{gapped within a stable
Z2 spin liquid phase}. In this work, we investigate visons in Kitaev-type models where the Majorana fermions form a Fermi surface. In this case, the creation of a vison pair is analogous to introducing a local impurity potential in a metal. Since the gapless modes lead to strong finite-size effects, we compare the numerical calculation of the \new{two-vison gap} on finite lattices with the result from an analytical approach based on Green's function techniques. We find that the \new{two-vison gap} decreases as the size of the Fermi surface increases, signalling an instability of the quantum spin liquid ground state. We also show that larger Fermi surfaces tend to suppress the change in local spin correlations due to the Majorana-vison scattering potential.