Machine-learned phase diagrams of generalized Kitaev honeycomb magnets

Abstract

We use a recently developed interpretable and unsupervised machine-learning method, the tensorial kernel support vector machine, to investigate the low-temperature classical phase diagram of a generalized Heisenberg-Kitaev-$\mathrm{\ensuremath{\Gamma}}$ ($J\text{\ensuremath{-}}K\text{\ensuremath{-}}\mathrm{\ensuremath{\Gamma}}$) model on a honeycomb lattice. Aside from reproducing phases reported by previous quantum and classical studies, our machine finds a hitherto missed nested zigzag-stripy order and establishes the robustness of a recently identified modulated ${S}_{3}\ifmmode\times\else\texttimes\fi{}{Z}_{3}$ phase, which emerges through the competition between the Kitaev and $\mathrm{\ensuremath{\Gamma}}$ spin liquids, against Heisenberg interactions. The results imply that, in the restricted parameter space spanned by the three primary exchange interactions---$J$, $K$, and $\mathrm{\ensuremath{\Gamma}}$, the representative Kitaev material $\ensuremath{\alpha}\text{\ensuremath{-}}{\mathrm{RuCl}}_{3}$ lies close to the boundaries of several phases, including a simple ferromagnet, the unconventional ${S}_{3}\ifmmode\times\else\texttimes\fi{}{Z}_{3}$, and nested zigzag-stripy magnets. A zigzag order is stabilized by a finite ${\mathrm{\ensuremath{\Gamma}}}^{\ensuremath{'}}$ and/or ${J}_{3}$ term, whereas the four magnetic orders may compete in particular if ${\mathrm{\ensuremath{\Gamma}}}^{\ensuremath{'}}$ is antiferromagnetic.