Competing spin-liquid states in the spin- 12 Heisenberg model on the triangular lattice

Abstract

We study the spin-12 Heisenberg model on the triangular lattice with antiferromagnetic first- (J1) and second- (J2) nearest-neighbor interactions using density matrix renormalization group. By studying the spin correlation function, we find a 120 magnetic order phase for J20.07J1 and a stripe antiferromagnetic phase for J20.15J1. Between these two phases, we identify a spin-liquid region characterized by exponential decaying spin and dimer correlations, as well as large spin singlet and triplet excitation gaps on finite-size systems. We find two near degenerating ground states with distinct properties in two sectors, which indicates more than one spin-liquid candidate in this region. While the sector with spinons is found to respect time reversal symmetry, the even sector without spinons breaks such a symmetry for finite-size systems. Furthermore, we detect the signature of the fractionalization by following the evolution of different ground states with inserting spin flux into the cylinder system. Moreover, by tuning the anisotropic bond coupling, we explore the nature of the spin-liquid phase and find the optimal parameter region for gapped Z2 spin liquids.