Emergent helical edge states in a hybridized three-dimensional topological insulator

Abstract

As the thickness of a three-dimensional (3D) topological insulator (TI) becomes comparable to the penetration depth of surface states, quantum tunneling between surfaces turns their gapless Dirac electronic structure into a gapped spectrum. Whether the surface hybridization gap can host topological edge states is still an open question. Herein, we provide transport evidence of 2D topological states in the quantum tunneling regime of a bulk insulating 3D TI BiSbTeSe2. Different from its trivial insulating phase, this 2D topological state exhibits a finite longitudinal conductance at ~2e2/h when the Fermi level is aligned within the surface gap, indicating an emergent quantum spin Hall (QSH) state. The transition from the QSH to quantum Hall (QH) state in a transverse magnetic field further supports the existence of this distinguished 2D topological phase. In addition, we demonstrate a second route to realize the 2D topological state via surface gap-closing and topological phase transition mechanism mediated by a transverse electric field. The experimental realization of the 2D topological phase in a 3D TI enriches its phase diagram and marks an important step toward functionalized topological quantum devices.