Competing instabilities of the extended Hubbard model on the triangular lattice: Truncated-unity functional renormalization group and application to moir\'e materials

A simple yet paradigmatic model for the interplay of strong electronic correlations and geometric frustration is the triangular lattice Hubbard model. Recently it was proposed that moir\'e structures of transition metal dichalcogenides can be used to simulate extended versions that include non-local density-density interactions. We study competing instabilities of interacting electrons in such an extended Hubbard model on the triangular lattice near a filling where the density of states has a Van Hove singularity. We employ a truncated-unity functional renormalization group approach to investigate two cases: a paradigmatic minimally extended Hubbard model and a specific model with parameters that are applicable to hetero-bilayers of transition metal dichalcogenides. We unravel rich phase diagrams, including tendencies to spin-density-wave order and unconventional pairing, which can give rise to topological superconductivity. We classify the symmetry of the superconducting instabilities according to their irreducible representations and show that higher lattice harmonics are dominant when the nearest-neighbor interaction is sizable indicating pair formation between second-nearest neighbors. The phenomenological consequences can be enhanced spin and thermal quantum Hall responses in a topological superconductor.Comment: 11+7 pages, 11+2 figure