Physics

Topological insulators are materials that have a bulk band gap similar to commonly known insulators, but have conducting states on their edge or surface. The bulk band gap is generated because of the strong spin-orbit coupling inherent to these systems, which also modifies the band in a fundamental way, leading to unconventional spin-polarized Dirac fermions on the boundary of the insulator. In this thesis, we present our scanning tunneling spectroscopy studies of topological surface states in Bi1−xSbx, Sb, Bi2Te3 and Bi2Se3. Due to their helical spin texture, the surface states of topological insulators are distinct from the well-known conventional surface states of noble metals. A key predicted feature of these metallic surface states is their immunity to localization and ability to overcome barriers caused by material imperfections. These predictions can be examined by studying the scattering of surface states from random alloying disorder. We have mapped the interference of the surface states in Bi0.9Sb0.1 and demonstrated that despite strong atomic scale disorder, backscattering between states of opposite momentum and opposite spin is absent, resulting from the helical spin texture of the surface states. Furthermore, we have measured transmission and reflection of the topological surface states of Sb by atomic terraces. In contrast to surface state