Using first principles, we calculate the lattice thermal conductivity of Bi, Sb, and Bi-Sb alloys, which are of great importance for thermoelectric and thermomagnetic cooling applications. Our calculation reveals that the ninth-neighbor harmonic and anharmonic force constants are significant; accordingly, they largely affect the lattice thermal conductivity. Several features of the thermal transport in these materials are studied: (1) the relative contributions from phonons and electrons to the total thermal conductivity as a function of temperature are estimated by comparing the calculated lattice thermal conductivity to the measured total thermal conductivity, (2) the anisotropy of the lattice thermal conductivity is calculated and compar...