Interplay of magnetic, orthorhombic, and superconducting phase transitions in iron-based superconductors

The physics of iron pnictides has been the subject of intense research for half a decade since the discovery of superconductivity in doped LaFeAsO in 2008. By now there exists a large number of different materials that are summarized under the term "pnictides'' with significant differences in their crystal structure, electronic properties, and their phase diagrams. This thesis is concerned with the investigation of the various phase transitions that are observed in the underdoped compounds of the pnictide subgroups RFeAsO, where R is a rare-earth element, and AFe_2As_2, where A is an alkaline-earth element. These compounds display two closely bound transitions from a tetragonal to an orthorhombic phase and from a paramagnetic to an antiferromagnetic metal. Both symmetry-broken phases are suppressed by doping or pressure and close to their disappearance superconductivity sets in. The superconducting state is stabilized until some optimal doping or pressure is reached and gets suppressed thereafter. The central goal of this thesis is to improve our understanding of the interplay between these three phases and to describe the various phase transitions. We start from an itinerant picture that explains the magnetism as a result of an excitonic instability and show how the other phases can be included into this picture. This approach is based on the the observation that the compounds we are interested in have a Fermi surface with multiple nested electron and hole pockets and that they have small to intermediate interaction strengths. The thesis starts with a study of the doping dependence of the antiferromagnetic phase transition in four different five-orbital models. We use the random-phase approximation to determine the transition temperature, the dominant ordering vector, and the contribution of the different orbitals to the ordering. This allows us to identify the more realistic models, which give results that are in good agreement with experimental observations. In addition to the frequently made assumption of orbital-independent interaction potentials we study the effect of a reduction of the interaction strengths that involve the d_{xy} orbital. We find that this tunes the system between two different nesting instabilities. A reduction of the interactions that involve the d_{xy} orbital also enhances the tendency towards incommensurate (IC) order. For a weak reduction this tendency is compensated by the presence of the orthorhombic phase. However, for a reduction of 30%, as it is suggested by constrained random-phase-approximation calculations, we always find large doping ranges, where a state with IC order has the highest transition temperature. We continue the investigation of the magnetic phase transition by studying the competition of different possible types of antiferromagnetic order that arises from the presence of two degenerate nesting instabilities with the ordering vectors (pi,0) and (0,pi). We derive a Ginzburg-Landau free energy from a microscopic two-band model and find that the presence of the experimentally observed stripe phase strongly depends on the number and size of the hole pockets in the system and on the doping. We show that within the picture of a purely magnetically driven nematic phase transition, which breaks the C_4 symmetry and induces the orthorhombic distortion, the nematic phase displays exactly the same dependence on the model parameters as the magnetic stripe phase. We propose that in addition to the purely magnetically driven nematic instability there is a ferro-orbital instability in the system that stabilizes the nematic transition and, thus, explains the experimentally observed robustness of the orthorhombic transition. We argue that including a ferro-orbital instability into the picture may also be necessary to reproduce the transition from simultaneous first-order transitions into an orthorhombic antiferromagnetic state to two separate second-order transitions, which is observed as a function of doping. Finally, a study of the superconducting phase transition inside the antiferromagnetic phase that is observed in some pnictide compounds is presented. We present an approach to calculate the fluctuation-mediated pairing interaction in the spin-density-wave phase of a multiband system, which is based on the random-phase approximation. This approach is applied to a minimal two-band model for the pnictides to study the effect of the various symmetry-allowed bare on-site interactions on the gap symmetry and structure. We find a competition between various even- and odd-parity states and over a limited parameter range a p_x-wave state is the dominant instability. The largest part of the parameter space is dominated by even parity states but the gap structure sensitively depends on the bare interactions. We propose that the experimentally observed transition from a nodeless to a nodal gap can be due to changes in the on-site interaction potentials