High efficiency, high brightness X-ray free electron laser pulses via fresh bunch self-seeding and undulator tapering

The advent of the X-ray Free Electron Laser (XFEL) as the world's brightest light source has opened the door to the study of natural phenomena occurring on Angstrom level spatial scales and femtosecond time intervals. After more than five years of successful operation, there is a demand from the scientific community to further increase the brightness and peak power one hundred times more than the state-of-the art to reach TW power X-ray pulses in the next generation of XFELs.The central subject of this dissertation is the study of strategies to achieve high efficiency, high power XFEL pulses via optimized undulator tapering and the novel technique of fresh bunch self-seeding which we demonstrate experimentally in this work. Our work begins with theoretical and numerical investigations of high efficiency XFELs in which we determine the primary factors which prevent the efficiency from reaching values on the order of 10 %, which correspond to TW-power pulses. The initial studies focus on the dependence of the peak power of a TW-level self-seeded tapered XFEL on the transverse electron distribution. We show via numerical simulations that more uniform transverse distributions (parabolic or flat) can increase the efficiency compared to transversely Gaussian beams from 2.8 % to 4.8 %, corresponding to an increase in peak power from 1.6 TW to 2.6 TW in a 200 m undulator. These numerical studies emphasize the contribution of two physical phenomena, diffraction and the synchrotron sideband instability, as the primary obstacles to reaching TW peak power in a compact undulator system. To circumvent these issues we propose two improvements for high extraction efficiency tapered XFELs: a new development in FEL hardware through the proposal of an Advanced Gradient Undulator (AGU), and a new technique for the efficient generation of high power XFEL radiation which we name fresh bunch self-seeding. We analyze in detail the performance of a high efficiency AGU by carrying out the first dedicated comparative numerical optimization study of a tapered FEL including and intentionally disabling time dependent effects. Combining the AGU design with an optimized undulator taper and a fresh bunch self-seeded system yields a peak efficiency in simulation of 12 % with 6.3 TW of peak power in a 100 m undulator. Following the numerical and theoretical studies of tapered XFELs, the remainder of the work describes the first experimental demonstration of fresh bunch self seeding in an XFEL. With this scheme we are able to generate high power (50 GW) narrow bandwidth ($\delta \omega/\omega_0=8\times 10^{-5}$) short X-ray pulses ($\delta t<10$ fs), well suited for X-ray diffraction imaging experiments and nonlinear science applications. We compare the performance of the fresh bunch self-seeding scheme with the established methods of Self-Amplified Spontaneous Emission (SASE) and regular self-seeding, measuring a relative brightness increase of a factor of 12.5 and a factor of 2.4 respectively. The results are a promising step forward towards enabling TW-level X-ray FEL pulses in future XFEL light sources