On Signal Constellations and Coding for Long-Haul Fiber-Optical Systems

Motivated by the realization that even the enormous bandwidth available in an optical fiber is finite and valuable, the design of spectrally efficient long-haul fiber-optical communication systems has become an important research topic. Compared to other wireline technologies, e.g., transmission over coaxial cables, the main challenge comes from the inherent nonlinearity of the underlying communication channel caused by the relatively high signal intensities. In this thesis, we study the design of spectrally efficient fiber-optical systems for both uncoded and coded transmission scenarios. We consider the problem of designing higher-order signal constellations for a system that is severly impaired by nonlinear phase noise. By optimizing amplitude phase-shift keying constellations, which can be seen as the union of phase-shift keying constellation with different amplitude levels, gains of up to 3.2 dB at a symbol error probability of 10^(−2) are shown to be achievable compared to conventional constellations. We also illustrate a somewhat counterintuitive behavior of optimized constellations for very high input powers and nonlinear distortions. In particular, sacrificing a constellation point or ring may be beneficial in terms of the overall performance of the constellation. Furthermore, we study polarization-multiplexed transmission, where spectral efficiency is increased by encoding data onto both polarizations of the light. For a memoryless fiber-optical channel, we introduce a low-complexity detector which is based on an amplitude- dependent phase rotation and subsequent threshold detection. The complexity compared to the four-dimensional maximum likelihood detector is considerably reduced, albeit at the expense of some performance loss. Lastly, we consider the design of a coded fiber-optical system operating at high spectral efficiency. In particular, we study the optimization of the mapping of the coded bits to the modulation bits for a polarization-multiplexed fiber-optical system that is based on the bit-interleaved coded modulation paradigm. This technique, which we refer to as bit mapper optimization, is extended to the class of spatially coupled low-density parity- check codes, which have shown outstanding performance over memoryless binary-input channels. For a transmission scenario without optical inline dispersion compensation, the results show that the transmission reach can be extended by roughly up to 8%, without significantly increasing the system complexity