On Nonlinear Compensation Techniques for Coherent Fiber-Optical Channel

Fiber-optical communication systems form the backbone of the internet, enabling global broadband data services. Over the past decades, the demand for high-speed communications has grown exponentially. One of the key techniques for the efficient use of existing bandwidth is the use of higher order modulation formats along with coherent detection. However, moving to high-order constellations requires higher input power, and thus leads to increased nonlinear effects in the fiber. In long-haul optical communications (distancesspanning from a hundred to a few thousands of kilometers), amplification of the signal is typically needed as the fibers exhibit power losses. Amplifiers add noise and the signal and noise interact, leading to nonlinear signal–noiseinteractions, which degrade the system performance.The propagation of light in an optical fiber is described by the nonlinear Schr\uf6dinger equation (NLSE). Due to the lack of analytical solutions for the NLSE, deriving statistics of this nonlinear channel is in general cumbersome. The state-of-the-art receiver for combating the impairments existing in a fiber-optical link is digital backpropagation (DBP), which inverts the NLSE, and is widely believed to be optimal. Following this optimality, DBP has enabled system designers to determine optimal transmission parameters and provides a benchmark against which other detectors are compared. However, a number of open questions remain: How is DBP affected by noise? With respect to which criterion is DBP optimal? Can we estimate the optimal transmit power for a system when DBP is used?In paper A, starting from basic principles in Bayesian decision theory, we consider the well-known maximum a posteriori (MAP) decision rule, a natural optimality criterion which minimizes the error probability. As the closed-form expressions required for MAP detection are not tractable for coherent optical transmission, we employ the framework of factor graphs and the sum-product algorithm, which allow a numerical evaluation of the MAP detector. The detector turns out to have similarities with DBP (which can be interpreted as a special case) and is termed stochastic digital backpropagation, as it accounts for noise, as well as nonlinear and dispersive effects. Through Monte Carlosimulations of a single-channel communication system, we see significant performance gains with respect to DBP for dispersion-managed links.In paper B, we investigate the performance limits of DBP for a non dispersion-managed fiber-optical link. An analytical expression is derived that can be used to find the optimal transmit power for a system when DBP is used. We found that a first-order approximation is reasonably tight for different symbol rates and it can be used to approximately compute the optimum transmit power in terms of minimizing the symbol error rate. Moreover, the first-order approximation results show that the variance of the nonlinear noise grows quadratically with transmitted power, which limits the performance of a system with DBP