Mitigation of fiber nonlinearity using a digital coherent receiver

Coherent detection with receiver-based DSP has recently enabled the mitigation of fiber nonlinear effects. We investigate the performance benefits available from the backpropagation algorithm for polarization division multiplexed quadrature amplitude phase-shift keying (PDM-QPSK) and 16-state quadrature amplitude modulation (PDM-QAM16). The performance of the receiver using a digital backpropagation algorithm with varying nonlinear step size is characterized to determine an upper bound on the suppression of intrachannel nonlinearities in a single-channel system. The results show that for the system under investigation PDM-QPSK and PDM-QAM16 have maximum step sizes for optimal performance of 160 and 80 km, respectively. Whilst the optimal launch power is increased by 2 and 2.5 dB for PDM-QPSK and PDM-QAM16, respectively, the Q-factor is correspondingly increased by 1.6 and 1 dB, highlighting the importance of studying nonlinear compensation for higher level modulation formats. © 2010 IEEE