Polarization-ring-switching for nonlinearity-tolerant geometrically-shaped four-dimensional formats maximizing generalized mutual information

In this paper, a new four-dimensional 64-ary polarization ring switching (4D-64PRS) modulation format with a spectral efficiency of $6$~bit/4D-sym is introduced. The format is designed by maximizing the generalized mutual information (GMI) and by imposing a constant-modulus on the 4D structure. The proposed format yields an improved performance with respect to state-of-the-art geometrically shaped modulation formats for bit-interleaved coded modulation systems at the same spectral efficiency. Unlike previously published results, the coordinates of the constellation points and the binary labeling of the constellation are \emph{jointly} optimized. When compared against polarization-multiplexed 8-ary quadrature-amplitude modulation (PM-8QAM), gains of up to $0.7$~dB in signal-to-noise ratio are observed in the additive white Gaussian noise (AWGN) channel. For a long-haul nonlinear optical fiber system of $8,000$~km, gains of up to $0.27$~bit/4D-sym ($~5.5$\% data capacity increase) are observed. These gains translate into a reach increase of approximately $16\%$ ($1,100$~km). The proposed modulation format is also shown to be more tolerant to nonlinearities than PM-8QAM. Results with LDPC codes are also presented, which confirm the gains predicted by the GMI