Accurate probabilistic treatment of bit-pattern-dependent nonlinear distortions in BER calculations for

Abstract—We introduce a fully deterministic computationally efficient method for characterizing the effect of nonlinearity in optical fiber transmission systems that utilize wavelength division multiplexing and return-to-zero (RZ) modulation. The method is based on a probability analysis of the bit patterns, and it accurately accounts for bit-pattern-dependence effect due to both nonlinearly induced amplitude and timing jitter. We apply this method in calculating the error probability in a prototypical mul-tichannel RZ undersea system in the presence of both nonlinear distortion and amplifier noise. Index Terms—Communication system nonlinearities, communi-cation system performance, error analysis, optical fiber communi-cation, optical propagation in nonlinear media. I