The effect of coherent coupling nonlinearity on modulation instability and rogue wave excitation

We study modulation instability (MI) in both anomalous and normal dispersion regimes of a coherently coupled system. It is found that there exist three types of MI spectra with distinct characteristics termed baseband, passband, and zero-baseband based on the instability analysis of the in-phase and out-of-phase CW solutions. The coherent coupling nonlinearity is the source of the passband and zero-baseband variants. Guided by analytical predictions, we investigate numerically the excitation of rogue waves on weakly perturbed in-phase and out-of-phase continuous wave solutions in the parameter space where different kinds of MI exist. Simulations provide supporting evidence that rogue waves can only emerge in regimes where baseband or zero-baseband MI occurs. Moreover, the peak intensity of rogue waves in the case of baseband MI is greater than in the zero-baseband case. Finally, a combination of analysis and numerics uncovers the parameter conditions necessary for the generation of rogue waves