Nonlinear mode coupling in rotating stars and the r-mode instability in neutron stars

We develop the formalism required to study the nonlinear interaction of modes in rotating Newtonian stars, assuming that the mode amplitudes are only mildly nonlinear. The formalism is simpler than previous treatments of mode-mode interactions for spherical stars, and simplifies and corrects previous treatments for rotating stars. At linear order, we elucidate and extend slightly a formalism due to Schutz, show how to decompose a general motion of a rotating star into a sum over modes, and obtain uncoupled equations of motion for the mode amplitudes under the influence of an external force. Nonlinear effects are added perturbatively via three-mode couplings, which suffices for moderate amplitude modal excitations; the formalism is easy to extend to higher order couplings. We describe a new, efficient way to compute the modal coupling coefficients, to zeroth order in the stellar rotation rate, using spin-weighted spherical harmonics. The formalism is general enough to allow computation of the initial trends in the evolution of the spin frequency and differential rotation of the background star. We apply this formalism to derive some properties of the coupling coefficients relevant to the nonlinear interactions of unstable r modes in neutron stars, postponing numerical integrations of the coupled equations of motion to a later paper. First, we clarify some aspects of the expansion in stellar rotation frequency Ω that is often used to compute approximate mode functions. We show that, in zero-buoyancy stars, the rotational modes (those modes whose frequencies vanish as Ω → 0) are orthogonal to zeroth order in Ω. From an astrophysical viewpoint, the most interesting result of this paper is that many couplings of r modes to other rotational modes are small: either they vanish altogether because of various selection rules, or they vanish to lowest order in Ω or in compressibility. In particular, in zero-buoyancy stars, the coupling of three r modes is forbidden entirely and the coupling of two r modes to one hybrid, or r-g rotational, mode vanishes to zeroth order in rotation frequency. The coupling of any three rotational modes vanishes to zeroth order in compressibility and in Ω. In nonzero-buoyancy stars, coupling of the r modes to each other vanishes to zeroth order in Ω. Couplings to regular modes (those modes whose frequencies are finite in the limit Ω → 0),such as f modes, are not zero, but since the natural frequencies of these modes are relatively large in the slow rotation limit compared to those of the r modes, energy transfer to those modes is not expected to be efficient