Two Higgs bosons near 125 GeV in the NMSSM: beyond the narrow width approximation

Abstract In the next-to-minimal supersymmetric (NMS) Standard Model (SM), it is possible for either one of the additional singlet-like scalar and pseudoscalar Higgs bosons to be almost degenerate in mass with the $$\sim $$ ∼ $$125$$ 125 GeV SM-like Higgs state. In the real NMSSM (rNMSSM), when the mass difference between two scalar states is comparable to their individual total decay widths, the quantum mechanical interference, due to the relevant diagonal as well as off-diagonal terms in the propagator matrix, between them can become sizeable. This possibility invalidates usage of the narrow width approximation (NWA) to compute the cross section for the production of a di-photon pair with a given invariant mass via resonant Higgs boson(s) in the gluon fusion process at the Large Hadron Collider (LHC). When, motivated by the baryon asymmetry of the universe, CP-violating (CPV) phases are explicitly invoked in the Higgs sector of the NMSSM, all the interaction eigenstates mix to give five CP-indefinite physical Higgs bosons. In this scenario, the interference effects due to the off-diagonal terms in the Higgs mass matrix that mix the pseudoscalar-like state with the SM-like one can also become significant, when these two are sufficiently mass-degenerate. We perform a detailed analysis, in both the real and complex NMSSM, of these interference effects, when the full propagator matrix is taken into account, in the production of a photon pair with an invariant mass near 125 GeV through gluon fusion. We find that these effects can account for up to $$\sim $$ ∼ $$40$$ 40 % of the total cross section for certain model parameter configurations. We also investigate how such mutually interfering states contributing to the $$\sim $$ ∼ $$125$$ 125 GeV signal observed at the LHC can be distinguished from a single resonance