Anomalous and anisotropic nonlinear susceptibility in the proximate Kitaev magnet α-RuCl3

The leading order nonlinear (NL) susceptibility, χ3, in a paramagnet is negative and diverges as T → 0. This divergence is destroyed when spins correlate and the NL response provides unique insights into magnetic order. Dimensionality, exchange interaction, and preponderance of quantum effects all imprint their signatures in the NL magnetic response. Here, we study the NL susceptibilities in the proximate Kitaev magnet α-RuCl3, which differs from the expected antiferromagnetic behavior. For T 0 implies a broken sublattice symmetry of magnetic order at low temperatures. Classical Monte Carlo (CMC) simulations in the standard K − H − Γ model secure such a quadratic B dependence of M, only for T ≈ Tc with χ2 being zero as T → 0. It is also zero for all temperatures in exact diagonalization calculations. On the other hand, we find an exclusive cubic term (χ3) that describes the high field NL behavior well. χ3 is large and positive both below and above Tc crossing zero only for T > 50 K. In contrast, for B ∥ c-axis, no separate low/high field behaviors are measured and only a much smaller χ3 is apparent