Geometric analysis on small unitary representations of GL(N,R)

AbstractThe most degenerate unitary principal series representations πiλ,δ (λ∈R, δ∈Z/2Z) of G=GL(N,R) attain the minimum of the Gelfand–Kirillov dimension among all irreducible unitary representations of G. This article gives an explicit formula of the irreducible decomposition of the restriction πiλ,δ|H (branching law) with respect to all symmetric pairs (G,H). For N=2n with n⩾2, the restriction πiλ,δ|H remains irreducible for H=Sp(n,R) if λ≠0 and splits into two irreducible representations if λ=0. The branching law of the restriction πiλ,δ|H is purely discrete for H=GL(n,C), consists only of continuous spectrum for H=GL(p,R)×GL(q,R) (p+q=N), and contains both discrete and continuous spectra for H=O(p,q) (p>q⩾1). Our emphasis is laid on geometric analysis, which arises from the restriction of ‘small representations’ to various subgroups