Maps preserving the nilpotency of products of operators

This paper is dedicated to Professor Roger Horn on the occasion of his 65th birthday. Let B(X) be the algebra of all bounded linear operators on the Banach space X, and let N (X) be the set of nilpotent operators in B(X). Suppose φ: B(X) → B(X) is a surjective map such that A,B ∈ B(X) satisfy AB ∈ N (X) if and only if φ(A)φ(B) ∈ N (X). If X is infinite dimensional, then there exists a map f: B(X) → C \ {0} such that one of the following holds: (a) There is a bijective bounded linear or conjugate-linear operator S: X → X such that φ has the form A 7 → S[f(A)A]S−1. (b) The space X is reflexive, and there exists a bijective bounded linear or conjugate-linear operator S: X ′ → X such that φ has the form A 7 → S[f(A)A′]S−1. If X has dimension n with 3 ≤ n <∞, and B(X) is identified with the algebra Mn of n × n complex matrices, then there exist a map f: Mn → C \ {0}, a field automorphism ξ: C → C, and an invertible S ∈Mn such that φ has one of the following forms: A = [aij] 7 → f(A)S[ξ(aij)]S−1 or A = [aij] 7 → f(A)S[ξ(aij)]tS−1, where At denotes the transpose of A. The results are extended to the product of more than two operators and to other types of products on B(X) including the Jordan triple product A ∗B = ABA. Furthermore, the results in the finite dimensional case are used to characterize surjective maps on matrices preserving the spectral radius of products of matrices. AMS classification: 47B49