We present a comprehensive analysis of the convergence properties of the frame operators of Weyl-Heisenberg systems and shift-invariant systems, and relate these to the convergence of the Walnut representation. We give a deep analysis of necessary conditions and sufficient conditions for convergence of the frame operator. We show that symmetric, norm and unconditional convergence of the Walnut series are all different, but that weak and norm convergence are the same, while there are WH-systems for which the Walnut representation has none of these convergence properties. We make a detailed study of the CC-condition (a sufficient condition for WH-systems to have finite upper frame bounds) and show that (for ab rational) a uniform version of t...