Behavior of the generalized Rosenblatt process at extreme critical exponent values

The generalized Rosenblatt process is obtained by replacing the single critical exponent characterizing the Rosenblatt process by two different exponents living in the interior of a triangular region. What happens to that generalized Rosenblatt process as these critical exponents approach the boundaries of the triangle? We show by two different methods that on each of the two symmetric boundaries, the limit is non-Gaussian. On the third boundary, the limit is Brownian motion. The rates of convergence to these boundaries are also given. The situation is particularly delicate as one approaches the corners of the triangle, because the limit process will depend on how these corners are approached. All limits are in the sense of weak convergence in C[0,1]. These limits cannot be strengthened to convergence in L2(Ω).Supported in part by NSF Grants DMS-10-07616 and DMS-13-09009 at Boston University. (DMS-10-07616 - NSF at Boston University; DMS-13-09009 - NSF at Boston University)Accepted manuscrip