Using a counter rotating parallel plate shear flow cell, the shape relaxation of deformed droplets in a quiescent matrix is studied microscopically. Both the effects of geometrical confinement and component viscoelasticity are systematically explored at viscosity ratios of 0.45 and 1.5. The flow conditions are varied from a rather low to a nearly critical Ca number. Under all conditions investigated, viscoelasticity of the droplet phase has no influence on shape relaxation, whereas matrix viscoelasticity and geometrical confinement result in a slower droplet retraction. Up to high confinement ratios, the relaxation curves for ellipsoidal droplets can be superposed onto a master curve. Confined droplets with a sigmoidal shape relax in two st...