Experiments featuring non-equilibrium glassy dynamics under temperature changes still await interpretation. There is a widespread feeling that temperature chaos (an extreme sensitivity of the glass to temperature changes) should play a major role but, up to now, this phenomenon has been investigated solely under equilibrium conditions. In fact, the very existence of a chaotic effect in the non-equilibrium dynamics is yet to be established. In this article, we tackle this problem through a large simulation of the 3D Edwards-Anderson model, carried out on the Janus II supercomputer. We find a dynamic effect that closely parallels equilibrium temperature chaos. This dynamic temperature-chaos effect is spatially heterogeneous to a large degree ...