A theorem of Anderson and Bando-Kasue-Nakajima from 1989 states that to compactify the set of normalized Einstein metrics with a lower bound on the volume and an upper bound on the diameter in the Gromov-Hausdorff sense, one has to add singular spaces called Einstein orbifolds, and the singularities form as blow-downs of Ricci-flat ALE spaces. This raises some natural issues, in particular: can all Einstein orbifolds be Gromov-Hausdorff limits of smooth Einstein manifolds? Can we describe more precisely the smooth Einstein metrics close to a given singular one? In this first paper, we prove that Einstein manifolds sufficiently close, in the Gromov-Hausdorff sense, to an orbifold are actually close to a gluing of model spaces in suitable wei...