We study the matrix models that result from the localization of the partition functions of N=2 Chern-Simons-matter theories on the three-sphere. A large class of such theories are conjectured to be holographically dual to M-theory on Sasaki-Einstein seven-manifolds. We study the M-theory limit (large N and fixed Chern-Simons levels) of these matrix models for various examples, and show that in this limit the free energy reproduces the expected AdS/CFT result of N3/2/Vol(Y)1/2, where Vol(Y) is the volume of the corresponding Sasaki-Einstein metric. More generally we conjecture a relation between the large N limit of the partition function, interpreted as a function of trial R charges, and the volumes of Sasakian metrics on links of Calabi-Ya...