The aim of this article is to understand the role of dust reference fields, often also called clocks, on cosmological perturbations around a classical spatially flat Friedmann-Lemaître-Robertson-Walker (FLRW) universe. We derive the Mukhanov-Sasaki equation for the Brown-Kuchař and Gaussian dust models, which both consider four dust fields as reference fields. The reduced phase space of Dirac observables, that is the gauge-invariant part of the theory, is constructed by means of an observable map applied to all elementary phase space variables of the coupled system, consisting of gravity, a massive scalar field and the dust degrees of freedom and automatically yields the set of independent physical variables. The evolution of these observab...