12 pages LaTeX2eWe show that it is possible to consistently describe dynamical systems, whose equations of motion are of degree higher than two, in the microcanonical ensemble, even if the higher derivatives aren't coordinate artifacts. Higher time derivatives imply that there are more than one Hamiltonians, conserved quantities due to time translation invariance, and, if the volume in phase space, defined by their intersection, is compact, microcanonical averages can be defined and there isn't any instability, in the sense of Ostrogradsky, even though each Hamiltonian, individually, may define a non-compact (hyper)surface. We provide as concrete example of these statements the Pais--Uhlenbeck oscillator and show that it can describe a syst...