Effective models of black holes interior have led to several proposals for regular black holes. In the so-called polymer models, based on effective deformations of the phase space of spherically symmetric general relativity in vacuum, one considers a deformed Hamiltonian constraint while keeping a non-deformed vectorial constraint. In this article, we revisit and study further the question of covariance in these deformed gravity models. In particular, we propose a Lagrangian formulation for these deformed gravity models where polymer-like deformations are introduced at the level of the full theory prior to the symmetry reduction and prior to the Legendre transformation. This enables us to test whether the concept of deformed covariance foun...