A suitable coupling of the inflaton \u3c6 to a vector kinetic term F2 gives frozen and scale invariant vector perturbations. We compute the cosmological perturbations \u3b6 that result from such coupling by taking into account the classical vector field that unavoidably gets generated at large scales during inflation. This generically results in a too-anisotropic power spectrum of \u3b6. Specifically, the anisotropy exceeds the 1% level (10% level) if inflation lasts 3c5 e-folds ( 3c50 e-folds) more than the minimal amount required to produce the cosmic microwave background modes. This conclusion applies, among others, to the application of this mechanism for magnetogenesis, for anisotropic inflation, and for the generation of anisotropic ...