The relationship between stable holomorphic vector bundles on a compact complex surface and the same such objects on a blowup of the surface is investigated, where "stability" is with respect to a Gauduchon metric on the surface and naturally derived such metrics on the blowup. The main results are: descriptions of holomorphic vector bundles on a blowup; conditions relating (semi)-stability of these to that of their direct images on the surface; sheaf-theoretic constructions for "stabilising" unstable bundles and desingularising moduli of stable bundles; an analysis of the behaviour of Hermitian-Einstein connections on bundles over blowups as the underlying Gauduchon metric degenerates; the definition of a topology on equivalence classes ...