We consider the phenomenology of the 511 keV emission in the Galactic bulge, as recently observed by INTEGRAL, and propose a model is which the positrons are produced by gamma-ray bursts (GRB) associated with mini starbursts in the central molecular zone (CMZ). We show that the positrons can easily diffuse across the bulge on timescales of about 10^7 yr, and that their injection rate by GRBs is compatible with the observed fluxes if the mean time between two GRBs in the bulge is about 8 10^4 yr x E_GRB_51. We also explain the low disk-to-bulge emission ratio by noting that positrons from GRBs in the Galactic disk should annihilate on timescales of < 10^4 yr in the dense shell of the underlying supernova remnant, after the radiative transiti...