We study the time complexity of the discrete k-center problem and related (exact) geometric set cover problems when k or the size of the cover is small. We obtain a plethora of new results: - We give the first subquadratic algorithm for rectilinear discrete 3-center in 2D, running in O?(n^{3/2}) time. - We prove a lower bound of ?(n^{4/3-?}) for rectilinear discrete 3-center in 4D, for any constant ? > 0, under a standard hypothesis about triangle detection in sparse graphs. - Given n points and n weighted axis-aligned unit squares in 2D, we give the first subquadratic algorithm for finding a minimum-weight cover of the points by 3 unit squares, running in O?(n^{8/5}) time. We also prove a lower bound of ?(n^{3/2-?}) for the same problem ...