We consider the problem of finding a small hitting set in an infinite range space F=(Q,R) of bounded VC-dimension. We show that, under reasonably general assumptions, the infinite-dimensional convex relaxation can be solved (approximately) efficiently by multiplicative weight updates. As a consequence, we get an algorithm that finds, for any delta>0, a set of size O(s_F(z^*_F)) that hits (1-delta)-fraction of R (with respect to a given measure) in time proportional to log(1/delta), where s_F(1/epsilon) is the size of the smallest epsilon-net the range space admits, and z^*_F is the value of the fractional optimal solution. This exponentially improves upon previous results which achieve the same approximation guarantees with running time pro...