In this article we demonstrate the net acceleration of relativistic electrons using a direct, in-vacuum interaction with a laser. In the experiment, an electron beam from a conventional accelerator is first energy modulated at optical frequencies in an inverse-free-electron-laser and bunched in a chicane. This is followed by a second stage optical accelerator to obtain net acceleration. The optical phase between accelerator stages is monitored and controlled in order to scan the accelerating phase and observe net acceleration and deceleration. Phase jitter measurements indicate control of the phase to {approx}13{sup o} allowing for stable net acceleration of electrons with lasers