A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical for-mulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and mod-ern implementation practices influence accuracy. We dis-cover that “classical” flow formulations perform surprisingly well when combined with modern optimization and imple-mentation techniques. One key implementation detail is the median filtering of intermediate flow fields during optimiza-tion. While this improves the robustness of classical meth-ods it actually leads to higher energy solutions, meaning that these methods are not optimizing the original objective function. To understand the principles behind this phenom-enon, we derive a new objective function that formalizes the median filtering heuristic. This objective function includes a non-local smoothness term that robustly integrates flow esti-mates over large spatial neighborhoods