Abstraction techniques provide formal guarantees for generic optimal control problems on nonlinear and hybrid systems. Computing an abstraction solving the problem over the whole state-space is computationally demanding in highdimensional spaces. We circumvent this curse of dimensionality by introducing a hierarchical abstraction approach for solving an optimal control problem for nonlinear systems with three nested partitions. These nested partitions allow the construction of auxiliary systems that characterize simulation relations, which are suitably exploited to provide upper and lower bounds for a branch and bound algorithm to yield an optimal solution for the control problem. An example illustrates the proposed method