Let $(X_t)_{t\ge 0}$ be the overdamped Langevin process on $\mathbb R^d$, i.e. the solution of the stochastic differential equation $$dX_t=-\nabla f(X_t) \, dt +\sqrt h \, dB_t.$$Let $\Omega\subset \mathbb R^d$ be a bounded domain. In this work, when $X_0=x\in \Omega$, we derive new sharp asymptotic equivalents (with optimal error terms) in the limit $h\to 0$ of the mean exit time from $\Omega$ of the process $(X_t)_{t\ge 0}$ (which is the solution of $(-\frac h2 \Delta+\nabla f\cdot \nabla)w=1 \text{ in } \Omega \text{ and } w=0 \text{ on } \pa \Omega$), when the function $f:\overline \Omega\to \mathbb R$ has critical points on $\partial \Omega$. Such a setting is the one considered in many cases in molecular dynamics simu...