In this paper, we develop a loop group description of harmonic maps $\mathcal{F}: M \rightarrow G/K$ ``of finite uniton type", from a Riemann surface $M$ into inner symmetric spaces of compact or non-compact type. This develops work of Uhlenbeck, Segal, and Burstall-Guest to non-compact inner symmetric spaces. To be more concrete, we prove that every harmonic map of finite uniton type from any Riemann surface into any compact or non-compact inner symmetric space has a normalized potential taking values in some nilpotent Lie subalgebra, as well as a normalized frame with initial condition identity. This provides a straightforward way to construct all such harmonic maps. We also illustrate the above results exclusively by Willmore surfaces, s...