We study the special values at $ s=2 $ and $ 3 $ of the spectral zeta function $ zeta_Q (s) $ of the non-commutative harmonic oscillator $ Q(x,D_x) $ introduced in A. Parmeggiani and M. Wakayama (Proc. Natl Acad. Sci. USA 98 (2001), 26-31; Forum Math. 14 (2002), 539-604). It is shown that the series defining $ zeta_Q (s) $ converges absolutely for $ {Re}_s>1 $ and further the respective values $ zeta_Q (2) $ and $ zeta_Q (3) $ are represented essentially by contour integrals of the solutions, respectively, of a singly confluent Heun ordinary differential equation and of exactly the same but an inhomogeneous equation. As a by-product of these results, we obtain integral representations of the solutions of these equations by rational function...