Estimation of Relative Entropy Using Novel Taylor-Like Representations

Sharp bounds are obtained for perturbed generalised Taylor series. The perturbation involves the arithmetic sum of the upper and lower bounds of the (n+1)th derivative. The sharpest bound is in terms of the one norm of the Appell polynomial which constitutes the coefficients of the derivative of the function to be approximated. The results are demonstrated for the estimation of the Kullback-Leibler distance, Shannon entropy and mutual information