A coordinate-free theory of eigenvalue analysis related to the method of principal components and the Karhunen—Loève expansion

It is well known that the eigenvalue analyses such as the method of principal components and the Karhunen-Loève expansion are coordinate dependent. This paper deals with a development of a coordinate-free theory of eigenvalue analysis with observation noise taken into consideration. Results obtained are stated with no reliance on a particular coordinate system. When data and observation noise are normally distributed, the results are derived using the average mutual information between the data and the measurements. From this new point of view, it will be seen that in the traditional theories of principal components and the Karhunen-Loève expansion, the noise covariance matrix is presupposed to be of a special form implicitly