Performance Evaluation for Fleet-based and Unit-based Prognostic Methods

Within the last decade several new methods for prognostics have been developed and an overall understanding of the various issues involved in predictions for health management has significantly improved. However, it appears that there is still a lack of consensus on how prognostics is defined and what constitutes good performance for prognostics. This paper first differentiates prognostics from other prediction approaches before highlighting key attributes of performance for prediction methods. Then it argues that it is important to understand what factors affect the performance of a prognostic approach. Factors such as the application and end use of a prognostic output, the various methods to make predictions, purpose of performance evaluation, etc. are discussed. This paper presents a comprehensive view of various such aspects that dictate or should dictate what performance evaluation must be as far as prognostics is concerned. It is also discussed what should be used as baseline to assess performance and how to interpret commonly used comparisons of algorithm predictions to observed failure times. The primary goal of this paper is to present some arguments of how these issues can be addressed and to stimulate a discussion about meaningful evaluation of prognostic performance. These discussions are followed by a brief description of prognostics metrics proposed recently, their applicability, and limitations. This paper does not intend to suggest any metrics in particular rather highlights important aspects that must be covered by any performance evaluation method for prognostics. 1