Overcoming Human Weaknesses in Validation of Knowledge-Based Systems

Abstract: Human experts employed in validation exercises for knowledge-based sys-tems often have limited time and availability. They often have different opinions from each other as well as from themselves over time. We address this situation by intro-ducing the use of validation knowledge used in prior validation exercises for the same knowledge-based system. We present a Validation Knowledge Base (VKB) that is the collective best experience of several human expert validators. We also present the con-cept of Validation Expert Software Agents (VESA) that represent a particular expert’s knowledge. VESA is a software agent corresponding to a specific human expert val-idator. It models the validation knowledge and behavior of its human counterpart by analyzing similarities with the responses of other experts. We also describe experi-ments with a small prototype system to evaluate the usefulness of these concepts.