Inductive classifier learning from data: An extended Bayesian belief function approach

A central problem in artificial intelligence is reasoning under uncertainty. This thesis views inductive learning as reasoning under uncertainty and develops an Extended Bayesian Belief Function approach that allows a two-layer representation of the probabilistic rules: basic probabilistic belief and their confidences, which are independent of each other and represent different semantics of the rules. The use of the confidence measure of probabilistic rules can thus handle many difficult problems in inductive learning, including noise, missing values, small samples, inter-attribute dependency, and irrelevant or partially relevant attributes, all of which are characteristics of real-world induction tasks.The theoretical framework is based upon an uncertainty calculus, Dempster-Shafer theory which allows an explicit representation of complete or partial lack of knowledge. This explicit representation is used to quantify and discount the effects of unreliable probability estimates due to noise and small samples, and to account for inter-attribute dependency and irrelevant or partially relevant attributes. Based on this methodology, a learning system, called IUR (Induction of Uncertain Rules) that uses only the first-order correlation information, is developed and experimentally demonstrated to outperform the major existing induction systems on many of the standard test sets.Future research includes extending IUR to use higher-order correlation information and integrating the Extended Bayesian Belief Function approach to other learning paradigms such as decision trees and neural networks.U of I OnlyETDs are only available to UIUC Users without author permissio