Knowledge-based feature generation for inductive learning

Inductive learning is an approach to machine learning in which concepts are learned from examples and counterexamples. One requirement for inductive learning is an explicit representation of the characteristics, or features, that determine whether an object is an example or counterexample. Obvious or easily available representations do not reliably satisfy this requirement, so constructive induction algorithms have been developed to satisfy it automatically. However, there are some features, known to be useful, that have been beyond the capabilities of most constructive induction algorithms. This dissertation develops knowledge-based feature generation, a stronger, but more restricted, method of constructive induction than was available previously. Knowledge-based feature generation is a heuristic method of using one general and easily available form of domain knowledge to create functional features for one class of learning problems. The method consists of heuristics for creating features, for pruning useless new features, and for estimating feature cost. It has been tested empirically on problems ranging from simple to complex, and with inductive learning algorithms of varying power. The results show knowledge-based feature generation to be a general method of creating useful new features for one class of learning problems