Learning from Hints

AbstractWe present a systematic method for incorporating prior knowledge (hints) into the learning-from-examples paradigm. The hints are represented in a canonical form that is compatible with descent techniques for learning. All the hints are fed to the learning process in the form of examples, and examples of the function are treated on equal footing with the rest of the hints. During learning, examples from different hints are selected for processing according to a fixed or adaptive schedule. Fixed schedules specify the relative emphasis of each hint, and adaptive schedules are based on how well each hint has been learned so far. We discuss adaptive minimization which is based on estimates of the overall learning error