Some aspects of the history of bayesian information processing

For many years, traditional Bayesian (TB) and information theoretic (IT) procedures for learning from data were viewed as distinctly different approaches. Derivations of the TB and IT learning models are reviewed and compared. Then the 1988 synthesis of the TB and IT learning models and generalizations of them are described along with descriptions of selected applications. Included are learning procedures that do not require use of likelihood functions and/or priors. Works by leading Bayesians and information theorists are cited and related to TB/IT issues. Key words: Bayes’s theorem, information theory, optimal information processing rules, learning from data, statistical inference