Probability Models for Complex Systems

This thesis is a collection of essays on probability models for complex systems. Chapter 1 is an introduction to the thesis. The main point made here is the importance of probabilistic modeling to complex problems of machine perception. Chapter 2 studies minimum complexity regression. The results include: (1) weak consis-tency of the regression, (2) divergence of estimates in L2-norm with an arbitrary complexity assignment, and (3) condition on complexity measure to ensure strong consistency. Chapter 3 proposes compositionality as a general principle for probabilistic modeling. The main issues covered here are: (1) existence of general compositional probability measures, (2) subsystems of compositional systems, and (3) Gibbs representation of compositional probabilities. Chapter 4 and 5 establish some useful properties of probabilistic context-free grammars (PCFGs). The following problems are discussed: (1) consistency of estimated PCFGs, (2) finiteness of entropy, momentum, etc, of estimated PCFGs, (3) branching rates and re-normalization of inconsistent PCFGs, and (4) identifiability of parameters of PCFGs. Chapter 6 proposes a probabilistic feature based model for languages. Issues dealt with i