Pearl's and Jeffrey's Update as Modes of Learning in Probabilistic Programming

The concept of updating a probability distribution in the light of newevidence lies at the heart of statistics and machine learning. Pearl's andJeffrey's rule are two natural update mechanisms which lead to differentoutcomes, yet the similarities and differences remain mysterious. This paperclarifies their relationship in several ways: via separate descriptions of thetwo update mechanisms in terms of probabilistic programs and samplingsemantics, and via different notions of likelihood (for Pearl and for Jeffrey).Moreover, it is shown that Jeffrey's update rule arises via variationalinference. In terms of categorical probability theory, this amounts to ananalysis of the situation in terms of the behaviour of the multiset functor,extended to the Kleisli category of the distribution monad