Reasoning with uncertainties over existence of objects

In this paper we consider planning problems in relational Markov processes where objects may “appear ” or “disap-pear”, perhaps depending on previous actions or properties of other objects. For instance, problems which require to ex-plicitly generate or discover objects fall into this category. In our formulation this requires to explicitly represent the un-certainty over the number of objects (dimensions or factors) in a dynamic Bayesian networks (DBN). Many formalisms (also existing ones) are conceivable to formulate such prob-lems. We aim at a formulation that facilitates inference and planning. Based on a specific formulation we investigate two inference methods—rejection sampling and reversible-jump MCMC—to compute a posterior over the process conditioned on the first and last time slice (start and goal state). We will discuss properties, efficiency, and appropriateness of each one