A Bayesian Account of Depth from Shadow

When an object casts a shadow on a background surface, the offset of the shadow can be a compelling cue to the relative depth between the object and the background (e.g., Kersten et al 1996, Fig. 1). Cavanagh et al (2021) found that, at least for small shadow offsets, perceived depth scales almost linearly with shadow offset. Here we ask whether this finding can be understood quantitatively in terms of Bayesian decision theory. Estimating relative depth from shadow offset is complicated by the fact that the shadow offset is co-determined by the slant of the light source relative to the background. Since this is often difficult or impossible to estimate directly, the observer must employ priors for both the relative depth and the light source slant. To establish an ecological prior for relative depth, we employed the SYNS dataset (Adams et al., 2016) and the methods of Ehinger et al (2017) to measure the distribution of relative depths at depth edges near the horizon (Fig. 2). Lacking comparable empirical statistics for illumination slant, we considered two possible distributions: A zero-parameter uniform distribution, and a two-parameter beta distribution. To model the human data, we assumed that the visual system makes use of these priors and the observed shadow offset to minimize expected squared error in perceived relative depth. Fig. 3 shows that while the empirical depth prior brings the model into the range of the human data, a flat illumination prior predicts a more compressive scaling than observed. Fitting a beta distribution to minimize weighted squared deviation between human and optimal depth judgements corrects this deviation, and predicts a broadly peaked distribution over illumination slant, peaking at 37.4 deg away from the surface normal (Fig. 4). We will discuss possible ecological explanations for this illumination prior