Learning Perceptual Causality from Video Amy Fire and Song-Chun

Computer vision and artificial intelligence research has long danced around the subject of causality: vision re-searchers use causal relationships to aid action detec-tion, and AI researchers propose methods for causal in-duction independent of video sensors. In this paper, we argue that learning perceptual causality from video is a necessary step for understanding scenes in video. We explain how current object and action detection is suf-fering without causality, and we explain how current causality research is suffering without grounding on raw sensors. We then go on to describe one plausible solu-tion for grounding perceptual causality on raw sensors. Applying causal knowledge to vision research provides a much deeper level of understanding than considering action