Relational Networks for visual Relationship detection in Images

Graduation date:2018This thesis is about visual relationship detection. This is an important task in computer vision. The goal is to detect all visual relationships in a given image between objects. This thesis presents a new approach to this problem. Our approach does not use an object detector as a common pre-processing step of prior work. In this way, we overcome the limitation of large computational complexity when considering a large object categories. We focus on relationships that can be specified as triplets $<subject, predicate, object>$. Our approach has four modules including object pair detection, triplet proposal, imposing common-sense constraints and ranking. To boost the performance of our system, we use Message Passing on the triplet proposal module. Our evaluation shows effectiveness of our approach. We also discuss some improvements of our visual relationship detection system in the future