Improving Object Detectors by Exploiting Bounding Boxes for Augmentation Design

Recent advancements in developing pre-trained models using large-scale datasets have emphasized the importance of robust protocols to adapt them effectively to domain-specific data, especially when the available data is limited. To achieve data-efficient fine-tuning of pre-trained object detection models, data augmentations are crucial. However, selecting the appropriate augmentation policy for a given dataset is known to be challenging. In this study, we address an overlooked aspect of this problem - can bounding box annotations be utilized to develop more effective augmentation policies? Our approach InterAug reveals that, by leveraging the annotations, one can deduce the optimal context for each object in a scene, rather than manipulating the entire scene or just the pre-defined bounding boxes. Through rigorous empirical research involving various benchmarks and architectures, we showcase the effectiveness of InterAug in enhancing robustness, handling data scarcity, and maintaining resilience to diverse high background contexts. An important advantage of InterAug is its compatibility with any off-the-shelf policy, requiring no modifications to the model architecture, and it significantly outperforms existing protocols. We will release the codes upon acceptance. Our codes can be found at https://github.com/kowshikthopalli/InterAug