Weed Detection in UAV Images of Cereal Crops with Instance Segmentation

Modern weeding is predominantly carried out by spraying whole fields with toxic pesticides, a process that accomplishes the main goal of eliminating weeds, but at a cost of the local environment. Weed management systems based on AI solutions enable more targeted actions, such as site-specific spraying, which is essential in reducing the need for chemicals. To introduce sustainable weeding in Swedish farmlands, we propose implementing a state-of-the-art Deep Learning (DL) algorithm capable of instance segmentation for remote sensing of weeds, before coupling an automated sprayer vehicle. Cereals have been chosen as the target crop in this study as they are among the most commonly cultivated plants in Northern Europe. We used Unmanned Aerial Vehicles (UAV) to capture images from several fields and trained a Mask R-CNN computer vision framework to accurately recognize and localize unique instances of weeds among plants. Moreover, we evaluated three different backbones (ResNet-50, ResNet101, ResNeXt-101) pre-trained on the MS COCO dataset and through transfer learning tuned the model towards our classification task. Some well-reported limitations in building an accurate model include occlusion among instances as well as the high similarity between weeds and crops. Our system handles these challenges fairly well. We achieved a precision of 0.82, recall of 0.61, and F1 score of 0.70. Still, improvements can be made in data preparation and pre-processing to further improve the recall rate. All and all, the main outcome of this study is the system pipeline which, together with post-processing using geographical field coordinates, could serve as a detector for half of the weeds in an end-to-end weed removal system.Site-specific Weed Control in Swedish Agricultur