Automatic non-destructive dimensional measurement of cotton plants in real-time by machine vision

[Abstract]Pressure on water resources in Australia necessitates improved application of water to irrigated crops. Cotton is one of Australia’s major crops, but is also a large water user. On-farm water savings can be achieved by irrigating via large mobile irrigation machines (LMIMs), which are capable of implementing deficit strategies and varying water application to 1 m2 resolution. However, irrigation amounts are commonly held constant throughout a field despite differing water requirements for different areas of a crop due to spatial variability of soil, microclimate and crop properties. This research has developed a non-destructive cotton plant dimensional measurement system, capable of mounting on a LMIM and streaming live crop measurement data to a variable-rate irrigation controller. The sensor is a vision system that measures the cotton plant attribute of internode length, i.e. the distance between main stem nodes (or branch junctions) on the plant’s main stem, which is a significant indicator of plant water stress. The vision system consisted of a Sony camcorder deinterlaced image size 720 × 288 pixels) mounted behind a transparent panel that moved continuously through the crop canopy. The camera and transparent panel were embodied in a contoured fibreglass camera enclosure (dimensions 535 mm × 555 mm × 270 mm wide) that utilised the natural flexibility of the growing foliage to firstly contact the plant, such that the top five nodes of the plant were in front of the transparent panel, and then smoothly and non-destructively guide the plant under the curved bottom surface of the enclosure. By forcing the plant into a fixed object plane (the transparent panel), reliable geometric measurement was possible without the use of stereo vision. Motorisation of the camera enclosure enabled conveyance both across and along the crop rows using an in-field chassis. A custom image processing algorithm was developed to automatically extract internode distance from the images collected by the camera, and comprised both single frame and sequential-frame analyses. Single frame processing consisted of detecting lines corresponding to branches and calculating the intersection of the detected lines with the main stem to estimate candidate node positions. Calculation of the ‘vesselness’ function for each pixel using the Hessian matrix eigenvalues determined whether the pixel was likely to belong to a stem (i.e. a curvilinear structure). Large areas of connected high-vesselness pixels were identified as branches. For each branch area, centre points were determined by solving the second order Taylor polynomial in the direction perpendicular to the line direction. The main stem was estimated with a linear Hough transform on the branch centre points within the image. Lines were then fitted to the centre points of other branch segments using the hop-along line-fitting algorithm and these lines were selectively projected to the main stem to estimate candidate node positions. The automatically-identified node positions corresponded to manual position measurements made on the source images. Within individual images, leaf edges were erroneously detected as candidate nodes (‘false positives’) and contributed up to 22% of the total number of detected candidate nodes. However, a grouping algorithm based on a Delaunay Triangulation mesh of the candidate node positions was used to remove the largely-random false positives and to create accurate candidate node trajectories. The internode distance measurement was then calculated as the maximum value between detected trajectories which corresponded to when the plant was closest to the transparent panel. From 168 video sequences of fourteen plants, 95 internode lengths were automatically detected at an average rate of one internode length per 1.75 plants for across row measurement,and one internode length per 3.3 m for along row measurement. Comparison with manually-measured internode lengths yielded a correlation coefficient of 0.86 for the automatic measurements and an average standard error in measurement of 3.0 mm with almost zero measurement bias. The second and third internode distances were most commonly detected by the vision system. The most measurements were obtained with the camera facing north or south, on a partially cloudy day in which the sunlight was diffused. Heliotropic effects and overexposed image background reduced image quality when the camera faced east or west. Night time images, captured with 850 nm LED illumination, provided as many measurements as the corresponding daytime measurements. Along row camera enclosure speeds up to 0.20 m/s yielded internode lengths using the current image processing algorithms and hardware. Calculations based on field programmable gate array (FPGA) implementation indicated an overall algorithm run-time of 46 ms per frame which is suitable for real-time application. It is concluded that field measurement of cotton plant internode length is possible using a moving, plant-contacting camera enclosure; that the presence of occlusions and other foliage edges can be overcome by analysing the sequence of images; and that real-time in-field operation is achievable