Autonomous machine vision for off-road vehicles in unstructured fields

A feasibility study of machine vision applications was conducted for agricultural vehicle navigation in open field environments, and focused on solving certain fundamental issues in vision-based agricultural vehicle navigation. Those issues were: (1) camera installation pose automatic calibration; (2) vehicle heading estimation and (3) field edge detection. A stereo color camera was selected to support the research on the three issues. Stereo cameras have been used as perception sensors for agricultural vehicle navigation for years. One problem impeding their broader application is the difficulty of determining the camera???s installation pose using conventional measuring tools, especially when they are used in an open-field agricultural environment. To solve this problem, an automated calibration method was developed to determine the camera???s installation pose with respect to the vehicle frame. Based on this method, a binocular stereo camera acquired a sequence of images as the vehicle moved straight forward a short distance on relatively even ground. A machine vision algorithm was used to detect static feature points on the ground and track their three-dimensional (3D) motions in relationship to the vehicle. A plane fitting for the ground features was then used to determine the camera roll and pitch, and the tracked motions were used to estimate the camera yaw. The results obtained from the field test validated that this method was capable of determining the camera installation pose automatically in order to achieve a calibration accuracy of ??1 ?? over approximately 10 m of test distance. The calibrated poses could be used to compensate for the navigation errors induced by the misalignment of the camera. An image processing algorithm was developed to investigate the feasibility of using stereovision to estimate the heading direction of a moving vehicle in open agricultural field environments. The algorithm first detected, and then tracked, static natural ground features in every two consecutive images that were taken by a stereo camera mounted on a vehicle while the vehicle was in motion. These static features were used as references to calculate the vehicle???s three-dimensional (3D) motion. In the final stage, the vehicle heading direction was estimated using the 3D motion. Working with a series of sequential image frames taken while the vehicle was in motion, the algorithm continuously estimated the vehicle heading direction. Field tests were conducted to evaluate its usability. When the vehicle traveled straight forward, the proposed algorithm worked properly. When the vehicle traveled in an oscillating mode, the algorithm responded properly when the vehicle turned, but with less estimation accuracy than in the straight traveling mode. The field tests showed that it is possible to use stereovision to estimate a moving vehicle???s heading direction in an open agricultural field. Field edges are important references for human drivers who steer vehicles in agricultural operations. This research explored the possibility of using machine vision to detect field edges in open field agricultural environments. A detecting algorithm was proposed based on the hue difference between an open field and its grass-covered edge. Field tests showed that the algorithm was capable of distinguishing a relatively clear edge from an open field. However, when the field edge was not clear, the algorithm was unable to identify it due to the existence of noise. This research showed that images with lower resolution were less affected by noise. The same algorithm detected unclear field edges after reducing noise by lowering image resolution. Color change adaptability was also implemented in order to improve the algorithm???s robustness. As a result, it was possible to use machine vision to detect the grass covered edges of an open agricultural field. This research proved the feasibility of the machine vision applications in the three targeted problems, and has shown that machine vision is capable of navigating agricultural vehicles in open field environments