Depth value approximation of 2D complex-shape objects for 3D modelling using optical flow and trigonometry

Three dimensional (3D) modelling of physical objects can be very useful in many areas, such as computer graphics and animation, robot vision, reverse engineering, and medical. 3D modelling can be done from the scratch using modelling software,or digitised from real world objects. The process of modelling with software often consumes much time and requires a steep learning curve. On the other hand,conventional digitisation methods utilise Coordinate Measuring Machines (CMMs) or laser scanners. Nevertheless, both of these devices are very costly and require a certain amount of technical knowledge during usage and maintenance. An alternative approach which sacrifices some accuracy to greatly reduce the implementation costs will be Image-Based Modelling (IBM). This research introduces an IBM technique using optical flow and trigonometry with images captured via webcams. The implementation cost is reduced as it only requires a laptop, a webcam and a simple experiment setup. Image pairs with known small angle rotations and distance from the camera are the required inputs. Feature points were detected using good features to track and the displacement magnitudes were obtained via pyramidal implementation of Lucas Kanade Optical Flow. Optical flow magnitudes were then related with trigonometry to deduct the depth values of the feature points. The solution was able to combine feature points from all sides to produce a set of 3D surface points. Colour information of the feature points can be extracted as well. Data enhancement algorithms were implemented to perform noise filtering and inverse perspective mapping (IPM). Experiments were carried out with eight small complex shaped objects placed 300 mm away from the webcam. On average, the processing capacity for the solution was 1519 points per second. The average error on the approximated width dimension was 3.27% of the actual width while the average error on the depth dimension was 6.88% of the actual depth. The solution may work with as few as four images to generate a full set of 3D surface points. Future research may work on using the detected 3D point cloud as control points for texture coordinates to produce a fully texture mapped 3D model