Quantitative and qualitative evaluation of performance and robustness of image stitching algorithms

Many different image stitching algorithms, and mechanisms to assess their quality have been proposed by different research groups in the past decade. However, a comparison across different stitching algorithms and evaluation mechanisms has not been performed before. Our objective is to recognize the best algorithm for panoramic image stitching. We measure the robustness of different algorithms by means of assessing image quality of a set of panoramas. For the evaluation itself, a varied set of assessment criteria are used, and the evaluation is performed over a large range of images captured using differing cameras. In an ideal stitching algorithm, the resulting stitched image should be without visible seams and other noticeable anomalies. An objective evaluation for image quality should give results corresponding to a similar evaluation by the Human Visual System. Our conclusion is that the choice of stitching algorithm is scenario dependent, with run-time and accuracy being the primary considerations