DETECTING IRREGULARY SHAPED OBJECTS IN NATURAL ENVIRONMENTS USING SPATIAL-FREQUENCY ANALYSIS AND ELASTIC IMAGE REGISTRATION

V doktorski disertaciji opisujemo nov način odkrivanja objektov nepravilnih oblik v naravnem okolju. Postopek se začne v slikovnem prostoru s posameznimi slikami iste opazovane scene. Upošteva uveljavljene rešitve, kot so predobdelava slik s klasičnimi prijemi za izboljšanje kontrasta, odpravljanje različnih osvetljenosti in odstranjevanje šuma ter pragovna barvna segmentacija slik, ki upošteva znane barvne lastnosti iskanih objektov. Osrednja prispevka disertacije pomenita izpopolnjeno odkrivanje objektov z nepravilnimi oblikami in razširjata analizo tekstur v frekvenčnem prostoru ter 3D rekonstrukcijo s pomočjo geometrije več pogledov. Teksturno analizo izvedemo z 2D prostorsko-frekvenčnim pristopom oziroma z 2D Wigner-Villeovo predstavitvijo. Z njo poiščemo območja podobnih frekvenčnih karakteristik, ki enolično določajo teksture iskanih objektov. Kadar so objekti delno zakriti ali imajo podobne barvne lastnosti kot ozadje, tudi analiza tekstur ni dovolj občutljiva. Da bi izboljšali robustnost, smo v postopku odkrivanja uporabili več slik iste scene in izsledke z geometrijo več pogledov. Bistvena novost te razširitve je določanje korespondenčnih točk, na katerih temelji 3D rekonstrukcija opazovane scene. Rekonstrukcija mora biti tako detaljna, da so nepravilne prostorske oblike zanesljivo razpoznavne in da jih lahko uporabimo pri končnem potrjevanju iskanih objektov. Ustrezno gostoto korespondenčnih točk dosežemo tako, da pare posnetkov elastično poravnamo in iz parametrov za poravnavo oziroma iz deformacijskih matrik izračunamo ujemanje slikovnih točk. Med njimi izberemo čim večje število dobro ujemajočih se parov, za kar smo uvedli tudi posebno mero zanesljivosti. Izbrane pare uporabimo kot korespondenčne točke v 3D rekonstrukciji opazovane scene. Končno odločitev o najdenih objektih sprejmemo s povezovanjem informacij, pridobljenih v prostorsko-frekvenčnih predstavitvah in njihovih 3D rekonstruiranih oblikah. Učinkovitost razvitega razpoznavalnega postopka smo preverili z realnimi posnetki sadnih dreves. Poskuse smo zasnovali tako, da smo ugotavljali, kako se izboljša razpoznavanje plodov, če poleg znanih postopkov barvne segmentacije v slikovnem prostoru uporabimo še predlagani nadgradnji s prostorsko-frekvenčnimi značilnicami in z geometrijo več pogledov. Opazovane posnetke smo razdelili v štiri skupine: barvno ločljivi objekti (npr. rdeči plodovi, brez večjih zakrivanj), barvno ločljivi objekti z zakrivanji, barvno teže ločljivi objekti (npr. zeleni plodovi, brez večjih zakrivanj) in barvno teže ločljivi objekti z zakrivanji. Rezultati raziskave kažejo, da je uvedeni postopek za odkrivanje objektov nepravilnih oblik v naravnem okolju primeren, saj dosega v povprečju 86 % natančnost določanja korespondenčnih točk, in pripomore k natančnejšemu odkrivanju objektov (plodov na sadnem drevju). Tako lahko pomembno izboljšamo dosedanjo avtomatizirano, računalniško spremljanje in napovedovanje pridelka v sadjarstvu.In this doctoral dissertation, we describe an innovative procedure for detecting natural objects that have irregular shapes. The procedure begins in a spatial domain in which we portray different images of a given scene. It includes some common image processing techniques such as contrast enhancement, uneven light distribution correction, noise elimination, and color segmentation applied by tresholding an image based on the known color properties of the objects we are trying to detect. The main contribution of this work is to introduce an improved object detection technique that is able to detect objects of irregular shapes both by extending texture analysis procedures to frequency domain, and by applying 3D reconstruction based on multiple-view geometry. The texture analysis was done with the help of a 2D spatial-frequency representation, for which we selected a 2D Wigner-Ville representation. With its help we were able to detect areas that have similar frequency characteristics as the described looked-for objects. In some cases we encountered partly occluded areas, or areas with a similar texture as that of an object. In these cases, we could not rely on texture-analysis alone. In order to improve the results, we collected images of the same scene captured at different viewpoints, and introduced a multiple-view geometry. The main novelty of the proposed reconstruction, then, was the way we determined corresponding ties between pixels, ties that are needed for 3D reconstruction. The reconstruction had to be both reliable and precise in order to be used in the final shape analysis. Thus, a sufficient number of corresponding points needed to be provided, which was achieved with the help of an elastic image registration that produces deformation matrices describing each pixel and its corresponding ties. By introducing a distance measure, each corresponding pair was evaluated and only the best were selected for 3D reconstruction. The final decision whether detected regions sufficiently described regions of interest was made by considering their spatial-frequency representation as well as their 3D reconstructed shapes. In order to evaluate the efficiency of our purposed solution, we used snapshots of natural scenes depicting real fruit trees. The tests were conducted in such a way to show how fruit detection methods could be improved with the help of spatial-frequency representation and multiple-view geometry. We used four different kinds of image sets: one representing colorful objects (red) with no occlusionsthe second representing colorful objects with occlusionsthe third less (distinctly) colorful objects with no occlusionsand, finally, the fourth representing less colorful and partly occluded objects. The results showed that the purposed solution does in fact improve object detection techniques, achieving corresponding points accuracy of 86 %, making them more reliable in cases in which we are trying to detect objects of irregular shape in natural environments, such as fruit on a fruit tree. This, in effect, improves automatic, computer-supported inspection, as well as the ability to forecast crop yield in agriculture