Augmented kinaesthetic teaching of precise robot movements

V doktorski disertaciji predstavljamo učenje finih robotskih gibov z uporabo pristopa kinestetičnega učenja. Kinestetično učenje je uveljavljen pristop učenja z demonstracijo saj operaterjem omogoča intuitivno izvedbo giba brez dodatnih krmilnih naprav. Operater lahko namreč izvede želen gib tako, da prime posamezne segmente robota in jih premakne v želeno lego. Primernost kinestetičnega učenja je tako že bila preučena v sklopu aplikacij, ki so zahtevale grobe gibe, ne pa v sklopu aplikacij, ki so zahtevale učenje finih gibov. Fini gibi so namreč gibi, pri katerih je zahtevana natančnost pozicioniranja znotraj velikostnega reda milimetra, za kar se pogosto uporablja teleoperacijo in kooperativno robotsko orodje, ki sta uveljavljena pristopa za demonstracijo finih gibov. Tako smo v prvem delu disertacije delovanje kinestetičnega učenja primerjali z omenjenima pristopoma, pri čemer smo pristope primerjali na dveh skrbno načrtovanih nalogah. Nalogi sta se razlikovali glede na tip giba, pri čemer je prva zahtevala natančen premik od točke do točke, druga pa natančno sledenje referenčni poti. Cilj študije je bila, poleg določanja primernosti posameznega pristopa za demonstracijo finih gibov, tudi analiza vpliva vizualnih modalitet na natančnost izvedbe. Razvili smo namreč neke vrste virtualni mikroskop, ki je omogočal slikovno povečavo delovnega območja pod vrhom robota in posledično izboljšal vizualno zaznavanje pozicijskih odstopanj med izvajanjem demonstracije. Operaterji so tako izvedli demonstracije brez in z uporabo vizualne povečave. V sklopu te študije smo vzporedno pripravili tudi manjšo študijo osredotočeno na izvajanje finih dinamičnih gibov. Pri teh gibih je za uspešno izvedbo potreben ustrezen dinamičen potek, pri čemer se sam gib izvede na relativno kratki prostorski razdalji velikostnega reda centimetra. Ugotovitve te študije so predstavljene v sklopu priloge Dodatek A, saj ugotovitve niso tako pomembne kot v primeru ostalih študij. Nadalje smo v drugem delu doktorske disertacije preučili metode zapisa finih gibov. Poleg primerne demonstracije je ustrezen zapis demonstracij namreč druga ključna stvar pri pristopu učenja z demonstracijo. Med seboj smo primerjali metodi DMP in GMM, ki sta uveljavljeni metodi zapisa demonstracij. Dodatno smo predlagali tudi nadgradnjo metode GMM na podlagi frekvenčne analize, ki omogoča ustrezen zapis finih gibov brez občutnega povečanja računske kompleksnosti metode. Disertacija se zaključi s preizkusom kinestetičnega učenja finih gibov na realnem procesu v kliničnem mikrobiološkem okolju. Z vpeljavo sodelujočih robotov v to delovno okolje je namreč potrebno vedeti, ali njihova uporaba omogoča primerljive rezultate v primerjavi z izkušenimi delavci. Tako smo uporabili sodelujočega robota za postopek zaznave, odvzema in nanosa bakterijskih kolonij v sklopu procesa identifikacije bakterijskih kolonij z uporabo masne spektrometrije. Delovanje sistema smo ocenili na podlagi rezultatov posameznega vmesnega postopka ter uspešnosti identifikacije kolonij, rezultate pa primerjali z objavljenimi podatki o uspešnosti izkušenih laboratorijskih tehnikov. Na podlagi prvih dveh študij, pa smo za namen te aplikacije pripravili tudi t.i. učni vmesnik. Vmesnik je bil sestavljen iz dveh ločenih delov. Prvi del je predstavljal obogateno okolje delovnega prostora, ki je operaterju, z uporabo očal za navidezno resničnost, omogočil bolj natančno izvedbo demonstracije. Drugi del pa je predstavljal sistem, ki je na podlagi metode DMP zapisal izvedeno demonstracijo ter jo prilagodil glede na trenutne zahteve procesa.Kinesthetic teaching is a well-established learning by demonstration (LfD) approach as it allows operators to intuitively generate the robot motion without additional control devices. That is so because the operator can perform the desired motion by grasping individual robot segments and moving them to the desired pose. The performance of kinesthetic teaching has thus already been studied in the context of an application requiring coarse movements, while the performance of generating fine movements has yet to be studied. Fine movements require high positional precision, for which teleoperation and cooperative robot tool are the two established LfD approaches. Thus, in the first part of the thesis, we compare the performance of kinesthetic teaching to the two approaches mentioned above. For comparison, we carefully designed two tasks based on the required motion, with the first task requiring a precise movement from point to point and the second task requiring a precise tracking of a reference trajectory. In addition, to determine the suitability of each LfD approach for fine movements, we also analyzed the influence of visual modalities on the operator’s performance. Specifically, we developed a visual enhancement tool that allowed us to visually zoom in on the work area under the robot’s end-effector and consequently improve the visual detection of positioning errors during the demonstration. Thus, operators performed demonstrations using each LfD approach with and without the use of the visual enhancement tool. As part of this study, a smaller parallel study which focused on the execution of fine dynamic movements was also performed. For these movements, the dynamics of the movement have to be appropriate in order for successful demonstration. Usually, these movements are also generated over a relatively short distance. The findings of this study are presented in Appendix A, as the findings are not as significant as it is the case with other studies. In the second part of the thesis, we analyzed the performance of different methods which are used for demonstration generalization. Apart from an appropriate demonstration, motion generalization is the other important issue regarding LfD. We compared DMP and GMM, which are both established methods for movement generalization. Additionally, we proposed a novel addition to the GMM method that improves the generalization of fine movements without increasing the computational complexity of the model. The thesis concludes with a study that implements kinesthetic teaching into a real-world environment. We used a collaborative robot to detect, collect and deposit bacterial colonies as part of a bacterial colony identification process using mass spectrometry. The system’s performance was evaluated based on each intermediate procedure’s results and colony identification’s success rate. The identification results were then compared with published data on the success rate of experienced laboratory technicians. We have also developed a so-called teaching agent for this application based on the findings from the first two studies. The agent consisted of two separate functionalities. The first was an augmented reality environment that allowed the operator to perform the demonstration more precisely using the virtual reality ovals. The second functionality was a system that, based on the DMP method, generalized the given demonstration and adapted it to the current process requirements