Text Entry on Mobile Devices using Continuous and Discrete Grapheme-Based Touch Gestures

U ovome radu modelirane su i implementirane dvije eksperimentalne metode unosa teksta za mobilne uređaje sa zaslonima osjetljivima na dodir. Obje metode unosa teksta zasnivaju se na oblikovanju grafema, a razlikuju se primarno po tipu dodirnih gesti u izvršavanju interakcije. Metoda KDI podrazumijeva diskretne dodirne geste, pri čemu korisnik za unos pojedinog slova mora slijedno aktivirati više različitih točaka koje u interaktivnoj mreži tvore odgovarajući grafem. KDI obuhvaća dva modaliteta rada: u jednome se za unos slova zahtijeva isključivo točna kombinacija aktiviranih točaka (Standard), dok se u drugome dodatno očekuje i njihov ispravan redoslijed prema pravilima pisanja grafema (Forced). Druga razvijena metoda unosa teksta zasniva se na kontinuiranim dodirnim gestama (KCI), kod koje korisnik na dijelu zaslona izravno oblikuje grafem koji se zatim klasificira pomoću modela neuronske mreže. Korišteni klasifikator u tipkovnici KCI izveden je iz MobileNet konvolucijske neuronske mreži trenirane na skupu podataka EMNIST, koristeći programski okvir TensorFlow. Svi modaliteti tipkovnica KDI i KCI komparativno su vrednovani u kontroliranom eksperimentu s dvadeset ispitnih korisnika. Rezultati testiranja pokazali su da je unos teksta značajno učinkovitiji uz korištenje metode KCI, nasuprot obje verzije metode KDI (Standard i Forced). Oblikovanje grafema kontinuiranim dodirom pokazalo se očekivano intuitivnijim pristupom, dok je koncept diskretnih dodira, primarno zbog potrebe za učenjem odgovarajućih sekvenci, izazvao veći mentalni napor i višu razinu frustracije.In this thesis, two experimental text entry methods for mobile touchscreen devices are modeled and implemented. Both text entry methods are based on the grapheme shaping, and differ primarily by the type of used touch gestures. The KDI method involves discrete touch gestures for entering a particular letter, wherein the user must activate a certain touch-point sequence that represents an appropriate grapheme in the touch-interactive network. KDI encompasses two operation modalities: Standard – which requires only the exact combination of touchpoints in a given sequence, and Forced – which additionally assumes correct order in activating such sequence, according to the standard writing rules. The second text entry method – KCI – is based on continuous touch gestures, allowing the user to directly draw the grapheme, which is then classified using a neural network model. The classifier used in KCI virtual keyboard is derived from the MobileNet convolution neural network trained on the EMNIST data set, using the TensorFlow programming framework. All mentioned text entry methods were comparatively evaluated in a controlled experiment with twenty users. Obtained results have shown that text entry is significantly more efficient with KCI method, versus both versions of the KDI (Standard and Forced). Shaping graphemes using continuous touch gestures was expectedly shown to be a more intuitive approach. On the other hand, the concept utilizing discrete touch gestures caused greater mental effort and higher levels of frustration, primarily due to the need for learning the appropriate grapheme sequences