Neighborhood sampling of heterogeneous graphs in graph neural networks

Grafovske nevronske mreže so v zadnjem času eno izmed najbolj aktivnih področij raziskovanja globokega učenja. Uspešno so bile uporabljene pri problemih, kjer so podatki predstavljeni v obliki grafa, na primer pri analizi družbenih omrežij, napovedovanju prometa, razvoju zdravil itd. Kljub nekaterim zelo dobrim rezultatom pa ostaja še veliko odprtih izzivov pri uporabi nevronskih mrež na zelo velikih grafih, kjer smo omejeni z zmogljivostjo strojne opreme. V magistrskem delu naslavljamo problem uporabe grafovskih nevronskih mrež na obsežnih heterogenih grafih, kjer se med učenjem izvaja vzorčenje soseščine na vsaki plasti mreže, pri čemer se velikosti vzorca omejijo s hiperparametri. Heterogeni grafi vsebujejo več različnih tipov vozlišč in povezav, kar je pri vzorčenju soseščine koristno upoštevati in optimizirati vrednosti hiperparametrov za posamezne tipe povezav. Za reševanje tega problema predstavimo in analiziramo lasten algoritem, ki odpravi potrebo po časovno zahtevnem procesu obravnavanja in nastavljanja hiperparametrov za vse tipe vozlišč ter povezav. Prednosti algoritma z vidika časovne zahtevnosti in uspešnosti klasifikacije prikažemo na dveh grafih – akademskem grafu MAG240M, ki vsebuje več kot 240 milijonov vozlišč in nekaj manj kot 2 milijardi povezav, ter grafu znanja Freebase.In the last few years, graph neural networks have become one of the most active research fields in deep learning. They have been successfully applied for different problems, which can be represented as graphs, such as analysis of social networks, traffic prediction, and drug development. Despite the many successful results, there are still several challenges that need to be addressed when applying graph neural networks to large graphs, where there are hardware limitations. In the master\u27s thesis, we tackle the problem of using graph neural networks on massive heterogeneous graphs, where neighborhood sampling is performed on each layer of the graph neural network, whereas the hyperparameters define the sample size. Heterogeneous graphs contain many node and edge types, which should be considered during the neighborhood sampling for more effective learning and optimized during hyperparameter tuning. For this purpose, we design and analyze the algorithm to remove the need for the time-consuming process of setting all the hyperparameters for all edge types. The algorithm\u27s advantages are presented from the perspective of time complexity and classification efficiency on two graphs – the academic graph MAG240M, which contains more than 240 million nodes and a little less than 2 billion edges, and the knowledge graph Freebase