Data quality management models and simulations for data quality evaluation

Prodiranje podatkovno gnanih storitev v organizacije prinaša poleg novih poslovnih procesov, možnosti in priložnosti tudi vrsto izzivov. Kakršnokoli na podatkih utemeljeno modeliranje ima lahko podlago le v kakovostnih, realno stanje natančno odsevajočih podatkih. V magistrskem delu je predstavljen model, ki s spremljanjem kakovosti podatkovnih virov omogoča njihovim upravnikom pridobiti informacije o kakovosti posameznih podatkovnih tokov, kar jim posledično pomaga pri odpravljanju nepravilnosti in dvigu njihove kakovosti. Modeli kakovosti podatkov iz literature lahko vključujejo kar več deset kategorij kakovosti, med katerimi so najpogosteje uporabljene popolnost, rednost, usklajenost in natančnost. Vsaka kategorija kakovosti ima definirana različna preverjanja oz. validacije, s pomočjo katerih se šteje ali kako drugače analizira meritve, ki ne izpolnjujejo pogojev te kategorije kakovosti in tako se pridobi kazalnike kakovosti podatkov. Predstavljen je tudi taksonomski pregled omenjenih kategorij kakovosti, področij, na katerih se uporabljajo, ter kriterijev za primernost njihove uporabe. Model in indikatorji kakovosti so ilustrirani s primerom vrednotenja kakovosti podatkov za potrebe napovedovanja proizvodnje sončnih elektrarn. Rezultati kažejo na to, da delež nekakovostnih podatkov občutno vpliva na napake pri napovedovanju fotovoltaične proizvodnje električne energije. Pomembno se zdi izpostaviti, da je dodana vrednost magisterija koncept vrednotenja modelskih napak in samostojno dodana preverjanja ter opis modela kazalnikov kakovosti, kar se je razvijalo in dopolnjevalo skozi več uspešno izpeljanih projektov na to temo.Penetration of data-driven services into organizations introduces several challenges, in addition to new business processes and opportunities. Any data modeling can only be based on high quality data that accurately reflects the state of the system. In the master\u27s thesis, a model is presented, which, by monitoring the quality of data sources, enables its users to obtain information on the quality of individual data streams. That helps them eliminate irregularities, increase data quality and, when appropriate processes are implemented, gives access to edited data with defined data quality indicators. The data quality models found in the literature can include dozens of data quality dimensions. The most commonly used dimensions are completeness, timeliness, consistency and accuracy. For each quality dimension, different validations are defined. With them, we can analyse the measurements that do not meet the conditions of particular quality dimension. Master\u27s thesis presents a taxonomic review of mentioned quality categories, domains on which they are used, and the criteria for the suitability of their use. The model and quality indicators are illustrated with an example of data quality evaluation of forecasting the photovoltaic production. The results indicate that the fraction of bad data quality significantly affects the errors when predicting the photovoltaic electricity production. It is important to point out that the added value of the Master\u27s thesis is the concept of evaluation of model errors and the independently added validations and descriptions of the model of quality indicators. This model has been developed and tested through several successfully implemented projects on this topic