Modeling of gathering and analysis system for textual media

This report presents an automated data scanning and processing system of articles and their comments of the most popular news portals in Lithuania. The main goal of this bachelor thesis was to create a system that collects and compares the data of articles and their comments of portals by displaying their graphical representation to system users with the ability to filter and receive only the most relevant information at that time. The goal was reached and all requirements were fully implemented and ensured their operation by implementing logs in database table by tracking and saving changes in various system parts. At the end of the thesis these conclusions were made: • after analyzing similar systems and tools, it has been observed that the spectrum of their use is very wide and can continue from the items price tracking tool to the complex plagiarization detection systems. The range of technologies used by these systems is also diverse and can even achieve optical data processing using Optical Character Recognition; • for the creation of a universal portal template using HTML tags and attributes, data can be accessed not only from any news portal, but also from web pages like blogs or similar which have text block structures in them. It also allows to react very quickly and without any major e ort to web pages structure changes; • it has been noted that over a period of more than 3 weeks, the system collects about 15,000 unique articles, with their comments and system activity registrations, in a full-time system reading two portal data and reading about 50 days of their information. This database con- tains about 300 MB of data, which suggests that reading one year’s data for one portal would require just over 1GB of free hard disk space