OPTIMIZATION OF DATABASE SIZE AND DATA ACCESS TIME TO STORE PARAMETERS WITH MULTIPLE RECORDING FREQUENCIES

A non-relational database was developed at JSC Element for the recording operating parameters of GTE and systems during the test flight. The database is modernized version of one which is a part of the GTE test bench. The purpose of the database is the recording and analysis of GTE and systems operating parameters. The parameters are measured (collected) with multiple frequencies in range from fractions of Hz to tens of kHz. Number of recorded operating parameters is 1000+. Correspondingly the database file is huge and may reach tens of Gbytes. Number of different parameters recording frequencies is 10…20 (e.g. 20 parameters – 10 Hz, 30 parameters – 50 Hz, 20 parameters – 10000 Hz). Data reading and chart displaying should be possible as right after the test flight as just in parallel with database recording. There should be no significant delays in getting the data displayed independently from the zoom level chosen. The user should see the result in time comparable with time of the displayed chart perception making the user feel the smooth application operating and not interrupting the process of test flight result comprehension – main purpose of our product. However, min-max chart displaying (minimal and maximal value are found to render one-pixel vertical lines) requires processing of all the data for the corresponding time interval. Correspondingly, time to process data depends on that. In order to provide data quickly on user’s actions the sample is read according to the chart horizontal resolution. Then whole the data is processed in background mode to build min-max chart. This dual approach allows to surf the database quickly losing no details when the surfing is stopped. Modern computers and SSD drivers provides high I/O performance. However huge files can be neither quickly read nor stored in RAM. Therefore, to read data in parallel with the recording only the choice is to store the data blocks which are either indexed by the time stamp or fixed size. Our solution is based on fixed size approach with the constant offset of each parameter storing. Size of the block is calculated as least common multiple of all the frequencies the parameters are recorded with. According to our statistic the disk usage rate is 95%. As result the time to display the sample for 2000 timestamps is less than 0.5 sec independently for the database size and the displayed duration of the test flight