Platform for Parallel Processing of Intense Experimental Data Flow on Remote Supercomputers

AbstractModern experimental facilities generate large amounts of data that should be processed, saved to hard disk and presented to the user as fast as possible. However, in-situ data analysis requires technical resources which are often not available. The existence of accessible high-speed networks allows to forward data processing and storage to a remote supercomputer centers and datacenters. These capabilities can be realized through the development of architectural solutions for effective data transmission trough a long-distance high-speed networks, data input/output and data distribution over computers and data storage systems. In this paper, we describe the results of investigations into the development of a software platform for parallel processing of intense experimental data-streams on ICMM UB RAS (Perm) and IMM UB RAS (Yekaterinburg) supercomputers, interconnected by a high-speed network. The reported studies was partially supported by RFBR, research project No. 14-07-96001-r_ural_a and by Program of UD RAS, project No 15-7-1-25