Advances in Modern Information Technologies for Data Analysis in CRYO-EM and XFEL Experiments

A new approach to the organization of data pipelining in cryo-electron microscopy (Cryo-EM) and X-ray free-electron laser (XFEL) experiments is presented. This approach, based on the progress in information technologies (IT) due to the development of containerization techniques, allows one to separate user’s work at the application level from the developments of IT experts at the system and middleware levels. A user must only perform two simple operations: pack application packages in containers and write a workflow with data processing logic in a standard format. Some examples of containerized workflows for Cryo-EM and XFEL experiments on study of the spatial structure of single biological nanoobjects (viruses, macromolecules, etc.) are discussed. Examples of program codes for installing applied packages in Docker containers and examples of applied workflows written in the high-level language CWL are presented at the site of the project. The examples have comments, which may help an IT-inexperienced researcher to gain an idea of how to organize Docker containers and form CWL workflows for Cryo-EM and XFEL data pipelining