A distributed imaging framework for the analysis and visualization of multi-dimensional bio-image datasets, in high content screening applications

This research presents the DFrame, a modular and extensible distributed framework that simplifies and thus encourages the use of parallel processing, and that is especially targeted at the analysis and visualization of multi-dimensional bio-image datasets in high content screening applications. These applications typically apply pipelines of complex and time consuming algorithms to multiple bio-image dataset stream and it is highly desirable to use parallel resources to exploit the inherent concurrency, in order to achieve results in much reduced time scales. The DFrame allows pluggable extension and reuse of models implementing parallelizing patterns, and similarly provides for application extensibility. This facilitates the composition of novel parallelized 3D image processing application. A client server architecture is adopted to support both batch and long running interactive sessions. The DFrame client provides functions to author applications as workflows, and mediates interaction with the server. The DFrame server runs as multiple cooperating distributed instances, that together orchestrate to execture tasks according to a workflow's implied order. An inversion of control paradigm is used to drive the loading and running of the models that themselves then coordinate to load and parallelize the running of each task specified in a workflow. The design opens up the opportunity to incorporate advanced management features, including parallel pattern selection based on application context, dynamic 'in application' resource allocation, and adaptable partitioning and composition strategies. Generic partitioning and composition mechanisms for supporting both task and data parallelism are provided, with specific implementation support applicable to the domain of 3D image processing. Evaluations of the DFrame are conducted at the component levelm where specific parallelizing models are applied to discrete 3D image filtering and segmentation operators and to a ray tracing implementation. A complete integrated case study is then presented that composes component entities into multiple image processing pipeline to more fully demonstrate the power and utility of the DFrame, not only in terms of performance, but also to highlight the extensibility and adaptability that permeates through the design, and its applicability to the domain of multi-dimensional image processing. Results are discussed that evidence the utility of the approach, and avenues of future works are considered