Transmitting huge amounts of data: design, implementation and performance of the Bulk Data Transfer mechanism

We present and discuss the design, implementation and performance of the bulk data transfer mechanism developed in the framework of the ALMA (Atacama Large Millimeter Array) Common Software (ACS) [2][6]. ALMA will be the largest millimetre wavelength astronomical interferometer in the world consisting of 64 12-meters antennas, and the need for transferring efficiently huge amounts of data arises consequently. For example, a typical output data rate expected from the correlator (the device responsible for the processing of raw digitized data from the antennas) will be of the order of 64 MB per second [1]. Since all subsystems in ALMA rely on a communication infrastructure (ACS), which is CORBA-based, this poses some problems to meet the stringent QoS (quality-of-service) requirements. It is well known in fact that DOC (Distributed Object Computing) middleware, such as CORBA, increases the packet latency due to marshalling/de-marshalling, to usage of the IIOP protocol, etc. To cope with ALMA requirements and to overcome the CORBA potential bottleneck, we developed a transfer mechanism based on the ACE/TAO CORBA Audio/Video (A/V) Streaming service. This architecture uses CORBA for handshaking, but allows an efficient data transfer by creating out-of-bound stream(s) of data (i.e. bypassing the CORB