High rate packet transmission on 10 Gbit/s Ethernet LAN using commodity hardware

Commodity links, such as Gigabit Ethernet, are more and more often used in Data Acquisition, Event Building and Trigger systems, in the framework of High Energy Physics (HEP), due to relatively limited costs, maintainability and upgradability. The demand of data throughput in HEP is increasing following the rise of the physical event rate and of the number of electronic channels to be read out. Advances in data link technologies allow, as well, to reduce the number of the on-line event filter stages, permitting a finer detector granularity to be available when making the trigger decision, and thus granting a larger selection efficiency, improving at the same time the background rejection. We report on the measurements performed to test the maximum achievable IP transfer rate through a 10 Gigabit Ethernet (10GbE) link, when transmitters and receivers are commodity PCs, in the prospect of an employment in the on-line systems of HEP experiments and for the network and storage design of offline analysis data centers. In real operating conditions, the maximum transfer rate through a network link is limited not only by the capacity of the link itself, but also by the capacity of the data bus and by the ability of the CPUs and of the Operating System kernels of handling the data transfer rate and of processing the interrupts raised by the network interfaces in due time. We present measurements of the maximum achievable IP transfer rates and of the corresponding CPU loads of the sender/receiver processes and of the interrupt and soft-interrupt handlers as a function of the packet size, using 10GbE NICs, either using standard or "jumbo" Ethernet frames, with the UDP and the TCP protocols. We also show the results of the same measurements obtained by simultaneously reading data from Fiber Channel links and by forwarding them through a 10GbE link, hence emulating the behavior of a disk server in a Storage Area Network which exports the data via 10GbE