InfiniCortex - From Proof-of-concept to Production

The global effort to build ever more powerful supercomputers is faced with the challenge of ramping up High Performance Computing systems to ExaScale capabilities and, at the same time, keeping the electrical power consumption for a system of that scale at less than 20 MW level. One possible solution, bypassing this local energy limit, is to use distributed supercomputers to alleviate intense power requirements at any single location. The other critical challenge faced by the global computer industry and international scientifc collaborations is the requirement of streaming colossal amounts of time-critical data. Examples abound: i) transfer of astrophysical data collected by the Square Kilometre Array to the international partners, ii) streaming of large facilities experimental data through the Pacifc Research Platform collaboration of DoE, ESnet and other partners in the US and elsewhere, iii) the Superfcilities vision expressed by DoE, iv) new architecture for CERN LHC data processing pipeline focusing on more powerful processing facilities connected by higher throughput connectivity. The InfniCortex project led by A*STAR Computational Resource Centre demonstrates a worldwide InfniBand fabric circumnavigating the globe and bringing together, as one concurrent globally distributed HPC system, several supercomputing facilities spanned across four continents (Asia, Australia, Europe and North America). Using global scale InfniBand connections, with bandwidth utilisation approaching 98% link capacity, we have established a new architectural approach which might lead to the next generation supercomputing systems capable of solving the most complex problems through the aggregation and parallelisation of many globally distributed supercomputers into a single hive-mind of enormous scale.Global Research PlatformInfiniCorte