A Cloud-based Science Gateway for the Atmospheric Sciences

The atmospheric modeling community in the United States has relied mostly on high performance computing facilities (e.g., NCAR-Wyoming Supercomputing facility and XSEDE resources) and local computing clusters to perform weather prediction research. Cloud computing represents a fundamental change in the way IT services are developed, deployed, operated, and paid for, placing science communities in the middle of a major paradigm shift. The cloud appears to be a potential avenue for atmospheric science researchers to gain access to significant and seamless computing resources beyond the traditional supercomputing centers for end-to-end weather and climate modeling studies, democratizing access to high performance computing resources, vast amounts of storage, and unprecedented access to large volumes of data. The three common models of cloud service, Software as a Service, Platform as a Service, and Infrastructure as a Service, are well recognized. There is also broad consensus that as data volumes grow rapidly, it is important to reduce data movement and bring processing and computations to the data. The aim is to give scientists an ecosystem that includes data, tools, models, and workflows. Instead of moving data to users, bring analysis and visualization to data – so called data proximate workbench capabilities. Unidata, a cyberinfrastructure facility, is developing data-driven scientific workflows using cloud computing technologies for accessing, analyzing, and visualizing geoscience data. The Unidata Science Gateway (http://science-gateway.unidata.ucar.edu) has implemented Unidata-related technologies, hosted on Jetstream (https://jetstream-cloud.org), a cloud facility funded by the U. S. National Science Foundation. Through the Science Gateway, researchers presently can make use of well-integrated resources. Currently, the Unidata Gateway provides the following functionalities: 1.A data ingest service that moves real-time data from over 30 meteorological data streams and in excess of 1 TB/day, into the Unidata Gateway via the Local Data Manager software, a TCP/IP-based data transfer technology. Those data include radar, satellite, surface, upper-air, ship, aircraft and other observations as well as forecast model output from several operational weather prediction centers. 2.Remote access to the above data via different protocols, including access to subsets of data 3.Data transformation and format conversion 4.Extensive data analysis capabilities 5.Visualization of meteorological data 6.A collection of Jupyter Notebooks for data analysis 7.Pre-configured Docker virtual images of tools 8.Access to Advanced Weather Information Processing System for remote data analysis and visualization Our hope is that by combining the resources available in the gateway – and coupling them with HPC (https://www.xsede.org) resources – community members can create and use end-to-end workflows in the cloud. We will present our initial work in developing the Unidata Science Gateway as well as our future directions to benefit the atmospheric sciences community