Massive Date Sets Issues in Earth Observing

Current and next decade global Earth observing, other remote sensing and related climate analysis data collected by space and operational U.S. agencies such as NASA and NOAA, the European ESA, the Japanese NASDA and other international agency missions of India, China, Russia, etc. will reach unprecedented data volumes, exceeding many petabytes. This is rushing in a new era in Earth system science. Along with the technology and data volumes afforded by remote sensing, there has been an unprecedented increase of capabilities in distributed data systems in the last few years. The existence of the Internet and the World Wide Web afford users access to data at diverse distributed sites that would had been very difficult in the past and only available to specialists. These data can be accessed by a variety of scientists, applications experts and the general public. Yet, the unprecedented large volumes of such missions are also presenting formidable challenges to wide user access and require both higher bandwidths of future Internet systems as well as more focused, user-centered data productions. Specialized data productions are best achieved in federated data systems. The large data volumes present a challenge of access, storage and distribution. What is most important is not just the volume of the data itself but the information they contain. A data system\u27s usefulness is related to the ease that its users can search and access products and as such obtain information on the actual content of data before proceeding to order large volumes of data sets which may or may not serve their needs. Here we explore different functionalities in distributed Earth observing data systems and associated interoperability options. As an example, we examine options in the Earth Science Information Partners Federation (ESIPs) funded by NASA to extend the usage of NASA remote sensing data holdings. Results of several interoperability options applicable to federated systems are also presented. We also examine challenges presented by the use of regional remote sensing missions such as hyperspectral imaging.https://digitalcommons.chapman.edu/scs_books/1011/thumbnail.jp