Serving netCDF-based NASA EOS Data Products with ArcGIS Image Service

Abstract:

CF-NetCDF is one of the recommended data formats for the archive and distribution of NASA EOS science data. Furthermore, it is becoming the main archival format in the Goddard Earth Sciences (GES) Data and Information Services Center (DISC). However, netCDF is not the best format to work with Geospatial Information System (GIS) processing tools, such as ArcGIS or QGIS, primarily due to its complexity and performance issues, when dealing with large volumes of data in online platforms. To enable GIS communities to use these NASA data which includes multivariable, multi-spatiotemporal dimension, and rich science metadata, ArcGIS GIS software suite has been used in NASA data centers to analyze and publish EOS data. It is recommended that netCDF data be converted to other formats such as MRF and GeoTIff when they are served through the ArcGIS image servers. While it is always feasible to convert archived EOS data from netCDF to geoTiff and MRF, this requires a duplication of archived data. In this presentation, we describe an approach that directly serves long time data records in the netCDF format through ArcGIS image services without having to convert the data into another format, and still has good performance. We break a supposedly single point service for a huge data store into spatiotemporally and variable-based multi-level hierarchical services at the backend and create one or a few proxy services at the frontend for users to access. The proxy services will dispatch user requests to the appropriate backend services based on the user’s request criteria. Such a service architecture not only avoids the need to convert and duplicate the huge archival data but also provides a much more flexible service framework for achieving optimal service performance and scalability.