IN042-0008
Hybrid Serverless Cloud and Supercomputing Workflow to Support Methane Plume Detection and Regional Analysis

Wednesday, 16 December 2020
Poster
Joseph Charles Jacob1, Brian D Bue1, Daniel Cusworth1, Kevin Michael Gill1, Winston Olson-Duvall1, E. Natasha Stavros1, Robert K Tapella1, Andrew K Thorpe1, Vineet Yadav1, Elizabeth Yam1 and Riley M Duren1,2, (1)Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, United States, (2)University of Arizona, Tucson, AZ, United States
Abstract:
NASA and ESA collectively host a variety of satellite (e.g., TROPOMI) and airborne (e.g., AVIRIS-NG) observations related to methane in Earth’s atmosphere at local, regional and global scales. However, current tools to analyze these varied data are ad hoc and require large data download and duplication of not just the methane products but also ancillary products (e.g., NOAA HRRR and RTMA winds) needed to estimate local methane emission rates and for regional inversions. This makes it costly to analyze the data to understand the major methane sources and their consequences on the Earth system processes. NASA’s Multi-scale Methane Analytic Framework (M2AF) project will develop and mature technologies to bridge this gap and enhance the data discovery, efficient processing, analysis and use of the available methane data.

M2AF will deliver an end-to-end automated workflow with a hybrid architecture that will seamlessly span Amazon Web Services (AWS) cloud and NASA’s supercomputing resources. This includes components to: (i) Harvest the data to AWS S3 object storage; (ii) Apply machine learning (e.g., convolutional neural network) to detect and quantify methane plumes from AVIRIS-NG L1 calibrated radiances; (iii) Analyze local prevailing winds to estimate methane emission rates for each plume; (iv) Attribute plumes to sources and specific infrastructure elements; (v) Run regional methane inversions using gridded methane and wind measurements on AWS and NASA’s Pleiades supercomputer; (vi) Compute high resolution analytics via an adaptation of the Science Data Analytics Platform (SDAP) deployed to AWS; and (vii) Control and manage the data, algorithms and compute resources via a web interface.

This cloud-scale automation will greatly enhance our ability to identify local anomalies in methane measurements and accurately associate plumes with sources. We will report mid-term progress in our 2-year schedule and describe future plans.

<< Previous Abstract | Next Abstract