Statistical and Machine Learning Methods for Evaluating Emissions Reduction Policies under Changing Meteorological Conditions

Abstract:

Impacts of energy and environmental policy on air quality and human health are potentially confounded by underlying meteorological variability, which need to be correctly adjusted for in order to attribute observed changes in air quality to the policies. However, there is little consensus on what statistical methods should be used to correct for meteorological variability when estimating policy effects on air quality. Most previous studies use a multiple linear regression (MLR) model with basic meteorological variables to correct for meteorological variability, but the ability of such models to assess policy signals remains unknown. Here, we quantify the performance of MLR model and other regression techniques using a model experiment, and show that they do not perform well in correcting for the meteorological variability. To do this, we simulate the impacts of emissions control policies in the U.S. from 2011 to 2017 with the chemical transport model GEOS-Chem and examine the 7-year linear trend in daily PM_2.5 and O₃. We simulate two sets of scenarios – “observed scenarios” with assimilated meteorological inputs (with interannual variability) and “counterfactual scenarios” with constant meteorological inputs. We then attempt to reproduce the policy-driven trends in the counterfactual scenarios, by using quantitative methods to remove meteorological variability in the observed scenarios. Compared with the policy signal in the counterfactual scenarios, the trend estimated using MLR is biased by 38% (PM_2.5) and 115% (O₃). The bias could be reduced by more flexible machine learning models (e.g. random forest) that use both local and synoptic scale meteorological features, but remains statistically significant (26% for PM_2.5 and 36% for O₃). Our analysis suggests that previous analyses using MLR models could result in biased estimates of policy effects. We conclude by proposing statistical methods that could better correct for meteorological variability in estimating policy impacts with observational data, and discussing the potential limit of such statistical corrections due to the ignorance of emissions-meteorology interactions.