Robust Robustness: A Sensitivity Analysis of MORDM with Competing Assumptions about Future States of the World

Abstract:

In this research we perform a sensitivity analysis of robustness calculations and Scenario Discovery (SD) in the Many-Objective Robust Decision Making (MORDM) framework. In MORDM, solutions are identified through many-objective optimization and then stress-tested in future States of the World (SOW) that sample uncertain model inputs. Solutions are ranked with a robustness metric such as satisficing or regret, and finally SD is implemented on a small selection of solutions to map regions of the uncertainty space to failure in system performance. The goal is to identify the most robust solutions, meaning they are least sensitive to the uncertainty sampled in the SOW. However, recent research shows that robustness ranking and SD can be sensitive to the distribution and correlation of the SOW, raising concern that the robustness of some solutions is relative to the experimental design of the SOW. To address this concern, we implement conditioned Latin Hypercube Sampling to create two competing SOW ensembles, which we name scenario-informed and maximally-diverse, respectively. The distribution and correlation of the scenario-informed SOW ensemble reflects historical and climate-change informed hydrology, whereas the maximally-diverse SOW ensemble samples the uncertainty space more uniformly. To assess the sensitivity of MORDM results to experimental design, we perform a case study in the Colorado River Basin. We rank candidate operating policies for Lake Mead according to their robustness and perform SD on the most robust solutions using each SOW ensemble. We expect this research to establish a computationally efficient method to utilize the MORDM framework with multiple SOW ensembles while providing insight to which robustness metrics and SD algorithms are less sensitive to experimental design of the SOW. Ultimately, this work will identify Lake Mead policies that are robust not only to uncertainty but also to analytic decisions in the MORDM framework.