Using Publish-Subscribe Methodology to Improve Water Data Sharing

Abstract:

The State of New Mexico recently launched the Water Data Initiative (NM WDI), which aims to provide comprehensive access to water data in the state. Following Findable, Accessible, Interoperable, and Reusable (FAIR) principles, the initiative is collating data from state databases, researchers, and NGOs (among others) to eventually provide a federated view for all interested parties. Development of the NM WDI platform has quickly identified the multiple locations that a given dataset currently resides within the state. This not only creates unneeded replication and orchestration of data from multiple locations for publishers and subscribers but also causes confusion regarding data origins. In this presentation, we will discuss the potential for simplifying data repository interactions to follow more of a “Publish-Subscribe” approach. This methodology, which has become essential in Enterprise Integration and “Big Data” architectures, provides solutions to sharing data across organizations. Through Publish-Subscribe, data providers publish the data and designated taxonomy to a topic without knowledge of which subscribers are listening. This decoupling between publishers of content and interested subscribers eliminates the concern of where to send content and also ensures that metadata is accessible to a broad audience. Such approaches can advance the current state of data publication models to reduce the burden of repository selection while increasing accessibility and interoperability of published data.