Extracting Disaster Variables with Deep Learning Methods to Improve Earth Science Data Set Retrieval

Abstract:

NLP (Natural Language Processing) methods have made great advances, mainly in the introduction of transfer learning models such as BERT (Bidirectional Encoder Representations from Transformers). These tools provide a strong baseline for NLP tasks that can then be fine tuned using more domain specific data. In our case, we collected a data set of scientific research papers focusing on various natural disasters. By fine tuning a pre-trained model on a natural disaster topic classification task, we were able to use the results to extract keywords from those research papers. These keywords provide both a summarized understanding of the important concepts and scientific variables pertaining to the natural disasters. We extracted these keywords using the attention values that are calculated by the BERT model to aid in its classification task. Aggregating these values on a per-token basis allowed us to find the tokens that BERT paid the most attention to when predicting classes for input text and therefore had the greatest relevance in relation to the topic class. This work is one component in a larger, in-development pipeline dedicated to extracting text variables from scientific sources. The end goal for these variables is to improve semantic links between user queries in a search engine and the datasets that search engine references. These variables can be connected with variables pertaining to specific datasets to facilitate better data set retrieval and improve the utility of the search engine.