A Hybrid Approach to determine Earth Science Datasets in Scientific Publications.

Abstract:

NASA Data Centers provide the public with thousands of datasets that result in published papers, reports, and conference proceedings. Collecting accurate metrics on these datasets is key to connecting different areas of knowledge and evaluating the datasets’ impact. While most of the datasets have Digital Object Identifiers (DOIs), most publications do not cite them hampering the automated search of these publications. Instead, articles mention attributes like instrument, mission, resolution, organization, or a publication describing the dataset. Often only domain experts can deduce the dataset that was used. The lack of a citation slows the spread of information and reduces the research’s impact. With thousands of papers produced each year, an automated means of labeling datasets used is critical. This paper explores a hybrid approach of two natural language processing models to find and label the datasets used within Earth science papers. Each approach outputs the possible mentions and the probability the dataset is in the paper. The first approach is a three step Question answering (QA) model. First, a reading comprehension (RC) model which uses context to answer a question, in this case “what datasets are used in this paper?”. Next, a fine entity typing model labels the RC model’s answer. Finally, a one layer neural net assesses the accuracy of the labelled answer. The second approach aims to obtain a specific reference rather than finding the answer through context. It combines a named entity recognition model, which identifies explicit mentions of the dataset, with heuristics to label dataset mentions. This approach finds specific mentions but may fail to identify context based references. By combining it with the much more broad QA model, both the explicit and context based mentions can be found. Additionally, the approaches can verify each other which generates more certainty about the labels. For each article analyzed, the final output is the combined mentions of both models and a limited number of datasets most likely to be in the article. This output is further reviewed by a domain expert and correct labels are determined to be further used as additional training data. Because all the possible mentions are gathered by the model, the domain expert can quickly and easily label the papers resulting in large time savings.