Detecting abnormal signals from two wavelengths of MFRSR

Abstract:

The U.S. Department of Agriculture (USDA) UV-B Monitoring and Research Program (UVMRP) focuses on monitoring the surface ultraviolet (UV) irradiance across the United States, and provids 3-minute UV spectral irradiance (7 wavelengths) and visible spectral irradiance (6 wavelengths) using Multifilter Rotating Shadowband Radiometers (MFRSR). Occasionally there are instrumentation problems and failures due to aging, lightning strikes, power fluctuations, etc., which can cause an abnormal time series of voltage for one or more wavelengths. These anomalies can significantly deviate from expected general patterns that would follow physical laws (e.g., Beer-Lambert Law). An automated quality control (QC) process is desirable to detect these abnormal signals to flag the corresponding problematic measurements and not provide them to the public. In addition, these QC codes will alert service personnel to replace or repair the affected instrument. In this study, it is assumed there are two daily time series of two adjacent wavelengths with strong physically inherent relationships, and two instruments that generated these two time series are unlikely to fail simultaneously. The times series are divided into shorter segments (e.g., 0.5-hour for each segment), which can approximate a linear correlation of these two times series within each segment. In this way, comparing the two original time series or modeling the correlation of them can be turned into the analysis of the correlations by segments. Statistical analyses of these segmental correlations can help to automatically identify the possible problematic signals for a given wavelength. Typically, such data anomalies are detected by personnel visually examining the data graphs. Current experiments using 368nm and 415nm wavelengths from the Illinois and Oklahoma network sites show that this method can successfully detect most of these known anomalies. In addition, these experiments successfully identified some other problematic signals that are not easily detected by human visual inspections, implying that this method is a potentially effective automatic tool for daily QC processes. Furthermore, the methodology of segmental correlation analysis can be applied into other disciplines to study time series or carry out QC processes.