Verification of Precipitation Forecast by Pattern Recognition

Abstract

 This paper proposes a new verification metric, the Pattern Similarity Index (PSI), that can simultaneously evaluate location errors and shapes of rainfall areas. Pixel-by-pixel verification methods such as the threat score and root mean squared error involve difficulties in evaluating location errors and shapes of rainfall areas and in addition small rainfall areas. To address these difficulties, various object-based methods have been developed. However, object-based methods tend to be complicated and computationally expensive. Therefore, PSI adopts a simpler, computationally more efficient algorithm, as follows: Firstly, bounding rectangles of individual rainfall areas are computed, and neighboring rectangles are combined so that they are treated as a single precipitation system to mimic how human recognizes. Next, shape parameters are computed for each integrated bounding rectangle. For each pair of the observed and forecasted rainfall areas, the location error weighted by the differences of the shape parameters is used as the verification score. If no observed rainfall area with a similar size exists near a forecasted rainfall area, this location-error-based score of the forecasted area is set to a large value. The integration method of the bounding rectangle and the precipitation threshold are the only tunable parameters in this method, and we repeat computing the verification score by varying these parameters. The best value is used as the final verification score.

 Idealized cases showed the ability of PSI to evaluate location errors and differences in the shape parameters. A real case with global precipitation nowcasting showed that the proposed evaluation value increased almost linearly with the forecast time, whereas the threat score and root mean squared error tended to saturate as the forecast time increased, showing a potential advantage of PSI. Comparison of PSI with another object-based method revealed its advantage in its computational efficiency, while providing similar verification scores.