Multi-scale object-based probabilistic forecast evaluation of WRF-based CAM ensemble configurations

Abstract

Recent developments in numerical weather prediction have included increased usage of ensemble forecasts in contrast to single, deterministic forecasts. In particular, convection-allowing model (CAM) ensembles have been utilized as they contain a distinctive ability to predict convective initiation location, mode and morphology. Such information can be extremely useful to forecasters predicting severe weather threats associated with particular storm modes and their morphological transitions spatially and temporally. Numerous studies verifying CAM ensemble forecast performance have been conducted. However, the primary focus from these studies have been on neighborhood-based verification of spatial coverage rather than convective mode and morphology. A limitation of neighborhood-based verification methods is their inability to adequately represent storm scale features of greatest subjective interest to forecasters. In contrast, as an alternative to neighborhood-based methods, recent object-based probabilistic framework has been introduced to assess forecasts, providing a unique setting to re-evaluate aspects of optimal CAM ensemble design with a focus on mode and morphology prediction. Herein, we adopt an optimized object-based probabilistic (OBPROB) forecasting method in conjunction with a traditional neighborhood-based (NMEP) method to evaluate forecasts of four separately designed 10-member ensembles. The ensemble configurations evaluated include single-model, single-physics (SMSP) configurations, single vs. multi-model, single vs. multi-physics, and multi-model vs multi-physics. Due to the implementation of multiple verification techniques and separate verification of objects on different spatial scales, comparisons of forecast performance in terms of mesoscale precipitation locations and forecasted storm mode and morphology can be made explicitly, leading to insight on optimal CAM ensemble design for specific storm modes and morphologies. Both OBPROB and NMEP frameworks show ensembles with a more diverse member-to-member design improve probabilistic forecasts over single-model, single-physics designs through greater sampling of forecast uncertainties. However, individual case studies suggest each methods’ systematic results are reliant on separate forecast features. For example, subjective analysis shows neighborhood-based verification, even at high reflectivity thresholds, are impacted by both convective and stratiform precipitation whereas the OBPROB method explicitly focuses on convective precipitation. OBPROB verification of storm morphology forecasts also display the need for probabilistic calibration to improve ensemble reliability.