Over the last few decades, it has become more important than ever to provide accurate forecasts for severe hazards that have become more common due to climate change effects. Over this time, several forecasting experiments have been performed with increasing computer power to better our understanding of these hazards. Currently, severe thunderstorms are diagnosed through a Warn-on-Detection paradigm, which bases severe warnings on storm reports or live radar data. To increase severe warning lead times, a Warn-on-Forecast has been developed, which focuses on the forecast evolution of ensemble systems to focus on probabilistic guidance of individual thunderstorm hazards. From this, the Warn-on-Forecast System (WoFS, defined in Section 1.3) and Warn-on-Forecast Hybrid System (WoF-Hybrid, defined in Section 1.5) have been developed.

The purpose of this study is to demonstrate and evaluate the capability of WoFS and WoF-Hybrid for predicting short-term severe weather forecasts that occurred during 2021 and help to identify room for future improvement. In addition to the 3 km grid-spacing, 18-member Experimental Warn-on-Forecast System (WoFS), which uses a Gridpoint Statistical Interpolation- Ensemble Kalman Filter (GSI-EnKF) data assimilation method, a 1.5 km grid-spacing, single member hybrid three-dimensional ensemble–variational data assimilation (3DEnVar System; referred to hereafter as WoF-Hybrid) has been tested for several years in NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiments (SFE). Whether WoF-Hybrid exhibits attributes that complement WoFS is an open question. This is addressed by applying a spatial verification method, Fractions Skill Score (FSS), and object-based verification measures to 31 cases between April–December 2021. For the spatial verification method, WoF-Hybrid reflectivity FSSs are significantly lower than the WoFS member average at forecast initialization. Scores converge after one hour and are higher for WoF-Hybrid through the rest of the period. The difference in skill at initialization may be attributed to a higher reflectivity bias and a greater number of spurious convective cells in WoF-Hybrid.

The object-based method applied to reflectivity shows similar probability of detections (PODs) and false alarm ratios (FARs) for both systems. For updraft helicity (UH), however, WoF-Hybrid yields better PODs at all lead times. WoF-Hybrid better resolves UH swaths seen in plan-view analysis of forecast output, which tend to be narrow in the verification dataset. Partitioning cases based on event severity per the number of local storm reports, this study shows WoF-Hybrid has higher PODs and lower FARs than the WoFS member average for both reflectivity and UH in eight high-end events. WoF-Hybrid forecast reflectivity objects are both closer in size and location to observed objects than are WoFS objects.

Three severe weather events of 26 May and 27 May (multi-mode severe and heavy rain), and 10 December (rare winter tornado outbreak) are selected for detailed investigation. In each May case, PODs and FARs for both prediction systems were similar; however, for 10 December, WoF-Hybrid far outperformed WoFS in forecasts of the single most impactful thunderstorm in the dataset (quad-state tornadic supercell). These results suggest that using both WoF-Hybrid and WoFS forecast guidance may better support NWS forecasters' warning and forecast decisions.