Clustering methodologies applied to short-term ensemble forecasting: An exercise in data mining.

Abstract

In ensemble forecasting system, the divergence of the ensemble members during evolution may lead to the identification of clusters of different sizes in the finale state. Based on the size of each cluster, probabilities can be assigned objectively to different outcomes. Hence, identifying the clustering structures of the ensemble members is very significant in the ensemble forecasting system which in turn may lead to better forecasts for public consumption. In this work, we use a fundamental methodology from data mining, namely clustering, to detect and identify the clustering structures of the ensemble members during their evolution. To achieve this goal, in this research, we use two tools from the multivariate data analysis: cluster analysis and principal component analysis. Various clustering methodologies and cluster validation techniques are applied to the output of two short-term ensembles. The first one is a uni-model ensemble, which is the output of the Eta model, consists of 10 members. The second is a multi-model ensemble consists of 25 members. The multi-model, known as the Storm And Mesoscale Ensemble Experiment (SAMEX), is a project coordinated by CAPS at the University of Oklahoma. Four models CARPS, Eta, RSM, MM5) are used to construct the SAMEX ensemble data. The ultimate goal of our research is to automate the aspects of clustering the ensemble members in the ensemble forecasting system. Such automation is implemented by developing an expert system. Our research provides the infrastructures for building this expert system.