Application of Least Squares Support Vector Regression with Regrouping Particle Swarm Optimzation

Abstract

Least Squares Support Vector Regression (LSSVR) is a powerful machine learning tool. The performance of LSSVR is not only directly linked to the proper selection of its hyper-parameters, but also to the proper feature selection of the targeted dataset. In time series forecasting, features selection can be viewed as selecting the numbers of past data points. It became important for selecting a good combination of both these parameters and features, if we want to do any meaningful short-term forecasting for time series data. The existing parameter selection methods employ many optimizing techniques that range from grid search to neural networks and particle swarm optimization, but they all left the feature selection of the series to users. A novel method is proposed here to select both LSSVR parameters and the features of the time series at the same time. The real world data used in this study demonstrate the proposed method achieves better performance in terms of recursive short-term forecasting, when compared to existing standard PSO and grid search methods that focus on hyper-parameters selection and leaves the feature selection to Average Mutual Information (AMI).