Damped Newton Method - an Ann Learning Algorithm

Abstract

This paper presents a new learning algorithm for training fully-connected, feedforward artificial neural networks. The proposed learning algorithm will be suitable for training neural networks to solve approximation problems. The framework of the new ANN learning algorithm is based on Newton's method for solving non-linear least squares problems. To improve the stability of the new learning algorithm, the Levenberg-Marquardt technique for safe-guarding the Gauss-Newton method is incorporated into the Newton method. This damped version of Newton's method has been implemented using FORTRAN 77, along with some other well-known ANN learning algorithms in order to evaluate the performance of the new learning algorithm. Satisfactory numerical results have been obtained. It is shown that the proposed new learning algorithm has a better performance than the other algorithms in dealing with function approximation problems and problems which may require a high precision of training accuracy.