Abstract
The normalized risk-averting error (NRAE) training method presented in ISNN 2012 is capable of overcoming the local-minimum problem in training neural networks. However, the overall success rate is unsatisfactory. Motivated by this problem, a modification, called the NRAE-MSE training method is herein proposed. The new method trains neural networks with respect to NRAE with a fixed λ in the range of 106-1011, and takes excursions to train with the standard mean squared error (MSE) from time to time. Once an excursion produces a satisfactory MSE with cross-validation, the entire NRAE-MSE training stops. Numerical experiments show that the NRAE-MSE training method has a success rate of 100% in all the testing examples each starting with a large number of randomly selected initial weights.
This material is based upon work supported in part by the National Science Foundation under Grant ECCS1028048, but does not necessarily reflect the position or policy of the Government.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aarts, E., Korst, J.: The Neuron. Oxford University Press (1989)
Zurada, J.M.: Introduction to Artificial Neural Networks. West Publishing Company, St. Paul (1992)
Hassoun, M.H.: Fundamentals of Artificial Neural Networks. MIT Press, Cambridge (1995)
Principe, J.C., Euliano, N.R., Lefebvre, W.C.: Neural and Adaptive Systems: Fundamentals through Simulations. John Wiley and Sons, Inc., New York (2000)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Du, K.L., Swamy, M.: Neural Networks in a Softcomputing Framework. Springer, New York (2006)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C: The Art of Scientific Computing, 3rd edn. Cambridge University Press, New York (2007)
Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice Hall, Upper Saddle River (2008)
Lo, J.T.-H.: Convexification for data fitting. Journal of Global Optimization 46(2), 307–315 (2010)
Lo, J.T.-H., Bassu, D.: An adaptive method of training multilayer perceptrons. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2001), vol. 3, pp. 2013–2018 (July 2001)
Lo, J.T.-H., Gui, Y., Peng, Y.: Overcoming the local-minimum problem in training multilayer perceptrons with the NRAE training method. In: Proceedings of the 9th International Conference on Advances in Neural Networks (ISNN 2012), vol. Part I, pp. 440–447 (July 2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lo, J.TH., Gui, Y., Peng, Y. (2013). Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE-MSE Training Method. In: Guo, C., Hou, ZG., Zeng, Z. (eds) Advances in Neural Networks – ISNN 2013. ISNN 2013. Lecture Notes in Computer Science, vol 7951. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39065-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-39065-4_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39064-7
Online ISBN: 978-3-642-39065-4
eBook Packages: Computer ScienceComputer Science (R0)