Training Neural Networks by Adaptive Random Search Techniques
Publication: Journal of Engineering Mechanics
Volume 125, Issue 2
Abstract
A relatively simple stochastic optimization procedure based on the adaptive random search algorithm is presented to train artificial neural networks of the type encountered in applied mechanics applications. After discussing some essential features of the algorithm that influence its search efficiency, a procedure is outlined for replacing the back-propagation training approach by the new method in order to train networks involving high-dimensional parameter vectors. The method is successfully used in conjunction with a multilayer network involving a parameter vector of very high dimension. It is shown that the adaptive random search approach shifts the training effort from the user to the computer by exchanging additional computer search effort for easier training tasks on the part of the user. Extensive simulation studies are presented to provide statistically significant results related to the characteristics of the stochastic training approach. Guidelines are provided for applying the method to generic neural network training episodes.
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Brooks, S. H. ( 1958). “Discussion of random methods for locating surface maxima.” Operations Res., 6, 244–251.
2.
Brown, M., and Harris, C. ( 1994). Neurofuzzy adaptive modeling and control. Prentice-Hall, Englewood Cliffs, N.J.
3.
Geman, S., and Geman, D. ( 1984). “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.” IEEE Trans. on Pattern Analysis and Machine Intelligence, 6, 721–741.
4.
Gill, P. E., Murray, W., and Wright, M. ( 1981). Practical optimization. Academic, New York.
5.
Goldberg, D. E. ( 1989). Genetic algorithms in search optimization and machine learning. Addison-Wesley, New York.
6.
Hassoun, M. H. ( 1995). Fundamentals of artificial neural networks. MIT Press, Cambridge, Mass.
7.
Holland, J. H. ( 1975). Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, Mich.
8.
Housner, G. W., et al. (1997). “Structural control: Past, present and future.”J. Engrg. Mech., ASCE, 123(9), 897–971.
9.
Lippmann, R. P. ( 1987). “An introduction to computing with neural nets.” IEEE ASSP Mag., 4–22.
10.
Lippmann, R. P. ( 1980). “Pattern classification using neural networks.” IEEE Comm. Mag., 47–69.
11.
Masri, S. F., Bekey, G. A., Caughey, T. K., and Van de Velde, E. ( 1995). “Adaptive stochastic optimization using multiprocessors.” Appl. Mathematics and Computation, 72, 225–257.
12.
Masri, S. F., Bekey, G. A., and Safford, F. B. ( 1976). “An adaptive random search method for identification of large scale nonlinear systems.” Proc., 4th IFAC Symp. of Identification and Sys. Parameter Estimation.
13.
Masri, S. F., Bekey, G. A., and Safford, F. B. ( 1980). “A global optimization algorithm using adaptive random search. Appl. Mathematics and Computation, 7, 353–375.
14.
Masri, S. F., Chassiakos, A. G., and Caughey, T. K. ( 1993). “Identification of nonlinear dynamic systems using neural networks.” Trans. of the ASME J. Appl. Mech., 60, 123–133.
15.
Masri, S. F., Nakamura, M., Chassiakos, A. G., and Caughey, T. K. (1996). “A neural network approach to the detection of changes in structural parameters.”J. Engrg. Mech., ASCE.
16.
Metropolis N., Rosenbluth, A., Teller, A., and Teller, E. ( 1953). “Equation of state calculations by fast computing machines.” J. Chemical Phys., 21(6), 1087–1092.
17.
Miller, W. T., Sutton, R. S., and Werbos, P. J. ( 1990). Neural networks for control. MIT Press, Cambridge, Mass.
18.
Munson, J. K., and Rubin, A. I. ( 1959). “Optimization by random search on the analog computer.” IRE Trans. Electronic Comp., EC8, 2, 200– 203.
19.
Rumelhart, D. E., and McClelland J. L., eds. ( 1986). Parallel distributed processing. MIT Press, Cambridge, Mass.
20.
Widrow, B., and Lehr, M. A. ( 1990). “Thirty years of adaptive neural networks: Perceptron, madaline and backpropagation.” Proc., IEEE, 78(9), 1415–1441.
21.
Widrow, B., and Stearns, S. D. ( 1985). Adaptive signal processing. Prentice-Hall, Englewood Cliffs, N.J.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 125 • Issue 2 • February 1999
Pages: 123 - 132
History
Received: Jan 20, 1998
Published online: Feb 1, 1999
Published in print: Feb 1999
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.