Neural Networks. A General Framework for Non-Linear Function Approximation

Fischer, Manfred M. (2006): Neural Networks. A General Framework for Non-Linear Function Approximation. Published in: Transactions in GIS , Vol. 10, No. 4 (2006)

Abstract

The focus of this paper is on the neural network modelling approach that has gained increasing recognition in GIScience in recent years. The novelty about neural networks lies in their ability to model non-linear processes with few, if any, a priori assumptions about the nature of the data-generating process. The paper discusses some important issues that are central for successful application development. The scope is limited to feedforward neural networks, the leading example of neural networks. It is argued that failures in applications can usually be attributed to inadequate learning and/or inadequate complexity of the network model. Parameter estimation and a suitably chosen number of hidden units are, thus, of crucial importance for the success of real world neural network applications. The paper views network learning as an optimization problem, reviews two alternative approaches to network learning, and provides insights into current best practice to optimize complexity so to perform well on generalization tasks.

• Item Type: MPRA Paper
• Language: English
• Keywords: n.a.
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C45 - Neural Networks and Related Topics

References

Bishop, M. (1995). Neural Networks for Pattern Recognition. Oxford University Press, Oxford.

Chen, A.M., Lu, H.M. and Hecht-Nielsen, R. (1993). On the geometry of feedforward neural-network error surfaces. Neural Computation, Vol. 5, 910-926.

Cichocki, A. and Unbehauen, R. (1993). Neural Networks for Optimization and Signal Processing. John Wiley, Chichester.

Coetzee, F.M. and Stonick, V.L. (1995). Topology and geometry of single hidden layer network. Neural Computation, Vol. 7, 672-705.

Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control Signals and Systems, Vol. 2, 303-314.

Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. Society for Industrial and Applied Mathematics, Philadelphia.

Efron, B. and Tibshirani, R. (1983). An Introduction to the Bootstrap. Chapman and Hall, New York.

Fischer, M.M. (2002). Learning in neural spatial interaction models: A statistical perspective. Journal of Geographical Systems, Vol. 4(3), 287-299.

Fischer, M.M. (2001). Spatial analysis in geography, in: Smelser, N.J. and Baltes, P.B. (Eds.), International Encyclopedia of the Social and Behavioral Sciences, Vol. 22. Elsevier, Oxford, pp. 14752-14758.

Fischer, M.M. (2000). Methodological challenges in neural spatial interaction modelling: The issue of model selection, in: Reggiani, A. (Ed.), Spatial Economic Science: New Frontiers in Theory and Methodology. Springer, Berlin, Heidelberg and New York, pp. 89-101.

Fischer, M.M. and Getis A. (Eds.) (1997). Recent Developments in Spatial Analysis. Spatial Statistics, Behavioural Modelling and Computational Intelligence. Springer, Berlin, Heidelberg and New York.

Fischer, M.M. and Gopal, S. (1994). Artificial neural networks: A new approach to modelling interregional telecommunication flows. Journal of Regional Science, Vol. 34(4), 503-527.

Fischer, M.M. and Leung, Y. (Eds.) (2001). GeoComputational Modelling: Techniques and Applications. Springer, Berlin, Heidelberg and New York.

Fischer, M.M. and Leung, Y. (1998). A genetic-algorithm based evolutionary computational neural network for modelling spatial interaction data. The Annals of Regional Science, Vol. 32(3), 437-458.

Fischer, M.M. and Reismann M. (2002a). Evaluating neural spatial interaction modelling by bootstrapping. Networks and Spatial Economics, Vol. 2(3), 255-268.

Fischer, M.M. and Reismann M. (2002b). A methodology for neural spatial interaction modeling. Geographical Analysis, Vol. 34(2), 207-228.

Fischer, M.M., Hlavackova-Schindler K. and Reismann M. (1999). A global search procedure for parameter estimation in neural spatial interaction modelling. Papers in Regional Science, Vol. 78, 119-134.

Funahashi, K. (1989). On the approximate realization of continuous mappings by neural networks. Neural Networks, Vol. 2, 183-192.

Gallant, A.R. and White, H. (1988). A Unified Theory of Estimation and Inference for Nonlinear Dynamic Models. Basil Blackwell, Oxford.

Hassoun, M.H. (1995). Fundamentals of Artificial Neural Networks. MIT Press, Cambridge [MA] and London, England.

Hecht-Nielsen, R. (1989). Theory of the back-propagation neural network. Proceedings of the International Joint Conference on Neural Networks, Washington, D.C. IEEE, New York, pp. 593-606.

Hornik, K., Stinchcombe, M. and White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, Vol 2., 359-368.

Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flannery, B.P. (1992). Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, Cambridge.

Ripley, B.D. (1996). Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge.

Rumelhart, D.E., Hinton, G.E. and Williams, R.J. (1986). Learning internal representations by error propagation, in: Rumelhart, D.E., McClelland, J.L. and the PDP Research Group (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of cognition. MIT Press, Cambridge [MA], pp. 318-362.

Sussmann, H.J. (1992). Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks, Vol. 5, 589-593.

White, H. (1990). Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings. Neural Networks, Vol. 3, 535-550.

White, H. (1989a). Learning in artificial neural networks: A statistical perspective. Neural Computation, Vol. 1, 425-464.

White, H. (1989b). Some asymptotic results for learning in single hidden layer feedforward network models. Journal of the American Statistical Association, Vol. 84, 1008-1013.

White, H. and Racine, J. (2001). Statistical inference, the bootstrap, and neural-network modeling with application to foreign exchange rates. IEEE Transactions on Neural Networks, 12(4), 657-673.

White, H. and Wooldridge, J. (1991). Some results for sieve estimation with dependent observations, in: Barnett, W., Powell, J. and Tauchen, G. (Eds.), Nonparametric and Semiparametric Methods in Econometrics and Statistics. Cambridge University Press, New York.

Zapranis, A. and Refenes, A.-P. (1999). Principles of Neural Identification, Selection and Adequacy. With Applications to Financial Econometrics. Springer, London, Berlin and Heidelberg.