Performance of Differential Evolution and Particle Swarm Methods on Some Relatively Harder Multi-modal Benchmark Functions

Mishra, SK (2006): Performance of Differential Evolution and Particle Swarm Methods on Some Relatively Harder Multi-modal Benchmark Functions.

Abstract

This paper aims at comparing the performance of the Differential Evolution (DE) and the Repulsive Particle Swarm (RPS) methods of global optimization. To this end, some relatively difficult test functions have been chosen. Among these test functions, some are new while others are well known in the literature. We use DE method with the exponential crossover scheme as well as with no crossover (only probabilistic replacement). Our findings suggest that DE (with the exponential crossover scheme) mostly fails to find the optimum in case of the functions under study. Of course, it succeeds in case of some functions (perm#2, zero-sum) for very small dimension, but begins to falter as soon as the dimension is increased. In case of DCS function, it works well up to dimension = 5. When we use no crossover (only probabilistic replacement) we obtain better results in case of several of the functions under study. In case of Perm#1, Perm#2, Zero-sum, Kowalik, Hougen and Power-sum functions, a remarkable advantage is there.

Whether crossover or no crossover, DE falters when the optimand function has some element of randomness. This is indicated by the functions: Yao-Liu#7, Fletcher-Powell, and “New function#2”. DE has no problems in optimizing the “New function #1”. But the “New function #2” proves to be a hard nut. However, RPS performs much better for such stochastic functions. When the Fletcher-Powell function is optimized with non-stochastic c vector, DE works fine. But as soon as c is stochastic, it becomes unstable. Thus, it may be observed that an introduction of stochasticity into the decision variables (or simply added to the function as in Yao-Liu#7) interferes with the fundamentals of DE, which works through attainment of better and better (in the sense of Pareto improvement) population at each successive iteration.

The paper concludes: (1) for different types of problems, different schemes of crossover (including none) may be suitable or unsuitable, (2) Stochasticity entering into the optimand function may make DE unstable, but RPS may function well.

• Item Type: MPRA Paper
• Language: English
• Keywords: Differential Evolution; Repulsive Particle Swarm; Global optimization; non-convex functions; Fortran; computer program; benchmark; test; Stochastic functions; Fletcher-Powell; Kowalik; Hougen; Power-sum; Perm; Zero-sum; New functions; Bukin function
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

· Bauer, J.M.: “Harnessing the Swarm: Communication Policy in an Era of Ubiquitous Networks and Disruptive Technologies”, Communications and Strategies, 45, 2002. · Box, M.J.: “A new method of constrained optimization and a comparison with other methods”. Comp. J. 8, pp. 42-52, 1965. · Bukin, A. D.: New Minimization Strategy For Non-Smooth Functions, Budker Institute of Nuclear Physics preprint BUDKER-INP-1997-79, Novosibirsk 1997. · Cerny, V.: "Thermodynamical Approach to the Traveling Salesman Problem: An Efficient Simulation Algorithm", J. Opt. Theory Appl., 45, 1, 41-51, 1985. · Eberhart R.C. and Kennedy J.: “A New Optimizer using Particle Swarm Theory”, Proceedings Sixth Symposium on Micro Machine and Human Science, pp. 39–43. IEEE Service Center, Piscataway, NJ, 1995. · Fleischer, M.: “Foundations of Swarm Intelligence: From Principles to Practice”, Swarming Network Enabled C4ISR, arXiv:nlin.AO/0502003 v1 2 Feb 2005. · G.E.P. Box, “Evolutionary operation: A method for increasing industrial productivity”, Applied Statistics, 6 , pp. 81-101, 1957. · Glover F.," Future paths for Integer Programming and Links to Artificial Intelligence", Computers and Operations Research, 5:533-549, 1986. · Hayek, F.A.: The Road to Serfdom, Univ. of Chicago Press, Chicago, 1944. · Holland, J.: Adaptation in Natural and Artificial Systems, Univ. of Michigan Press, Ann Arbor, 1975. · Karush, W. Minima of Functions of Several Variables with Inequalities as Side Constraints. M.Sc. Dissertation. Dept. of Mathematics, Univ. of Chicago, Chicago, Illinois, 1939. · Kirkpatrick, S., Gelatt, C.D. Jr., and Vecchi, M.P.: "Optimization by Simulated Annealing", Science, 220, 4598, 671-680, 1983. · Kuhn, H.W. and Tucker, A.W.: “Nonlinear Programming”, in Neymann, J. (ed) Proceedings of Second Berkeley Symposium on Mathematical Statistics and Probability, Univ. of California Press, Berkrley, Calif. pp. 481-492, 1951. · Metropolis, N. The Beginning of the Monte Carlo Method. Los Alamos Science, No. 15, Special Issue, pp. 125-130, 1987. · Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, E.: "Equation of State Calculations by Fast Computing Machines", J. Chem. Phys.,21, 6, 1087-1092, 1953. · Mishra, S.K.: “Some Experiments on Fitting of Gielis Curves by Simulated Annealing and Particle Swarm Methods of Global Optimization”, Social Science Research Network (SSRN): http://ssrn.com/abstract=913667, Working Papers Series, 2006 (a). · Mishra, S.K.: “Least Squares Fitting of Chacón-Gielis Curves by the Particle Swarm Method of Optimization”, Social Science Research Network (SSRN), Working Papers Series, http://ssrn.com/abstract=917762 , 2006 (b). · Mishra, S.K.: “Performance of Repulsive Particle Swarm Method in Global Optimization of Some Important Test Functions: A Fortran Program” , Social Science Research Network (SSRN), Working Papers Series, http://ssrn.com/abstract=924339 , 2006 (c). · Mishra, S.K.: “Some New Test Functions for Global Optimization and Performance of Repulsive Particle Swarm Method”, Social Science Research Network (SSRN) Working Papers Series, http://ssrn.com/abstract=927134, 2006 (d). · Mishra, S.K.: “Repulsive Particle Swarm Method on Some Difficult Test Problems of Global Optimization” ,SSRN: http://ssrn.com/abstract=928538 , 2006 (e). · Nagendra, S.: Catalogue of Test Problems for Optimization Algorithm Verification, Technical Report 97-CRD-110, General Electric Company, 1997. · Nelder, J.A. and Mead, R.: “A Simplex method for function minimization” Computer Journal, 7: pp. 308-313, 1964. · Parsopoulos, K.E. and Vrahatis, M.N., “Recent Approaches to Global Optimization Problems Through Particle Swarm Optimization”, Natural Computing, 1 (2-3), pp. 235- 306, 2002. · Prigogine, I. and Strengers, I.: Order Out of Chaos: Man’s New Dialogue with Nature, Bantam Books, Inc. NY, 1984. · Silagadge, Z.K.: “Finding Two-Dimensional Peaks”, Working Paper, Budkar Insttute of Nuclear Physics, Novosibirsk, Russia, arXive:physics/0402085 V3 11 Mar 2004. · Simon, H.A.: Models of Bounded Rationality, Cambridge Univ. Press, Cambridge, MA, 1982. · Smith, A.: The Theory of the Moral Sentiments, The Adam Smith Institute (2001 e-version), 1759. · Sumper, D.J.T.: “The Principles of Collective Animal Behaviour”, Phil. Trans. R. Soc. B. 361, pp. 5-22, 2006. · Törn, A.A and Viitanen, S.: “Topographical Global Optimization using Presampled Points”, J. of Global Optimization, 5, pp. 267-276, 1994. · Törn, A.A.: “A search Clustering Approach to Global Optimization” , in Dixon, LCW and Szegö, G.P. (Eds) Towards Global Optimization – 2, North Holland, Amsterdam, 1978. · Tsallis, C. and Stariolo, D.A.: “Generalized Simulated Annealing”, ArXive condmat/9501047 v1 12 Jan, 1995. · Valentine, R.H.: Travel Time Curves in Oblique Structures, Ph.D. Dissertation, MIT, Mass, 1937. · Veblen, T.B.: "Why is Economics Not an Evolutionary Science" The Quarterly Journal of Economics, 12, 1898. · Veblen, T.B.: The Theory of the Leisure Class, The New American library, NY. (Reprint, 1953), 1899. · Vesterstrøm, J. and Thomsen, R.: “A comparative Study of Differential Evolution, Particle Swarm Optimization, and Evolutionary Algorithms on Numerical Benchmark Problems”, Congress on Evolutionary Computation, 2004. CEC2004, 2, pp. 1980-1987, 2004. · Whitley, D., Mathias, K., Rana, S. and Dzubera, J.: “Evaluating Evolutionary Algorithms”, Artificial Intelligence, 85, pp. 245-276, 1996. · Yao, X. and Liu, Y.: “Fast Evolutionary Programming”, in Fogel, LJ, Angeline, PJ and Bäck, T (eds) Proc. 5th Annual Conf. on Evolutionary programming, pp. 451-460, MIT Press, Mass, 1996.