Investigation of the Efficacy of Niche Techniques and Environmental Differentiation applied to Evolutionary Robotics Algorithms
DOI:
https://doi.org/10.5902/2448190485256Keywords:
Robótica Evolutiva, Niche Evolution, Estratégias de EvoluçãoAbstract
The Evolutionary Strategies (ES) algorithm has proven to be an efficient optimization technique over the decades. Recently, an adaptation of the method, proposed by researchers from the company OpenAI, demonstrated the advantages of using ES techniques in parallel as an important alternative to the also relevant Reinforcement Learning method. Techniques that use population optimization approaches, such as evolutionary strategies, benefit from the diversity of candidate solutions in the evolutionary process. For this reason, mechanisms that preserve diversity, such as the creation of islands and niches during the evolutionary process, have been proposed and investigated in other evolutionary algorithms. This work aims to analyze how the addition of niche techniques, which include environmental differentiation between subpopulations, can be relevant to Evolutionary Robotics algorithms using the version of the ES algorithm recently proposed by OpenAI. Using the well-known double-pole balancing problem as a test task, we compared the effectiveness of solutions generated with and without the niche mechanism in the OpenAI-ES and Stochastic Steady State (SSS) algorithms. The results obtained demonstrated performance increases of approximately 8.6\% and 53.5\% for OpenAI-ES and SSS, respectively, when the niche mechanism is used.
Downloads
References
(2019). The impact of environmental history on evolved robot properties, volume ALIFE
: The 2019 Conference on Artificial Life of ALIFE 2021: The 2021 Conference
on Artificial Life.
Bianchini, A. H. (2023). A stripped-down version of evorobotpy2 with openai-
es-ne and some experiments results. https://github.com/alvaporta/
evorobotpy2.
Bianchini, A. H., Machado, B. S., and Carvalho, J. T. (2023). The effectiveness of niching
on openai-evolution strategies in the evolution of robotic behavior. In Proceedings
of the Companion Conference on Genetic and Evolutionary Computation, GECCO
’23 Companion, page 2354–2357, New York, NY, USA. Association for Computing
Machinery.
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and
Zaremba, W. (2016). Openai gym.
Carvalho, J. T. and Nolfi, S. (2017). Favoring the evolution of adaptive robots through
environmental differentiation. In 2017 IEEE Symposium Series on Computational In-
telligence (SSCI), pages 1–7.
Chowdhury, A., Karmakar, G., Kamruzzaman, J., Jolfaei, A., and Das, R. (2020). Attacks
on self-driving cars and their countermeasures: A survey. IEEE Access, 8:207308–
Ekart, A. and Nemeth, S. Z. (2002). Maintaining the diversity of genetic programs. In
European Conference on Genetic Programming, pages 162–171. Springer.
Fogel, D. B. (1997). The advantages of evolutionary computation. In Bcec, pages 1–11.
Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization. International
Conference on Learning Representations.
Linder, M. H. and Nye, B. (2010). Fitness, environment and input: Evolved robotic
shepherding. Dept. Comput. Sci., Swarthmore College, Swarthmore, PA, USA, Tech.
Rep.
Lopez-Ibanez, M., Dubois-Lacoste, J., Perez Caceres, L., Stutzle, T., and Birattari, M.
(2016). The irace package: Iterated racing for automatic algorithm configuration. Ope-
rations Research Perspectives, 3:43–58.
Milano, N., Carvalho, J. T., and Nolfi, S. (2017). Environmental variations promotes
adaptation in artificial evolution. In 2017 IEEE Symposium Series on Computational
Intelligence (SSCI), pages 1–7.
Nolfi, S. (2020). A tool for training robots through evolutionary and reinforcement lear-
ning methods. https://github.com/snolfi/evorobotpy2.
Nolfi, S. (2021). Behavioral and cognitive robotics: An adaptive perspective. Roma,
Italy: Institute of Cognitive Sciences and Technologies, National Research Council
(CNR-ISTC).
Pagliuca, P., Milano, N., and Nolfi, S. (2018). Maximizing adaptive power in neuroevo-
lution. PloS one, 13(7):e0198788.
Pagliuca, P., Milano, N., and Nolfi, S. (2020). Efficacy of modern neuro-evolutionary
strategies for continuous control optimization. Frontiers in Robotics and AI, 7.
Rechenberg, I. (1973). Evolutionsstrategie. Optimierung technischer Systeme nach Prin-
zipien derbiologischen Evolution.
Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution strategies as
a scalable alternative to reinforcement learning. arXiv.
Sareni, B. and Krahenbuhl, L. (1998). Fitness sharing and niching methods revisited.
IEEE Transactions on Evolutionary Computation, 2(3):97–106.
Whitley, D., Rana, S., and Heckendorn, R. (1998). The island model genetic algorithm:
On separability, population size and convergence. Journal of Computing and Informa-
tion Technology, 7
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Revista ComInG - Communications and Innovations Gazette
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Os manuscritos aceitos e publicados são de propriedade da revista ComInG.
Os originais deverão ser acompanhados de documentos de transferência de direitos autorais contendo assinatura dos autores.
A carta de direitos autorais deve ser enviada para o e-mail coming@inf.ufsm.br
É vedada a submissão integral ou parcial do manuscrito a qualquer outro periódico. A responsabilidade do conteúdo dos artigos é exclusiva dos autores.
É vedada a tradução para outro idioma sem a autorização escrita do Editor ouvida a Comissão Editorial.
ENGLISH
Manuscripts accepted and published are the property of the journal ComInG.
The originals must be accompanied by documentation of copyright transfer containing the signature of the authors.
You may not submit full or partial manuscript to another journal. The responsibility of the article's content is exclusive of the authors.
You may not translating into another language without the written permission of the Editor after consultation with the Editorial Board.
