Comparação entre os algoritmos de aprendizado por reforço Q-learning e Sarsa no jogo educacional Xô Mosquito

Abstract

The application of machine learning algorithms in games can enable an environment that is more adaptable to the player's characteristics, whether by understanding moves or gradually increasing their level of difficulty. Reinforcement learning methods present behavior similar to human learning, using trial and error to obtain knowledge through rewards. This work presents the comparison between the Q-learning and Sarsa algorithms, two reinforcement learning algorithms, implemented in three different scenarios. Thus, the work seeks to define the implementation that solves the educational game Xô Mosquito in the shortest time. The scenarios use different parameterizations for the initial values of the algorithms, based on their proximity to the game's objectives. The first uses values equal to zero, the second has values based on block set and proximity logic, and finally the third applies values based on the proximity of individual blocks. The implementations were applied to the educational game Xô Mosquito, a game with few movement limitations and with three objectives. This game was developed in the Lesic extension project (Software engineering and computational intelligence laboratory). The comparison is made in a simplified environment of the base game and the main data analyzed is the total execution time of the implementations. With the results it is possible to see that within the proposed scenario the Sarsa algorithm with its third scenario solves the game in the shortest time. One can also understand the impact that different initial values have on the application of reinforcement learning algorithms.