Estratégia autônoma baseada em aprendizado por reforço para a categoria IEEE very small size soccer

Abstract

This work aims to develop an autonomous strategy for decision-making and motion control of differential robots in the 3 versus 3 robots IEEE Very Small Size Soccer category. In this strategy, models obtained using Deep Reinforcement Learning algorithms were used to define the robots’ behaviors according to match situations. Two types of training were designed to obtain these models. The first involves learning how to act based on a specific role (attacker, defender, or goalkeeper) that the robot can assume during a match, using the PPO (Proximal Policy Optimization) algorithm. The second focuses on learning which role, including an additional support role (which was created without Reinforcement Learning for robots without ball possession and whose behavior is to position themselves in locations that assist in attack and defense according to the positions of the robots and the ball), each robot should take based on the match situation, employing the Deep Q-learning algorithm. For positioning the robots in situations like free kicks and penalty kicks, ensuring they reach their positions without colliding with static and dynamic obstacles, Univector Field Navigation was used for path planning, and the PID (Proportional–Integral–Derivative) controller was applied for motion control. The simulation results demonstrate that the developed strategy can operate consistently in all match situations within this category, whether defending, attacking, or positioning in situations like free kicks and penalty, without requiring human intervention.