Controle de um pêndulo de Furuta com aprendizado por reforço

Abstract

This work proposed the control of a Furuta pendulum using reinforcement learning, addressing challenges such as underactuation, non-linearities, and instabilities. The classical control approach for this system involves positioning the pendulum close to the equilibrium region, a technique known as swing-up, and controlling it at the equilibrium point using a linear controller. However, this approach requires system linearizations, which leads to sub-optimal performance. The main goal of the study was to explore the application of reinforcement learning to handle the complexities of the system, such as its non-linear and unstable characteristics. For this, the Q-Learning algorithm was employed, conducting simulations to compare the performance of the classical controllers and the reinforcement learning-based controller, with metrics focusing on stability, accuracy, and energy efficiency. The results indicate that the Q-Learning-based model performed worse than the classical controllers in terms of tracking error and energy consumption for control, also requiring non-trivial adjustments of hyperparameters to optimize the system’s performance, which significantly increased the training time due to the need to explore and adjust different configurations to achieve acceptable performance. It is concluded that, although classical methods are more suitable for scenarios that require fast and reliable solutions, reinforcement learning represents a promising alternative for complex and uncertain systems, where explicit modeling of the environment is difficult or unfeasible.