Eficiência energética no posicionamento de estações rádio-base móveis baseadas em VANTS aplicando aprendizagem por reforço

Abstract

To meet users demands, 5G cellular networks require high operational efficiency, making energy efficiency one of the primary challenges and objectives of modern communication systems. In addition, the increase in the number of devices and user requirements in these networks may turn analytical solutions applied in the context of energy efficiency and optimization of communication parameters not feasible, given the complexity associated to these solutions. To overcome this problem, more robust and flexible solutions that exploit the data generated by the network and make real-time decisions are required by these future networks. In this context, the use of Machine Learning techniques, capable of analyzing a big amount of data and learning from it, is a possible solution and can be applied, for example, in different self-organizing network contexts. The aim of this work is to analyze a positioning optimization application of UAVs, Unmanned Aerial Vehicles, in a scenario of temporary events based on a Q-learning algorithm, a reinforcement learning technique in which the agents learn to act to maximize a given reward. Two different rewards are proposed and compared, with the first aiming only to minimize the number of users in outage and the second provided to maximize energy efficiency by considering the amount of energy used for UAVs movements. The Q-learning solution is responsible for determining the best three-dimensional positioning and optimal transmit power given some possible power levels. The results show that the optimization of the transmission power reduced the percentage of users in outage, since interference between UAVs was minimized by allocating adequate power. Furthermore, it is shown that the introduction of a discount factor in the algorithm reward for UAV movements has been able to considerably increase the energy efficiency of the network, with a performance penalty over the user coverage. Finally, for the two solutions considered and in accordance with their respective objectives, the Q-learning technique, based on its ability to learn from iteration with an environment and from previous experiences, proved to be able to solve the problem of self-organization of the UAV-based network in an efficient manner, presenting itself as a promising solution for future cellular networks.