Comparando BERT, RoBERTa e DistilBERT para análise de sentimentos em texto

Abstract

The desire to improve communication between man and machine has provided several advances in the area of Natural Language Processing (PLN), one of the great sub-areas of Artificial Intelligence (AI). Over the years, the creation and improvement of PLN techniques have changed the concept of best practices for building models of Artificial Neural Networks (ANNs) aimed at this area, giving rise to the possibility of using RNAS for PLN tasks such as the analysis of feelings. In 2018, a new architectural model called Bidirectional Encoder Representationfrom Transformer (BERT) reached a new state of the art, obtaining excellent results in different types of tasks. Some new models based on BERT have emerged with a proposal to obtain better results such as RoBERTa, and others to lower the training cost and try to maintain performance, such as DistilBERT. In this work, the BERT, RoBERTa and DistilBERT models are compared with their refined parameters for the binary sentiment analysis task. The results obtained in this work indicate the superiority of the BERT and RoBERTa models in terms of accuracy, but the DistilBERT requires about half the time needed to complete the fine-tuning. From these results, the choice of the best model for a binary sentiment analysis task will depend on the scenario to be applied.