Análise de sentimentos em comentários de aplicativos comerciais

Abstract

Online platform reviews frequently exhibit inconsistencies between textual content and numerical ratings assigned by users, compromising the reliability of the evaluation system. This work developed a sentiment analysis tool to automatically classify reviews into five sentiment classes. A total of 3,000 reviews were collected from five applications in the Shopping category on the Google Play Store, which were subjected to automatic re-annotation processes using the GPT-4 and Gemini 2.5 Pro language models, in addition to filtering with the LeIA sentiment analysis model, aiming to mitigate existing discrepancies between textual reviews and their numerical ratings. Four Transformer architectures were evaluated through fine-tuning: BERTimbau Base, BERTimbau Large, mBERT Base, and XLM-RoBERTa Base, varying hyperparameters such as batch size and learning rate, in addition to applying data cleaning and standardization techniques together with class weighting. The results demonstrated that re-annotation with Gemini 2.5 Pro was the most impactful factor, providing an average gain of 47.2% in macro F1-score, while re-annotation with GPT-4 resulted in a gain of 32.3%. Filtering with LeIA proved effective only in datasets with greater inconsistencies. The BERTimbau Base model trained with the dataset re-annotated by Gemini 2.5 Pro was selected to compose the final tool, achieving a macro F1-score of 0.79. The tool was made available through a RESTful API developed in FastAPI and a multiplatform front-end application in Flutter, deployed with Docker and Docker Compose. The experiments showed that data annotation quality has a direct and substantial impact on model performance, being more relevant than hyperparameter adjustments or filtering techniques.