Reconhecimento de expressões faciais com técnicas de visão computacional

Abstract

This work investigates strategies for facial expression recognition using convolutional neural networks (CNN) and vision transformers (ViT), evaluating different types of input: full facial images, facial landmarks, and mixed combinations. Experiments were conducted on the FER 2013, FER+, and RAF-DB datasets, applying approaches based on original images, landmarks extracted via MediaPipe and dlib, and configurations that combine these sources. The results showed that the ViT model, using only full-image input, achieved the best overall performance, with F1-scores reaching up to 0.8179 on RAF-DB. In contrast, CNN proved to be more efficient in inference time, making it suitable for real-time applications. Landmark-based approaches showed lower performance but stood out for their efficiency and privacy preservation, especially when using MediaPipe. The mixed approach, combining images and landmarks, yielded gains only in specific CNN configurations, without consistently outperforming image-only methods. The image loss analysis revealed that MediaPipe provided greater robustness in landmark extraction compared to dlib. It is concluded that while full-image input with ViT is the most accurate, geometric and mixed approaches offer promising alternatives in contexts with privacy or processing constraints, with the ideal choice depending on the application scenario.