Desenvolvimento de aplicação móvel para avaliação automatizada de exercícios de motricidade orofacial com visão computacional e aprendizado de máquina

Abstract

In cases where speech disorders resulting from motor difficulties pose significant barriers to verbal communication, Speech therapy provides support to help individuals improve their communication skills. However, patients often face difficulties or even lack motivation when attempting to practice therapeutic exercises outside the clinical environment, without the direct and continuous assistance of a speech therapist. Given this scenario, this study proposed a tool to support therapeutic follow-up by allowing patients to record their exercise executions and receive an automated evaluation on a three-level scale based on the quality of execution. To achieve this goal, the MediaPipe FaceMesh Convolutional Neural Network model was used to extract key facial landmarks, which were preprocessed and used as input for a Long Short-Term Memory (LSTM) neural network. For model development, hyperparameter optimization techniques were applied, and evaluation was performed using K-Fold Cross-Validation. To verify the replicability of this approach, three models were generated for the movements of lip puckering, mouth opening, and lip stretching. The results confirmed that movement evaluation could be performed. To enable the use of the generated models, a mobile application was developed with a focus on usability, aiming to provide an intuitive user experience while allowing patients to track their consistency in performing the activities. The study successfully achieved its objective of developing a solution that can complement speech therapy while also suggesting future research possibilities within this context.