Avaliação de técnicas de reconhecimento de gestos da língua de sinais utilizando sensores vestíveis

Abstract

Gesture recognition systems, especially those based on instrumented gloves, have emerged as promising tools to enhance accessibility for deaf individuals, both in society and in digital environments. Examples of applications include digital games and translation systems in virtual reality contexts. The main challenges in this research area involve the generalization capability of classification models when faced with new users, as well as the limitation of dataset size, which directly affect system performance. To investigate these factors, this thesis presents two complementary case studies. In the first, the influence of dataset size on classification performance was analyzed using a traditional classification approach involving a feature-engineering process. For this purpose, a public dataset comprising 50 American Sign Language (ASL) gestures, collected with a pair of gloves instrumented with triboelectric sensors, was employed. In the second case study, in addition to analyzing the influence of dataset size, the generalization capability of the models was examined through a comparison between traditional classification approaches and those based on Recurrent Neural Network (RNN)s. Additionally, an architecture incorporating a Global Max Pooling layer was proposed to improve RNN performance. For the experiments, a new dataset was created comprising 26 letters of the Brazilian Sign Language (Libras), captured using a glove equipped with flexible, inertial, and contact sensors. The results of the first case study indicate that, despite the large number of gesture classes, the traditional classification approach is less affected by reductions in dataset size. In the second case study, the proposed RNN architecture outperformed the traditional approach even under conditions with limited data and unseen users. The proposed architecture was further validated through comparisons with other deep learning models and evaluated on public datasets, achieving results that surpass the state of the art. In this thesis, the term ‘gestures” is adopted as a synonym for ‘patterns” and for ‘Libras signs”.