Sistema de interface homem-máquina para reconhecimento de padrões de gestos em Libras

Abstract

This work proposes the development of an innovative system for capturing and classifying gestures in Brazilian Sign Language (Libras) using wearable sensing technologies and Deep Learning. The justification for this research lies in the need to provide effective alternatives for communication in Libras, especially among individuals with and without hearing impairments. The objectives range from implementing instrumented gloves with a variety of sensors, including flexible and inertial sensors (such as accelerometers and gyroscopes), as well as the Armband Myo, to developing the system for data capture during gesture execution. Additionally, the application of Deep Learning models for gesture classification is proposed. The results obtained reveal a promising accuracy in gesture classification, reaching 89.77% with the gMLP model and 92.21% with InceptionTime. This approach stands out for its innovation in comparing and integrating different types of sensors, such as surface electromyography (EMG), flexible sensors, and inertial sensors of accelerometer and gyroscope, and for the demonstrated effectiveness in applying Deep Learning techniques in gesture classification in Libras.