Análise de modelos de conversão de fala sussurrada para normal emlíngua portuguesa

Abstract

Whispered speech is a common and secondary mechanism of communication. However, individuals with aphonia, such as laryngectomized patients, rely on whispered speech as their primary means of expression. Due to the significant differences between whispered and normal speech, speech recognition and conversion systems face major challenges in achieving accurate transformation between them. This doctoral thesis addresses, discusses, and proposes solutions across three main fronts: i) An evaluation of four voice conversion models without prior training on whispered speech, assessed through the Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) perceptual speech test for whisper-to-normal conversion. The k-nearest neighbors voice conversion model (KNN-VC) outperformed the others in both objective and subjective metrics; ii) The construction of a Brazilian Portuguese whispered speech database, duly approved by the Research Ethics Committee, aiming to address the scarcity of data in this linguistic modality and expand opportunities for scientific experimentation; iii) Enhancements to the KNN-VC model from different perspectives, including: experiments with alternative feature extraction models; transfer learning using the low-rank adaptation (LoRA) algorithm; mapping of parallel whispered and normal speech data with KNN (pKNN-VC), and with MLP (MLP-VC). Finally, the thesis highlights the contributions derived from each study and discusses opportunities and directions for future research.