Geração automática de textos alternativos para acessibilidade web com grandes modelos de linguagem multimodais

Abstract

Since the creation of the Web Content Accessibility Guidelines, the Web has become increasingly accessible to people with disabilities. However, related studies report that developers often lack awareness of these specifications, and many applications still present accessibility barriers. In this context, this dissertation proposes an approach that integrates visual data obtained through the Google Cloud Vision API with contextual information extracted from web pages, employing the multimodal models Gemini-1.5-Pro, Gemini-2.5-flash-preview-05-20, and GPT-o4-mini to generate descriptions capable of replacing alternative texts manually created by website authors.The main objective is to evaluate whether the descriptions produced by Multimodal Large Language Models (MLLM) demonstrate sufficient quality and relevance to be adopted as alternative texts for images on the Web. The methodology comprised two complementary studies. The first involved an evaluation of the quality of the generated descriptions, in which six graduate students in Computer Science provided responses after analyzing ten images extracted from Brazilian news portals. For each image, three descriptions were generated by the MLLM, each with a distinct level of contextual incorporation, and these descriptions were blindly evaluated for their relevance compared to the alternative texts provided by the page authors. The second study consisted of a quantitative evaluation involving 12 descriptions generated by MLLMs, considering different levels of contextual information, variations in text length, and distinct language models. For this evaluation, semantic and visual similarity metrics (BERTScore and CLIPScore) were applied to an additional set of 73 images. The results suggest that the descriptions generated by MLLM achieve a quality level comparable to that of human-produced texts. However, the inclusion of additional contextual information did not lead to significant improvements in the evaluations. These findings highlight the potential of multimodal models as tools for promoting digital accessibility, reinforcing the role of artificial intelligence as an ally in building a more inclusive Web, particularly in scenarios where the manual production of image descriptions on a large scale is unfeasible.