Combinação de características extraídas de imagens e metadados para classificação de lesões de pele

Abstract

Cancer is one of the most lethal diseases in existence, and the type that affects the skin has the highest incidence in Brazil; therefore, ensuring its detection is crucial to enable treatment. Research indicates that early-stage treatment increases survival chances by up to 99%, unlike advanced cases, in which the probability drops to 35%. To ensure this detection, it becomes important to reduce the bias of medical assessment based on the ABCDE rule by applying a generalist model that utilizes these rules to obtain greater reliability. Thus, it is possible to advance to subsequent stages, such as biopsy and histopathological examination, procedures that present a higher financial cost. In this context, this work used deep learning, including transformers and CNN, to maximize results. Combined with these tools, metadata representing information similar to that of the ABCDE rule were used, applying different fusion strategies, such as early and late. The experiments demonstrated that early fusion between metadata and features obtained by deep extractors achieved the best performance, with a Macro F1-Score of 83%. However, in some scenarios, a discrepancy was observed between accuracy and the Macro F1-Score, suggesting that the models were not effectively learning the target classes.