Margin-based learning to enhance open-set semantic segmentation

Abstract

Semantic segmentation has received wide attention as a feasible solution for effectively interpreting information in digital images. Solutions are typically built with a static closed-set perception, where all labels are known a priori. However, in real-world applications, one has to handle objects from unknown classes. Open-set semantic segmentation (OSSS) is an approach that incorporates open-set perception into semantic segmentation, allowing the recognition of unknown classes of objects. While various studies have explored OSSS, its performance remains limited due to low feature representation quality and challenges like imbalanced data, uncertain boundary regions, and semantic shifts. These challenges result in closed-set semantic segmentation models that build embedding spaces with overlapping decision boundaries and insufficient open space to accommodate unknown classes, thereby negatively impacting open-set recognition performance. This thesis presents two novel loss functions based on margin-based learning and hard sample attention to enhance open-set recognition. Two novel open-set classifiers are proposed: one that employs Principal Components and the other that uses Prototypical Distance. Both are designed to benefit from the features generated by the proposed loss functions. Experiments were conducted on six datasets, of which three were originally proposed in this work, and the results demonstrate that the proposed methods outperform state-of-the-art learning strategies in OSSS. These findings underscore the significance of such learning approaches, paving the way for their application in real-world systems requiring robust open-set recognition.