Deep learning approaches for soft biometrics classification in videos

Abstract

The number of surveillance cameras installed in public places has grown enormously during the past years due to the necessity to increase public security, allowing to obtain a large amount of images and videos in real time without much effort. Different types of problems can be solved by processing the data obtained by security cameras, such as the identification of individuals. Soft biometrics attributes can be useful to perform this task, since they provide information that can be used to differentiate one person from another without requiring their direct cooperation. However, this demands an exhaustive process of analysis to be carried by one or more human observers. Depending on the quantity of cameras, this could even become an impossible task for humans. Hence, computer vision methods could be a valid alternative to perform soft biometric classification in images or videos. Within this score, Deep Learning (DL) methods have risen recently, achieving state-of-the art performances for several computer vision tasks such as object recognition, object detection and image segmentation. This is possible due to their capability to learn both, features and classifier, at once, in order to solve a particular problem. Following this line, this work aims at empirically studying the suitability of DL methods for classifying soft biometrics in images or videos. We present three contributions regarding this subject in this dissertation. First, we perform a study on the effect of data augmentation on the performance of convolutional neural networks for soft biometrics classification. The second contribution is related to transferring information from one soft biometric dataset into another to perform classification. This process is achieved by training a model with data from a dataset in order to test it on data from another one. Finally, we evaluate the use of DL models to represent or learn temporal dependencies, so as to perform soft biometrics classification in videos. For this task, we propose a novel approach based on the use of bidirectional long short term memory networks. Results for the experiments regarding data augmentation show that large augmentation sizes do not induce overfitting and that balancing a dataset before performing on-line data augmentation leads to the necessity of smaller augmentation sizes in order to start improving the performance of the networks. As for transfer learning, results show that there could be a correlation between the complexity and the similarity of the datasets that are used for training and testing a model. Thus, if this technique is applied, the training set should preferably be very similar to the test data and should have a higher complexity. Although this is not definitive, since there could be exceptions depending on the soft biometric attribute to classify. Regarding video classification, in general, our approaches based on a recurrent network and a DL model that represents temporal dependencies through a low-pass filter yielded better results, in terms of overall accuracy and classification balance, than the baseline, based on classifying a video using only one of its frames.