A persistent incremental learning approach for object classification of unseen categories using convolutional neural networks on mobile robots

Kurzfassung

Neural Networks and especially Convolutional Neural Networks (CNN) show remarkable results in many fields, among others in object classification and recognition. But these networks are limited by the tasks they are trained on, as they are designed to learn all tasks they will need during their lifetime in the beginning and hence are frozen. If now new tasks arrive, the network has to be trained completely new. These networks are therefore usually not able to learn in a continual manner, like humans are capable of. In this work, a novel approach is presented, where a deep neural network is used to continually learn new unseen object categories on images, which can be used in different fields, like mobile robots. First, different architectural strategies are proposed to dynamically adapt the network according to the categories it learns over time. This includes one strategy, where the last layer of our network is adapted and another one where multiple fully-connected layers are created for each new sequence. In order to prevent forgetting, different regularization strategies are shown, including a novel loss function where the classification is replaced by a regression. So, it is ensured that already learned categories are not forgotten by simultaneously enabling the network to learn new categories. Furthermore, the emerging problem of a discrepancy in the output distribution is recognized and different solutions are proposed. This includes a novel regularization strategy, where the outputs are divided by the variance per category. Finally, a novel dataset for continual learning is presented, which is especially suited for object recognition in our mobile robot environment (HOWS-CL-25). It consists of 150,795 synthetic images of 25 different household object categories in a randomly changing environment. Our approach can be classified as online learning, a special variant of incremental learning, where one is limited by the data the network can observe in a specific time step, without the access to previous training examples - also called rehearsal-free. This is a challenging and unsolved problem in comparison to other incremental learning approaches, which also use previous training examples, but as this thesis is focusing on an approach for mobile robots, online learning is more relevant. Our approach is tested on different datasets and compared with other solutions from literature. Additionally, our method was evaluated in the CLVISION workshop at CVPR 2020.