Estimating Uncertainty of Deep Learning Multi-Label Classifications using Laplace Approximation

Abstract

With the huge successes of deep learning and its application in critical areas such as medical diagnosis or autonomous driving and in fields with noisy and very varying data such as remote sensing, the need for reliable confidence statements about such model's predictions becomes apparent. Therefore, uncertainty estimation methods for neural networks have raised rising interest in the machine learning community. While various methods for regression and multi-class classification tasks have been published, the field of multi-label classification has hardly been considered yet. In this work, we derive the Kronecker-factored Laplace approximation in the multi-label setting, a method to approximate the intractable posterior distribution over the parameters of neural networks. We employ this method in the remote sensing domain and estimate the model uncertainty of eight deep neural networks that have been trained on an aerial scene classification dataset. By comparing the probabilistic classifiers to their deterministic counterparts, we evaluate the potential for using the uncertainty estimates to improve the calibration of those classifiers as well as the out-of-distribution detection. We found that we can improve the calibration for overconfident classifiers whereas for underconfident ones, this method might not be beneficial. Furthermore, the ability to improve the separation from in- and out-of-distribution data seems to be depending on the depth of the neural network within one model family.