Uncertainty-Based Improvement of a Visual Classification System

Kurzfassung

Deep learning based classifiers have achieved tremendous success on different tasks. How- ever, this kind of classifiers cannot provide a reliable uncertainty estimation and express excessive overconfidence, which means that the models are always highly certain even on unfamiliar data. This behavior can easily lead to severe consequences in safety-critical ap- plications such as diagnosis of diseases, perception of self-driving car or robotics. Robots should be aware of the correctness of their predictions to prevent unnecessary accidents and/or to detect failure cases. Furthermore, this kind of introspection can help the robot to recognize unfamiliar environments and to adapt gradually. However, uncertainties of independent data examples do not take into account uncertainty related to dependencies between data examples. Given a classifier with good uncertainty estimations the classi- fication of objects with similar appearances should result in a high uncertainty about its prediction due to the appearance ambiguity, which can still lead to miss-classification. To overcome this issue, contextual information can be used to resolve the ambiguities and improve the initial prediction. In this work, the uncertainty estimation of deep neural networks are improved by employ- ing Bayesian neural networks. Within this context the applicability and performance of different inference techniques such as dropout variational inference and scalable Laplace approximation are evaluated. The resulting network is applied to a continuous learning use-case where the improved uncertainty estimation is used to select data samples for fine-tuning in an interactive (automatically and manually labeling) manner. For further improvements a conditional random field is employed to fuse the initial predictions with contextual information between objects. Extensive experiments are performed to show that the uncertainty estimation can be improved with Bayesian neural network in terms of different evaluation metrics. Besides, it is experimentally shown that manual efforts for collecting a dataset for fine-tuning a classifier can be reduced with the help of improved uncertainty estimation on two benchmark datasets. Last but not least, experiments also show that the conditional random field is able to further improve the performance by incorporating contextual information.