Humt, Matthias (2019) Laplace Approximation for Uncertainty Estimation of Deep Neural Networks. DLR-Interner Bericht. DLR-IB-RM-OP-2019-108. Master's. TUM.
PDF
20MB |
Abstract
Deep learning is at the frontier of machine learning and automation, using deep neural networks as the main workhorse, which have revolutionized the way we extract information from large amounts of data in computer vision, natural language processing and other domains. Moving from purely academic to real world scenarios has renewed interest into the way those powerful algorithms draw their conclusions and how to quantify the quality of their predictions beyond mere accuracy. From a practitioner’s point of view, the most important information to obtain alongside the algorithm’s prediction is the uncertainty attached to it, which is the basis for an accurate assessment of confidence. Unfortunately, extracting this information from large models and datasets has proven to be difficult. A common approach so far has been to devise a method that makes as many approximations as necessary to render the problem tractable while still yielding at least somewhat useful uncertainty estimates. This work looks at the problem from a slightly different angle: By first choosing a tractable and comparatively simple method, the burden then lies on the model design to lend itself to the chosen approximation. To this end, the most popular deep neural network architectures are compared based on their compliance to uncertainty estimation by Laplace approximation, assessing empirically the methods potentials and deficiencies as well as its applicability to large models and datasets while working towards an understanding how architectural choices correlate with the quality of obtained uncertainty estimates
Item URL in elib: | https://elib.dlr.de/131938/ | ||||||||
---|---|---|---|---|---|---|---|---|---|
Document Type: | Monograph (DLR-Interner Bericht, Master's) | ||||||||
Title: | Laplace Approximation for Uncertainty Estimation of Deep Neural Networks | ||||||||
Authors: |
| ||||||||
Date: | 15 July 2019 | ||||||||
Refereed publication: | No | ||||||||
Open Access: | Yes | ||||||||
Status: | Published | ||||||||
Keywords: | Neural Networks | ||||||||
Institution: | TUM | ||||||||
Department: | Informatics | ||||||||
HGF - Research field: | Aeronautics, Space and Transport | ||||||||
HGF - Program: | Space | ||||||||
HGF - Program Themes: | Space System Technology | ||||||||
DLR - Research area: | Raumfahrt | ||||||||
DLR - Program: | R SY - Space System Technology | ||||||||
DLR - Research theme (Project): | R - Vorhaben Intelligente Mobilität (old) | ||||||||
Location: | Oberpfaffenhofen | ||||||||
Institutes and Institutions: | Institute of Robotics and Mechatronics (since 2013) | ||||||||
Deposited By: | Beinhofer, Gabriele | ||||||||
Deposited On: | 13 Dec 2019 09:55 | ||||||||
Last Modified: | 22 Jun 2020 13:59 |
Repository Staff Only: item control page