elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Estimating Model Uncertainty of Neural Networks in Sparse Information Form

Lee, Jongseok and Humt, Matthias and Feng, Jianxiang and Triebel, Rudolph (2020) Estimating Model Uncertainty of Neural Networks in Sparse Information Form. In: 37th International Conference on Machine Learning, ICML 2020. Proceedings of Machine Learning Research. 37th International Conference on Machine Learning (ICML), Vienna, Austria. ISBN 978-171382112-0. ISSN 2640-3498.

[img] PDF
1MB

Official URL: https://www.webofscience.com/wos/woscc/full-record/WOS:000683178505077

Abstract

We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i.e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods.

Item URL in elib:https://elib.dlr.de/135531/
Document Type:Conference or Workshop Item (Other)
Title:Estimating Model Uncertainty of Neural Networks in Sparse Information Form
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Lee, JongseokUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Humt, MatthiasDLRUNSPECIFIEDUNSPECIFIED
Feng, JianxiangUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Triebel, RudolphUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Date:13 July 2020
Journal or Publication Title:37th International Conference on Machine Learning, ICML 2020
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In SCOPUS:Yes
In ISI Web of Science:Yes
Publisher:Proceedings of Machine Learning Research
ISSN:2640-3498
ISBN:978-171382112-0
Status:Published
Keywords:Bayesian Deep Learning, Uncertainty Quantification, Information Theory
Event Title:37th International Conference on Machine Learning (ICML)
Event Location:Vienna, Austria
Event Type:international Conference
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Space System Technology
DLR - Research area:Raumfahrt
DLR - Program:R SY - Space System Technology
DLR - Research theme (Project):R - Vorhaben Intelligente Mobilität (old)
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Deposited By: Lee, Jongseok
Deposited On:21 Jul 2020 09:45
Last Modified:20 Mar 2023 10:57

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.