DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

Schnaus, Dominik and Lee, Jongseok and Cremers, Daniel and Triebel, Rudolph (2023) Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks. In: 40th International Conference on Machine Learning, ICML 2023. Fortieth International Conference on Machine Learning, 2023-07-23 - 2023-07-29, Hawaii. ISSN 2640-3498.

[img] PDF

Official URL: https://proceedings.mlr.press/v202/schnaus23a.html


In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the favorable properties of our priors are desirable. Major enablers are our technical contributions: (1) the sums-of-Kronecker-product computations, and (2) the derivations and optimizations of tractable objectives that lead to improved generalization bounds. Empirically, we exhaustively show the effectiveness of this method for uncertainty estimation and generalization.

Item URL in elib:https://elib.dlr.de/195286/
Document Type:Conference or Workshop Item (Poster)
Title:Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Lee, JongseokUNSPECIFIEDhttps://orcid.org/0000-0002-0960-0809UNSPECIFIED
Triebel, RudolphUNSPECIFIEDhttps://orcid.org/0000-0002-7975-036XUNSPECIFIED
Journal or Publication Title:40th International Conference on Machine Learning, ICML 2023
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:No
Keywords:Bayesian Deep Learning, Uncertainty in Deep Learning
Event Title:Fortieth International Conference on Machine Learning
Event Location:Hawaii
Event Type:international Conference
Event Start Date:23 July 2023
Event End Date:29 July 2023
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Explainable Robotic AI
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Deposited By: Lee, Jongseok
Deposited On:13 Jun 2023 11:55
Last Modified:24 Apr 2024 20:55

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.