Schnaus, Dominik und Lee, Jongseok und Cremers, Daniel und Triebel, Rudolph (2023) Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks. In: 40th International Conference on Machine Learning, ICML 2023. Fortieth International Conference on Machine Learning, 2023-07-23 - 2023-07-29, Hawaii. ISSN 2640-3498.
PDF
1MB |
Offizielle URL: https://proceedings.mlr.press/v202/schnaus23a.html
Kurzfassung
In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the favorable properties of our priors are desirable. Major enablers are our technical contributions: (1) the sums-of-Kronecker-product computations, and (2) the derivations and optimizations of tractable objectives that lead to improved generalization bounds. Empirically, we exhaustively show the effectiveness of this method for uncertainty estimation and generalization.
elib-URL des Eintrags: | https://elib.dlr.de/195286/ | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Konferenzbeitrag (Poster) | ||||||||||||||||||||
Titel: | Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks | ||||||||||||||||||||
Autoren: |
| ||||||||||||||||||||
Datum: | 2023 | ||||||||||||||||||||
Erschienen in: | 40th International Conference on Machine Learning, ICML 2023 | ||||||||||||||||||||
Referierte Publikation: | Ja | ||||||||||||||||||||
Open Access: | Ja | ||||||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||||||
In SCOPUS: | Ja | ||||||||||||||||||||
In ISI Web of Science: | Nein | ||||||||||||||||||||
ISSN: | 2640-3498 | ||||||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||||||
Stichwörter: | Bayesian Deep Learning, Uncertainty in Deep Learning | ||||||||||||||||||||
Veranstaltungstitel: | Fortieth International Conference on Machine Learning | ||||||||||||||||||||
Veranstaltungsort: | Hawaii | ||||||||||||||||||||
Veranstaltungsart: | internationale Konferenz | ||||||||||||||||||||
Veranstaltungsbeginn: | 23 Juli 2023 | ||||||||||||||||||||
Veranstaltungsende: | 29 Juli 2023 | ||||||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||||||||||
HGF - Programmthema: | Robotik | ||||||||||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||||||||||
DLR - Forschungsgebiet: | R RO - Robotik | ||||||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Erklärbare Robotische KI | ||||||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||||||
Institute & Einrichtungen: | Institut für Robotik und Mechatronik (ab 2013) Institut für Robotik und Mechatronik (ab 2013) > Perzeption und Kognition | ||||||||||||||||||||
Hinterlegt von: | Lee, Jongseok | ||||||||||||||||||||
Hinterlegt am: | 13 Jun 2023 11:55 | ||||||||||||||||||||
Letzte Änderung: | 24 Apr 2024 20:55 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags