elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Learning Variational Models with Unrolling and Bilevel Optimization

Brauer, Christoph and Breustedt, Niklas and de Wolff, Timo and Lorenz, Dirk (2024) Learning Variational Models with Unrolling and Bilevel Optimization. Analysis and Applications. World Scientific. ISSN 0219-5305. (Submitted)

[img] PDF - Only accessible within DLR bis 31 December 2024 - Preprint version (submitted draft)
778kB

Abstract

In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot (and learning the stepsize gives a significant improvement), the number of unrolled iterations plays a minor role.

Item URL in elib:https://elib.dlr.de/199389/
Document Type:Article
Title:Learning Variational Models with Unrolling and Bilevel Optimization
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Brauer, ChristophUNSPECIFIEDhttps://orcid.org/0000-0003-2913-0768UNSPECIFIED
Breustedt, NiklasUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
de Wolff, TimoUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Lorenz, DirkUNSPECIFIEDhttps://orcid.org/0000-0002-7419-769XUNSPECIFIED
Date:2024
Journal or Publication Title:Analysis and Applications
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In SCOPUS:Yes
In ISI Web of Science:Yes
Publisher:World Scientific
Series Name:Interaction between Harmonic Analysis and Data Science (Special Issue)
ISSN:0219-5305
Status:Submitted
Keywords:algorithm unrolling; bilevel optimization; supervised learning; risk minimization
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Space System Technology
DLR - Research area:Raumfahrt
DLR - Program:R SY - Space System Technology
DLR - Research theme (Project):R - Project Factory of the Future
Location: Stade
Institutes and Institutions:Institut für Systemleichtbau > Production Technologies SD
Deposited By: Brauer, Dr. Christoph
Deposited On:21 Nov 2023 21:15
Last Modified:21 Nov 2023 21:15

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.