Traoré, Kalifou René und Camero, Andrés und Zhu, Xiao Xiang (2023) A Data-driven Approach to Neural Architecture Search Initialization. Annals of Mathematics and Artificial Intelligence, Seiten 1-28. Springer Nature. doi: 10.1007/s10472-022-09823-0. ISSN 1012-2443.
Dies ist die aktuellste Version dieses Eintrags.
![]() |
PDF
- Verlagsversion (veröffentlichte Fassung)
3MB |
Offizielle URL: https://rdcu.be/dgJ6b
Kurzfassung
Algorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed to tailor initialization techniques for NAS. However, the literature shows that a good initial set of solutions facilitates finding the optima. Therefore, in this study, we propose a data-driven technique to initialize a population-based NAS algorithm. First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm. We benchmark our proposed approach against random and Latin hypercube sampling initialization using three population-based algorithms, namely a genetic algorithm, an evolutionary algorithm, and aging evolution, on CIFAR-10. More specifically, we use NAS-Bench-101 to leverage the availability of NAS benchmarks. The results show that compared to random and Latin hypercube sampling, the proposed initialization technique enables achieving significant long-term improvements for two of the search baselines, and sometimes in various search scenarios (various training budget). Besides, we also investigate how an initial population gathered on the tabular benchmark can be used for improving search on another dataset, the So2Sat LCZ-42. Our results show similar improvements on the target dataset, despite a limited training budget. Moreover, we analyse the distributions of solutions obtained and find that that the population provided by the data-driven initialization technique enables retrieving local optima (maxima) of high fitness and similar configurations.
elib-URL des Eintrags: | https://elib.dlr.de/189823/ | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Zeitschriftenbeitrag | ||||||||||||||||
Titel: | A Data-driven Approach to Neural Architecture Search Initialization | ||||||||||||||||
Autoren: |
| ||||||||||||||||
Datum: | 22 März 2023 | ||||||||||||||||
Erschienen in: | Annals of Mathematics and Artificial Intelligence | ||||||||||||||||
Referierte Publikation: | Ja | ||||||||||||||||
Open Access: | Ja | ||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||
In SCOPUS: | Ja | ||||||||||||||||
In ISI Web of Science: | Ja | ||||||||||||||||
DOI: | 10.1007/s10472-022-09823-0 | ||||||||||||||||
Seitenbereich: | Seiten 1-28 | ||||||||||||||||
Verlag: | Springer Nature | ||||||||||||||||
ISSN: | 1012-2443 | ||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||
Stichwörter: | AutoML, Neural Architecture Search, Evolutionary Computation, Search, Initialization | ||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||||||
HGF - Programmthema: | Erdbeobachtung | ||||||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||||||
DLR - Forschungsgebiet: | R EO - Erdbeobachtung | ||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Künstliche Intelligenz | ||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||
Institute & Einrichtungen: | Institut für Methodik der Fernerkundung > EO Data Science | ||||||||||||||||
Hinterlegt von: | Traoré, Mr René | ||||||||||||||||
Hinterlegt am: | 22 Nov 2022 13:12 | ||||||||||||||||
Letzte Änderung: | 22 Mär 2024 03:00 |
Verfügbare Versionen dieses Eintrags
- A Data-driven Approach to Neural Architecture Search Initialization. (deposited 22 Nov 2022 13:12) [Gegenwärtig angezeigt]
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags