DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

A Data-driven Approach to Neural Architecture Search Initialization

Traoré, Kalifou René and Camero, Andrés and Zhu, Xiao Xiang (2023) A Data-driven Approach to Neural Architecture Search Initialization. Annals of Mathematics and Artificial Intelligence, pp. 1-28. Springer Nature. doi: 10.1007/s10472-022-09823-0. ISSN 1012-2443.

This is the latest version of this item.

[img] PDF - Only accessible within DLR bis 22 March 2024 - Published version

Official URL: https://rdcu.be/dgJ6b


Algorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed to tailor initialization techniques for NAS. However, the literature shows that a good initial set of solutions facilitates finding the optima. Therefore, in this study, we propose a data-driven technique to initialize a population-based NAS algorithm. First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm. We benchmark our proposed approach against random and Latin hypercube sampling initialization using three population-based algorithms, namely a genetic algorithm, an evolutionary algorithm, and aging evolution, on CIFAR-10. More specifically, we use NAS-Bench-101 to leverage the availability of NAS benchmarks. The results show that compared to random and Latin hypercube sampling, the proposed initialization technique enables achieving significant long-term improvements for two of the search baselines, and sometimes in various search scenarios (various training budget). Besides, we also investigate how an initial population gathered on the tabular benchmark can be used for improving search on another dataset, the So2Sat LCZ-42. Our results show similar improvements on the target dataset, despite a limited training budget. Moreover, we analyse the distributions of solutions obtained and find that that the population provided by the data-driven initialization technique enables retrieving local optima (maxima) of high fitness and similar configurations.

Item URL in elib:https://elib.dlr.de/189823/
Document Type:Article
Title:A Data-driven Approach to Neural Architecture Search Initialization
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Traoré, Kalifou RenéUNSPECIFIEDhttps://orcid.org/0000-0001-8780-2775UNSPECIFIED
Camero, AndrésUNSPECIFIEDhttps://orcid.org/0000-0002-8152-9381UNSPECIFIED
Date:22 March 2023
Journal or Publication Title:Annals of Mathematics and Artificial Intelligence
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:Yes
Page Range:pp. 1-28
Publisher:Springer Nature
Keywords:AutoML, Neural Architecture Search, Evolutionary Computation, Search, Initialization
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Artificial Intelligence
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Traoré, Mr Kalifou René
Deposited On:22 Nov 2022 13:12
Last Modified:18 Jul 2023 12:39

Available Versions of this Item

  • A Data-driven Approach to Neural Architecture Search Initialization. (deposited 22 Nov 2022 13:12) [Currently Displayed]

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.