Cloud Segmentation and Classification from All-Sky Images Using Deep Learning

Abstract

For transforming the energy sector towards renewable energies, solar power is regarded as one of the major resources. However, it is not uniformly available all the time, leading to fluctuations in power generation. Clouds have the highest impact on short-term temporal and spatial variability. Thus, forecasting solar irradiance strongly depends on current cloudiness conditions. As the share of solar energy in the electrical grid is increasing, so-called nowcasts (intra-minute to intra-hour forecasts) are beneficial for grid control and for reducing required storage capacities. Furthermore, the operation of concentrating solar power (CSP) plants can be optimized with high resolution spatial solar irradiance data. A common nowcast approach is to analyze ground-based sky images from All-Sky Imagers. Clouds within these images are detected and tracked to estimate current and immediate future irradiance, whereas the accuracy of these forecasts depends primarily on the quality of pixel-level cloud recognition. State-of-the-art methods are commonly restricted to binary segmentation, distinguishing between cloudy and cloudless pixels. Thereby the optical properties of different cloud types are ignored. Also, most techniques rely on threshold-based detection showing difficulties under certain atmospheric conditions. In this thesis, two deep learning approaches are presented to automatically determine cloud conditions. To identify cloudiness characteristics like a free sun disk, a multi-label classifier was implemented assigning respective labels to images. In addition, a segmentation model was developed, classifying images pixel-wise into three cloud types and cloud-free sky. For supervised training, a new dataset of 770 images was created containing ground truth labels and segmentation masks. Moreover, to take advantage of large amounts of raw data, self-supervised pretraining was applied. By defining suitable pretext tasks, representations of image data can be learned facilitating the distinction of cloud types. Two successful techniques were chosen for self-supervised learning: Inpainting- uperresolution and DeepCluster. Afterwards, the pretrained models were fine-tuned on the annotated dataset. To assess the effectiveness of self-supervision, a comparison with random initialization and pretrained ImageNet weights was conducted. Evaluation shows that segmentation in particular benefits from self-supervised learning, improving accuracy and IoU about 3% points compared to ImageNet pretraining. The best segmentation model was also evaluated on binary segmentation. Achieving an overall accuracy of 95.15%, a state-of-the art Clear-Sky-Library (CSL) is outperformed significantly by over 7% points.