Transfer learning of deep learning models for cloud masking in optical satellite images

Resumen

Earth observation through remote sensing sensors in orbiting satellites provide us with a great capacity to monitor our planet at high spatial and temporal resolutions. Nevertheless, to process all this ever-growing amount of data, we need to develop fast and accurate models adapted to the specific characteristics of the data of each sensor. For optical sensors, detecting the clouds in the image is an unavoidable first step to most of the land and ocean applications. Although detecting bright and opaque clouds is relatively easy, automatically identifying thin semi-transparent clouds or differentiating clouds from snow or bright surfaces is much more challenging. In addition, in the current scenario where the number of sensors in space is constantly growing, developing methodologies to transfer models across different satellite data is a pressing need. Henceforth, the overreaching goal of this Thesis is to develop accurate cloud detection models that exploit the different properties of the satellite images and to develop methodologies to transfer those models across different sensors. The four contributions of this Thesis are stepping stones in that direction. In the first contribution, "Multitemporal cloud masking in the Google Earth Engine", we implemented a lightweight multitemporal cloud detection model that runs on the Google Earth Engine platform and which outperforms the operational models for Landsat-8. The second contribution, "Transferring deep learning models for Cloud Detection between Landsat-8 and Proba-V", is a case-study of transferring a deep learning based cloud detection algorithm from Landsat-8 (30m resolution, 12 spectral bands and very good radiometric quality) to Proba-V which has a lower 333m resolution, only four bands and less good radiometric quality. The third paper "Cross sensor adversarial domain adaptation of Landsat-8 and Proba-V images for cloud detection" propose a learning-based domain adaptation transformation to Proba-V images to resemble those taken by Landsat-8 with the objective of transferring products designed on Landsat-8 to Proba-V. Finally, the fourth contribution, "Towards global flood mapping onboard low cost satellites with machine learning", tackles simultaneously cloud and flood water detection with a single deep learning model; in this case the model is implemented so that it could run onboard of a CubeSat (Phi-Sat) with an AI accelerator chip; the model is trained on Sentinel-2 images and we demonstrate how to transfer this model to the Phi-Sat camera. We trained this model in a newly compiled dataset of more than 100 verified flood events called WorldFloods. This model was launched on June 2021 onboard the Wild Ride D-Orbit mission and we are testing now its performance in space.