Intra- and inter-annual analysis of remote sensing time series with multitask learning

Multiannual analysis of remote sensing images for monitoring crop rotation, urbanization, or any other type of land cover transition helps to understand Earth dynamics. Existing methods studying images spanning over more than a year mainly focus only on land cover changes. Other methods perform land cover updates, but following a strategy guided by previous available maps and seeking to obtain a new one. Those methods are effective but may omit intra- and inter-annual interdependencies. In this work, we propose to perform multiannual remote sensing analysis using a multitask deep learning strategy that captures interdependencies across multiple objectives and is beneficial for performance and efficiency. The proposed method performs simultaneous intraannual semantic segmentation and inter-annual change detection by analyzing two time series of yearly data. It extracts spatio-temporal information using encoders with shared parameters to ensure robustness and enhance consistency across years and tasks. It further relies on transformer encoder mechanisms in the latent space for extracting global temporal information, and proceeds with separate decoders, outputting segmentation maps for each year, as well as a change map between the years. The model is trained using a multitask loss function. The proposed method captures local and global spatio-temporal information from the annual time series, allowing us to get extensive insights on the data content. Thus, the method is further directly applicable for the task of annual land-cover semantics updating and multitemporal semantic change detection. To validate our approach, we tested the algorithm on Sentinel-2 time series data of agricultural seasons 2021 and 2022, containing crop fields and labeled pixel-wise, examining crop rotation in an area of Austria. With our model architecture, we performed simultaneously the tasks of crop type classification pixel-wise and changed fields detection, and achieved promising results.