A Likelihood-Based Generative Approach for Spatially Consistent Precipitation Downscaling

Abstract

Deep learning has emerged as a promising tool for precipitation downscaling. However, current models rely on likelihood-based loss functions to properly model the precipitation distribution, leading to spatially inconsistent projections when sampling. This work explores a novel approach by fusing the strengths of likelihood-based and adversarial losses used in generative models. As a result, we propose a likelihood-based generative approach for precipitation downscaling, leveraging the benefits of both methods.

1 Introduction

Global Climate Models (GCMs) simulate the spatio-temporal evolution of climate by numerically solving the set of physical equations describing its constituent components and interconnections (Chen et al., 2021). By running these models under various emission scenarios, it is possible to generate future projections under climate change conditions (O’Neill et al., 2016). However, GCMs have coarse resolutions due to computational and physical constraints, limiting their use in regional-scale studies.

Statistical Downscaling (SD) attempts to overcome this limitation by modeling the relationship between the coarse (low-resolution) and regional (high-resolution) scales. Under the Perfect Prognosis (PP) approach (Maraun & Widmann, 2018), an empirical link is established between a set of large-scale synoptic variables (representing the state of the atmosphere) and the local variables of interest (e.g., precipitation). This link is learned via a statistical model using observational datasets. The model is then applied to the large-scale variables from a GCM to obtain the corresponding regional projections for future scenarios.

Recently, Deep Learning (DL) has emerged as a promising technique for Perfect Prognosis Statistical Downscaling (PP-SD), given its ability to model non-linear relationships and handle spatial data (Goodfellow et al., 2016). The resulting DL-based regional projections have proven useful for several climate change applications, even forming the first continental-wide contribution of a PP-based technique to the CORDEX initiative (Baño-Medina et al., 2022).

Unfortunately, regression-based DL models tend to focus on capturing the expected value of the output distribution, thus leading to the underrepresentation of extremes. This can be problematic for variables such as precipitation, highly characterized by these events (e.g., heavy rainfalls). To address this, recent DL models for PP-SD maximize the likelihood with respect to a explicit distribution reflecting the dynamics of precipitation (Baño-Medina et al., 2022). However, because these distributions are learned independently for each grid-point at the regional scale, the resulting projections may lack spatial consistency when sampling (González-Abad et al., 2021).

In this work, we explore generative models to address this issue. For the first time, we combine a likelihood-based training with a conditional Generative Adversarial Network (cGAN) for the downscaling of precipitation. Through our experimental setup, we show how the proposed approach enables the DL model to sample spatially consistent precipitation fields, while allowing defining a explicit probability distribution over the target data.

2 Background

2.1 Deep Learning for Precipitation Downscaling

Inspired by advances in the Super Resolution (SR) field (Wang et al., 2020), recent studies have explored SR models for the downscaling of precipitation (Vandal et al., 2017; Cheng et al., 2020a; Passarella et al., 2022; Sharma & Mitra, 2022). In this context, SR models aim to establish a link between the coarse and regional versions of a specific field. However, using a surface variable from a GCM often results in reproducing the regional biases caused by its coarse resolution, limiting their effectiveness for climate downscaling.

For this reason, in the climate context PP-SD models constitute the standard, as they rely on large-scale synoptic variables representing the state of the atmosphere, which are properly reproduced by the coarse resolution of GCMs. Several architectures have been explored such as recurrent networks (Misra et al., 2018), a combination of convolutional and dense layers (Pan et al., 2019; Baño-Medina et al., 2020) and fully-convolutional (Adewoyin et al., 2021; Quesada-Chacón et al., 2022), some of them even generating projections in future scenarios (Baño-Medina et al., 2021; Soares et al., 2023). Among these architectures, the U-Net (Ronneberger et al., 2015) has recently shown promising results (Quesada-Chacón et al., 2022; Adewoyin et al., 2021), even in related fields such as emulation (Doury et al., 2023, 2024).

2.2 Extreme Precipitation

Due to the dynamics of precipitation, which typically adheres to exponential probability distributions, and its non-continuous nature (occurrence and amount), regression-based DL models encounter difficulties in accurately modeling it. This often leads to significant issues, such as underestimation of extreme precipitation events (Rampal et al., 2024b).

To address this challenge, and drawing inspiration from previous work (Dunn, 2004; Cannon, 2008), authors in (Baño-Medina et al., 2022) train a DL model by minimizing the Negative Log-Likelihood (NLL) of a Bernoulli and gamma distributions for the occurrence and amount, respectively. By working under this assumption, which aligns with the dynamics of precipitation (Williams, 1997), they are able to model the whole distribution, including the extremes. Its success has led to its application across various regions (Sun & Lan, 2021; Rampal et al., 2022; Kheir et al., 2023; Hosseini Baghanam et al., 2024), making it the most extended DL-based PP-SD model. Unfortunately, this approach models a different probability distribution for each of the grid-points forming the downscaled variable, resulting in spatial inconsistency when sampling from these distributions, leading to unrealistic projections (González-Abad et al., 2021).

2.3 Generative Precipitation Downscaling

Recently, Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have attracted significant attention in the SD field. Unlike regression-based DL models, which generally minimize loss functions aiming at capturing the mean (e.g., Mean Squared Error, MSE), GANs minimize an adversarial loss which encourages the generator to better reproduce the underlying distribution of the data in order to fool the discriminator. This leads to improved reproduction of extremes and finer details in precipitation downscaling (Rampal et al., 2024b). In addition, GANs allow computing ensembles of predictions, although unlike the NLL approach, these are computed by passing noise as input to the generator, not by explicitly modeling the corresponding distribution. In the context of PP-SD, GANs have not been explored yet, although they show promise in related areas such as SR downscaling (Leinonen et al., 2020; Cheng et al., 2020b), meteorological downscaling (Price & Rasp, 2022; Harris et al., 2022) and emulation (Rampal et al., 2024a).

3 Experimental Framework

3.1 Region of Study and Data

In this work, we focus on daily precipitation downscaling over a domain centered on the Alps (37.6°N-50.4°N and 3.6°E-16.4°E), a region of interest due to its prominent orography, which significantly influences local precipitation. As we frame this study within the PP approach, we rely on the ERA5 reanalysis dataset (Hersbach et al., 2020) (quasi-observational) at $1^{\circ}$ resolution for the large-scale variables (predictors) and on the observational dataset E-OBS (Cornes et al., 2018) for the local-scale variable (predictand) at $0.1^{\circ}$ resolution. Following previous works (Baño-Medina et al., 2022; Soares et al., 2023), we select as predictors the air temperature, specific humidity, geopotential height, and the meridional and zonal wind components at 500, 700, and 850 hPa.

3.2 Standard Deep Learning Models

Following the aforementioned recent advances in PP-SD, we rely on the U-Net, a fully-convolutional model. Specifically, we adhere to the implementation details described in (Doury et al., 2023). We train two different versions of this model: U-Net (MSE) and U-Net (NLL). The former minimizes the MSE, whereas the latter minimizes the NLL of a Bernoulli-gamma distribution, as proposed in (Baño-Medina et al., 2022). While U-Net (MSE) directly computes the downscaled precipitation, U-Net (NLL) computes, for each grid-point in the predictand, the parameters $p$, $\alpha$ and $\beta$ defining the corresponding Bernoulli-gamma distribution. Consequently, for NLL-based models, the final prediction corresponds to a random sample from the modeled distributions.

We divide the observational dataset into a training (1980-2010) and a test (2011-2022) period. These models are trained using the Adam optimizer (Kingma & Ba, 2014) with a learning rate of $10^{-4}$ and a batch size of 64.

3.3 Likelihood-Based Generative Approach

The main contribution of this work involves cGAN models (Mirza & Osindero, 2014) for PP-SD. Unlike standard GANs, cGANs allow conditioning the generation process on specific data by feeding it to both the generator and discriminator. In addition to the adversarial loss, cGANs also minimize a content loss that ensures that conditionally generated samples align with the true target values. In the context of precipitation downscaling, combining these loss functions allows the generator to produce precipitation fields that both fool the discriminator and accurately reproduce the daily precipitation conditioned on the specific large-scale synoptic state (Harris et al., 2022; Rampal et al., 2024a).

To represent the state-of-the-art in generative precipitation downscaling, we follow (Ravuri et al., 2021; Harris et al., 2022) and implement what we denote as cGAN (MSE), as this model relies on the MSE as its content loss. However, to avoid the so-called blurry effect, where the model smooths out fine details and produces predictions resembling the mean, the content loss is applied to the mean of an ensemble of predictions. Note that, although the content loss is computed over an ensemble, at inference time the prediction of this model corresponds to a single sample.

The main contribution of this work is the introduction of the cGAN (NLL) model, which combines likelihood-based training with the cGAN framework. Unlike other approaches, this model uses the NLL of the Bernoulli-gamma distribution as the content loss. As a result, the generator produces a set of probability distributions, similarly to U-Net (NLL). However, by passing a random sample from these distributions to the discriminator, the generator is forced to improve the spatial consistency to minimize the adversarial loss, as spatially inconsistent precipitation fields are easily detectable by the discriminator. Therefore, adversarial training should lead the generator to learn spatially-aware distributions across grid-points in the downscaled field, addressing the drawbacks of standard likelihood-based DL models.

For both cGAN (MSE) and cGAN (NLL) models, we use the same architecture for the generator as that used for the U-Net (MSE) and U-Net (NLL) models. For the discriminator, we implement a fully convolutional network that processes both the large- and regional-scale data through a series of convolutional and dense layers. Following previous works (Leinonen et al., 2020; Harris et al., 2022; Rampal et al., 2024a), we rely on the Wasserstein formulation of GANs (Arjovsky et al., 2017) with a gradient penalty term (Gulrajani et al., 2017). This training framework is popular due to its theoretical properties, such as the possibility to train the discriminator to optimality. For both cGAN models, and following (Arjovsky et al., 2017), we use RMSprop as the optimizer with a learning rate of $10^{-5}$ and a batch size of 64. The generator and discriminator are trained in an adversarial manner, with the discriminator being updated five times for each update of the generator. The training and test sets cover the years detailed in Section 3.2.

4 Results

Figure 1 shows the violin plot for four different metrics computed on the test set: the relative bias of the mean, the relative bias of the Simple Daily Intensity Index (SDII), which corresponds to the precipitation amount for rainy days ($\geq 1$mm), the Root Mean Square Error (RMSE), and the ratio of standard deviations. These metrics are shown for the four intercompared models: U-Net (MSE), U-Net (NLL), cGAN (MSE), and cGAN (NLL).

Besides the U-Net (MSE) which overestimates it, all models reproduce the mean. However, for the SDII, which represents values further from the mean, MSE-based models, particularly U-Net, tend to underestimate it, whereas NLL-based models are able to reproduce it accurately. As expected, MSE-based models reveal superior performance for the RMSE due to their relation with the minimized loss function. Although NLL-based models are expected to exhibit higher RMSE due to sampling-induced errors, cGAN (NLL) achieves comparable performance to MSE-based models. In terms of the ratio of standard deviations, NLL-based models show superior performance, primarily attributed to sampling from the Bernoulli-gamma distribution. However, the cGAN (MSE) model, despite sampling, falls short in accurately reproducing this aspect, which may indicate that explicitly defining the distribution of precipitation is key.

Figure 1: Violin plot showing the results for four different metrics computed on the test set: the relative bias of the mean and the SDII, the RMSE, and the ratio of standard deviations. Each metric displays the results corresponding to the different DL models intercompared: U-Net (MSE), U-Net (NLL), cGAN (MSE), and cGAN (NLL).

Figure 2 displays the precipitation histogram during the test period across all grid-points in the predictand. The logarithmic scale of the y-axis facilitates the assessment of model performance, particularly for less frequent extreme events. Examining the histogram within the 0-50 mm interval (top-right corner), we observe a notable decline in precipitation values from 0 to approximately 1 mm in the target dataset. The U-Net (MSE) model fails to adapt to this pattern, resulting in an overestimation of precipitation. In contrast, other models accurately replicate this decrease. This discrepancy supports the U-Net (MSE) model’s overestimation of the mean in the corresponding violin plot. The adversarial training of the cGAN (MSE) allows it to adjust to this decrease, while NLL-based models effectively handle it due to the underlying distributional assumption. Additionally, the histogram reveals that MSE-based models underestimate the distribution beyond approximately 15 mm, as minimizing the MSE loss function mainly involves fitting the mean. Conversely, NLL-based models accurately reproduce this segment of the distribution, as also evidenced in the SDII violin plot.

Expanding our focus to the histogram spanning the 0-350 mm interval, representing precipitation extremes, a similar pattern emerges: only NLL-based models successfully replicate these extreme values, including those surpassing 200 mm. This underscores the effectiveness of assuming a gamma distribution for modeling the precipitation amount.

Figure 2: Histogram of the precipitation distribution for the test period, aggregated across all grid-points in the predictand. The black line represents the target observational dataset, while the different colors correspond to the various DL models being compared. A zoomed-in view for values in the 0-50 interval is provided in the top-right corner of the histogram.

Figure 3 depicts the predictions of each DL model for a specific day of the test period. As anticipated, the U-Net (MSE) exhibits the blurry effect, lacking fine details in the generated field. In contrast, the cGAN (MSE) addresses this issue by leveraging the adversarial loss, leading the generator to better capture these details to fool the discriminator, along with the effect of computing the MSE over the ensemble of predictions. As previously discussed, the independent nature of the distributions resulting from minimizing the NLL loss function leads to spatial inconsistencies in the generated field, evident in the U-Net (NLL) model’s prediction. However, incorporating the adversarial loss enables the cGAN (NLL) to achieve greater consistency while still adhering to the Bernoulli-gamma distribution. In fact, the prediction of the cGAN (NLL) model falls between that of the cGAN (MSE) and the U-Net (NLL).

Figure 3: Comparison of predictions generated by the DL models intercompared for a day in the test period.

5 Conclusions

In this work, we have introduced a novel likelihood-based generative approach for precipitation downscaling. This method leverages a combination of likelihood and adversarial losses, enabling the model to properly reproduce the target distribution while generating spatially consistent precipitation fields, addressing a main challenge for standard likelihood-based DL models. Furthermore, likelihood-based loss functions enable generative models to produce explicit probability distributions (e.g., Bernoulli-gamma) for precipitation. This capability is crucial when downscaling future GCM projections, as estimating probabilities of extreme events is vital for risk assessment.

Future work includes evaluating the likelihood-based generative model in the GCM space and compare the resultant projections with those of established PP-SD models. We also plan to employ eXplainable Artificial Intelligence (XAI) techniques to understand the impact of the adversarial loss on the modeled probability distributions and learned patterns.