Selecting CMIP 6-Based Future Climate Scenarios for Impact and Adaptation Studies

Climate change impact assessment studies often use future projections of only a few global climate models (GCMs) due to limited research resources. Here we develop a novel method to select a small subset of GCMs that widely capture the uncertainty range of large ensemble. By applying this method, we select a subset of five GCM projections from the Coupled Model Intercomparison Project Phase 6 (CMIP6) ensemble for impact and adaptation studies in Japan. At first, we omit GCMs whose global warming projections have been evaluated to be overestimated in the recent literature. Then, we select a subset of five GCMs that widely captures the uncertainty ranges for 8 climate variables and have good performances in present-climate simulations. These selected GCM simulations will be used to provide better climate scenarios for impact and adaptation studies than those in the previous impact assessment project. (Citation: Shiogama, H., N. N. Ishizaki, N. Hanasaki, K. Takahashi, S. Emori, R. Ito, T. Nakaegawa, I. Takayabu, Y. Hijioka, Y. N. Takayabu, and R. Shibuya, 2021: Selecting CMIP6based future climate scenarios for impact and adaptation studies. SOLA, 17, 57−62, doi:10.2151/sola.2021-009.)


Introduction
Climate change projections based on global climate models (GCMs) are essential for impact assessments and planning adaptation strategies. Because there are intrinsic uncertainties in climate change projections, it is necessary to investigate the ranges of GCM simulations. However, policy makers often prefer to use a small set of climate scenarios as representative climate futures (Whetton et al. 2012) or storylines (Shepherd 2019) rather than the full GCM ensemble. Limited research resources also cause many impact assessment researchers to use outputs of only a few GCMs out of the many ones that contributed to the Coupled Model Intercomparison Project Phase 5 (CMIP5) or 6 (CMIP6). For example, a global-scale multi-sector impact assessment project, phase 2b of the Intersectoral Impact Model Intercomparison Project (ISIMIP2b; https://www.isimip.org), used only four GCMs from CMIP5 (Frieler et al. 2017). It has been suggested that the ISIMIP2b GCM subset does not broadly represent the CMIP5 uncertainty ranges of future projections (McSweeney and Jones 2016; Ito et al. 2020). Another coordinated multi-sector impact assessment project in Japan called S-8 also used only four CMIP5 GCMs (Hanasaki et al. 2014;Ishizaki et al. 2020). Shiogama et al. (2020 suggested that the selected GCMs of S-8 do not cover the CMIP5 range of shortwave radiation changes in Japan. Shi2020 also developed a method to select a better subset of GCMs that are more widely distributed and are not biased regarding shortwave radiation. For the successor impact assessment project of S-8, called S-18 (2020-2024, https://s-18ccap.jp/en/), and the Climate Change Adaptation Research Program (CCARP) of National Institute for Environmental Studies (https://ccca.nies.go.jp/en/program/index. html), we will produce a CMIP6-based (Eyring et al. 2016;O'Neill et al. 2016) dataset of bias-corrected climate scenarios in Japan in a future study. This dataset should meet the following conditions: • The national (Japanese) GCMs MRI-ESM2.0 and MIROC6 should be involved because support from national modelling centres is useful for impact researchers. • The dataset includes 8 climate variables (see the next section) that are widely used in impact model research. • The performance of the present climate simulations of the selected GCMs should be good. • The selected GCM subset covers the CMIP6 distributions of future changes in the 8 variables well. • All the selected GCMs have the output data of three shared socioeconomic pathways (SSPs): SSP1-2.6, SSP2-4.5 and SSP5-8.5 (O'Neill et al. 2016). • The issue of overestimated future warming in CMIP6 (see Section 2) should be addressed.
Although several methods of model selection have been developed (McSweeney et al. 2012;Cannon 2015;Mendlik and Gobiet 2016;Shi2020), no methods meet the above conditions. For example, Shi2020's method is adaptive for only one variable. Shi2020 and Cannon (2015) selected GCMs to maximize the distance between GCMs for future climate change projections. The method of Mendlik and Gobiet (2016) is based on a combination of principal component analysis and clustering of future climate change projections. Those methods do not take into account the performance of the present climate simulations. McSweeney et al. (2012) eliminated GCMs that perform particularly poorly in the present climate simulations of the major features of Southeast Asian climate (specifically, the Asian summer monsoon), and then selected from those remaining a subset that captures a broad range of future changes in temperature, monsoon characteristics and precipitation. Their method is difficult to extend to 8 variables. The aim of this paper is to develop a novel method to select a small subset of GCMs that meets the above conditions.

Selection methods
We will select a subsample of GCMs that are distributed to widely capture the CMIP6* uncertainty ranges of all 8 variables. Additionally, we will consider the performances of the presentclimate simulations in the model selection process.
We define N as the total number of CMIP6* GCMs (13 here). Let R (i, v) be the spatiotemporal root mean square error (RMSE) over the 15°N−55°N, 120°E−157.5°E region (including land and ocean area) across the four seasons for the v-th variables of the i-th GCM. The NCEP Climate Forecast System Reanalysis (Saha et al. 2014) data during the 1995−2014 period are used as the reference. All the data have been interpolated to a 1° × 1° grid. We divide R (i, v) by the median across the GCMs to compute the normalized RMSE, R * (i, v). We also compute the weighted averages of R * (i, v) across the variables: where w (v) is the weight of the v-th variable. We can take into account the priorities of variables for targeted impact assessment studies by changing w (v) for each variable. For example, if impact researchers are interested in pr rather than the other variables, it is possible to set w ( pr) = 2 and set the weights to 1 for the other variables. Because S-18 and CCARP cover a wide area of impact research and important variables are different between research topics, we set w (v) = 1 for all the variables. Figure 1 shows R * (i, v) and R -* (i ). MRI-ESM2.0 has good performance for the present-climate simulations for all the variables. MIROC6 has better skills for tasmin, pr and hurs than the median of CMIP6* but does not have good skills for tasmax, rsds and sfcWind. We use R -* (i ) in part of the model selection process, as mentioned below. When we computed R * (i, v) and R -* (i ) using data in a longer period (1980− 2014), there is little changes in biases (not shown) and the selected GCMs are not different from those mentioned later.
The area-averaged anomalies (2080−2099 minus 1995−2014) for the west and east Japan (Fig. 2, land area only) are calculated for each variable, because these two areas have different climatology. Let X (i, s, a, e, v) be a value of the v-th variable for the i-th GCM, s-th season, e-th SSP and a-th region. The number of GCMs that can be selected is denoted by M. The GCM subsample from the CMIP6* ensemble is selected based on the following method. We collect the outputs of 21 GCMs from the CMIP6 ensemble that provide the monthly mean data of the 8 variables for all the historical, SSP1-2.6, SSP2-4.5 and SSP5-8.5 runs (Eyring et al. 2016;O'Neill et al. 2016) (Table 1). Because, in a future study, we will make bias-corrected data of a single ensemble member for each selected GCM, only one ensemble member for each GCM is examined here. Please note that most of our analyses are based on the monthly mean variables, although daily data are necessary for the bias correction process in a future study. We will consider the availability of daily data in our model selection process in Section 3.
First, FGOALS-g3 is omitted because of very large biases in the Japan area (e.g., a warm bias of 10°C relative to the NCEP Climate Forecast System Reanalysis (Saha et al. 2014)).
Several GCMs of CMIP6 have equilibrium climate sensitivities higher than the upper bound of the likely range in the IPCC 5th Assessment Report (i.e., 4.5°C) (Collins et al. 2013;Zelinka et al. 2020). Recent studies have suggested that some of those high-sensitivity GCMs have overestimated the global mean surface air temperature trends in recent decades compared to the observations, and the reliabilities of their future warming projections are low (Tokarska et al. 2020;Liang et al. 2020;Nijsse et al. 2020). Observationally constrained CMIP6 warming is consistent with the CMIP5 range (Tokarska et al. 2020). Therefore, based on the study by Tokarska et al. (2020), we exclude 5 GCMs with large warming trends in recent decades: CanESM5, EC-Earth3, EC-Earth3-Veg, HadGEM3-GC31-LL and UKESM1-0-LL. ACCESS-CM2 (the 1981−2017 global mean tas trend (2015−2017 is under SSP2-8.5) is 0.25°C/decade), ACCESS-ESM1-5 (0.34°C/ decade), CanESM5-CanOE (0.35°C/decade) and MPI-ESM1-2-LR (0.18°C/decade) were not analysed by Tokarska et al. (2020). We omit ACCESS-ESM1-5 and CanESM5-CanOE due to the large recent warming trends. To the best of our knowledge, this is the first study that incorporates the observational constraints of future warming trends in the CMIP6 ensemble into the model selection process. Hereafter, we call the remaining 13 GCMs CMIP6* (Table 1). We will select a GCM subset that well covers the uncertainty ranges of the CMIP6* ensemble instead of the original CMIP6 ensemble. These pre-filtering processes reduce the maximum values of future global mean surface air temperature changes (2080−2099 minus 1995−2014) from 2.0°C, 3.3°C and 5.7°C to 1.9°C, 2.8°C and 4.9°C for SSP1-2.6, SSP2-4.5 and SSP5-8.5, respectively.  (i) For each season (s), region (a), SSP (e) and variable (v), we divide the max-min ranges of {X (i, s, a, e, v), i = 1, …, N} into M equal-length bins (Fig. 3a). (ii) We randomly sample X (i, s, a, e, v) of the M GCMs R times, which are denoted by Y ( j, r, s, a, e, v), where j = 1, …, M and r = 1, …, R. Here, R is 10,000. (iii) For each r, s, a, e and v, we define {Z (m, r, s, a, e, v), m = 1, …, M} as the numbers of {Y ( j, r, s, a, e, v), j = 1, …, M} that fall into the m-th bin of (i) (Fig. 3a). (iv) For each r and v, we define the unevenness, U (r, v), as the variance of {Z (m, r, s, a, e, v), m = 1, …, M, s = 1, …, 4, a = 1, 2, e = 1, …, 3}. Smaller U (r, v) values indicate that the subset of GCMs has a less biased distribution. (v) The mean unevenness U -(r) is defined as where w (v) = 1 for all the variables as mentioned above. The black solid lines of Fig. 3b show the maximum and minimum values of the 10,000 U -(r) values as a function of the selected number of models M. As M increases, the maxmin range of U -(r) shrinks.
(vi) We search subsets of the M GCMs whose U -(r) value is less than or equal to the 10th percentile values of the 10,000 randomly generated GCM subsets (the black dashed line of Fig. 3b) and that include both MRI-ESM2.0 and MIROC6. Because some daily variables are missing for CNRM-CM6-1-HR, we omit the subset that includes CNRM-CM6-1-HR. If M = 5, 97 candidates of subsets pass these criteria. (vii) For each candidate of the GCM subsets that pass criterion (vi), we compute the maximum value of R -* (i ) across the M GCMs, Max (R -* (i )). The subsets having the lowest value of Max (R -* (i )) are selected.
(viii) Finally, from the candidates of GCM subsets that pass criterion (vii), we select a subset whose M-GCM-total-value of R -* (i ) is the lowest.
The orange line of Fig. 3b indicates U -(r) for the selected GCM subsets as a function of M. Figure 3c shows the unevenness for each variable for the selected GCMs with M = 5. The unevenness value of hurs is greater than those of the other variables. For the selected GCM subsets, we investigate the fraction of the range captured (FRC) (McSweeney et al. 2016) for each variable: (a) we compute the ratio between the max-min range of the M selected GCMs and the max-min range of all the CMIP6* GCMs; (b) the ratios of (a) are averaged across all the combinations of region, season and SSPs. When M = 4, the FRC values of tas, tasmax and tasmin are larger than those of the other variables (Fig. 3d). The FRC values generally increase as the number of selected models increases. For M ≥ 5, the FRC values exceed 72% except for pr (67%) and hurs (60%).

Spreads of the selected subset of GCMs
Although M = 6 has a better FRC than M = 5, by consulting with the members of S-18 and CCARP, we determine that M is 5 due to limited research resources of impact assessments. Because it highly depends on the impact researchers how many GCMs they can use, we also determine the two priorities of GCMs to be used in S-18 and CCARP: Tier 1 is composed of MRI-ESM2.0 and MIROC6; Tier 2 is composed of ACCESS-CM2, IPSL-CM6A-LR and MPI-ESM1-2-HR (Table 1). Impact researchers can choose the tiers of simulations to be performed for themselves. Coincidently, MRI-ESM2.0, MIROC6, IPSL-CM6A-LR and MPI-ESM1-2-HR are 4 of 10 GCMs used in the 3rd phase of the ISIMIP (ISIMIP3b) (Lange 2020) (MIROC6 is Tier 2 of ISIMIP3b and the other three are Tier 1). Therefore, we can compare Japanese impact assessments in S-18/CCARP and world-scale impact assessments in ISIMIP3b for these GCMs. Because tas, tasmin and tasmax are temperature variables, we test whether the selected GCMs can change if w = 1/3 for these 3 variables and w = 1 for the other variables. In that case, ACCESS-CM2, INM-CM5-0 and MPI-ESM1-2-HR are selected as the Tier 2 GCMs. Figure 4a shows the global mean air temperature changes for SSP1-2.6 and SSP5-8.5. Although the global mean air temperature changes are not used for the model selection process, the 5 selected GCMs cover the distributions of CMIP6* well. Figures 4b−e indicate the four examples of climate changes averaged over west and east Japan (see Supplementary Figs. S1−S8 for the changes in all the variables). It is confirmed that the 5 selected GCMs are widely distributed and are not biased in most cases. Figures 5, 6 and S9−S11 compare the maximum, median and minimum values and 2× the standard deviation (2σ) at each grid between all the CMIP6* GCMs and the selected 5 GCMs for changes in some variables in SSP5-8.5 for example: tasmax for DJF (Fig. 5), pr for JJA (Fig. 6), rsds for DJF (Fig. S9), hurs for JJA (Fig. S10) and sfcWind for DJF (Fig. S11). The 5 selected GCMs reasonably represent the maximum, median and minimum change patterns and 2σ of these variables. Although the amplitudes of the negative pr anomalies over the ocean region south of Japan are different between Figs. 6c and 6g (the minimum change patterns of pr), these differences may not influence impact assessment studies of the land area.

Summary and discussion
We develop a novel method of selecting GCMs based on the CMIP6 ensemble as representative climate future scenarios (Whetton et al. 2012). This new method can be used for GCM selection processes for impact assessments in any countries and regions. Here we apply it for impact and adaptation studies of Japan. We first consider the recent literature on the overestimated warming issue of the CMIP6 ensemble (Tokarska et al. 2020;Liang et al. 2020;Nijsse et al. 2020) in the model selection process. ISIMIP3 could not take into account this issue in its model selection process (Lange 2020). We selected a subset of 5 GCMs that widely captures the uncertainty ranges for the 8 variables, while the S-8 dataset considered only the spreads of tas and pr (Hanasaki et al. 2014). We also take into account the performances of the present-climate simulations, while Shi2020 did not. Shi2020 and Cannon (2015) selected GCM subsets with the maximum distance between the GCMs. If there is a single outlier GCM, the GCMs at the other end tend to be frequently selected in the maximum distance methods. To avoid such biased distributions of the selected GCMs, we apply the equal-length binning.
While McSweeney et al. (2012) and Mendlik and Gobiet (2016) considered the spatial patterns of climate change in their model selections, we do not. Instead, we will investigate and report spatial patterns of climate change and atmospheric circulation changes in each selected GCM in future studies. (red), the Tier 2 (blue) and the other CMIP6* GCMs (black) for SSP1-2.6 (horizontal axis) and SSP5-8.5 (vertical axis). (b) Changes in tas for DJF of SSP1-2.6 (°C). The red, blue and black symbols indicate the Tier 1 (red), the Tier 2 (blue) and the other CMIP6* GCMs (black) for west (horizontal axis) and east Japan (vertical axis), respectively. (c−e) The same as panel (b) but for (c) pr of JJA in SSP5-8.5 (mm day −1 ), (d) rsds of MAM in SSP2-4.5 (W m −2 ) and (e) sfcWind of JJA in SSP5-8.5 (m s −1 ).