1. Introduction
Monitoring the global distribution of clouds and their optical and thermal properties is a key task for space-based Earth observation systems, taking into account that cloud descriptions and cloud feedback processes in climate models are still considered to be major sources of uncertainties in most recent climate predictions [
1,
2,
3,
4,
5]. Satellite-based observations with global coverage and with a quality permitting cloud detection as well as analysis of fundamental cloud properties were introduced by the end of the 1970s. These imaging sensors were multispectral, meaning that they measured in both visible and infrared spectral bands. They consisted mainly of the Advanced Very High Resolution Radiometer (AVHRR, [
6]), carried by polar orbiting satellites, and several visible and infrared spinning radiometers onboard geostationary satellites (all sensor characteristics given in detail by [
7]). Since then, several upgrades of sensor content and sensor characteristics, and satellite platforms have been made. Despite this, it is still possible to construct the original set of radiance measurements from today’s satellites as from the first versions of these visible and infrared sensors. Thus, long time series with homogeneous observations from the same type of measurements can be constructed.
As a consequence of the increasing length of this measurement record (currently 40+ years), the value of these observations for climate monitoring applications and for climate change studies are steadily increasing. This has led to the compilation of several global cloud climate data records (CDRs) throughout the years where various cloud properties and their changes over time have been examined. The pioneering CDR here is the one from the International Satellite Cloud Climatology Project (ISCCP, [
8]); a project that started already in 1982. Their first data records were provided in 1990 [
9] followed by the release of the first official climate data record (ISSCP-D) in 1999 [
10]. The ISCCP CDR has become a well-established reference for climate analysis [
11] and for evaluation of climate model simulations of clouds through the use of advanced satellite dataset simulator tools [
12,
13].
Although the combination of polar and geostationary satellite data in the ISCCP CDR enables a global long-term data record with high temporal resolution, the desired homogeneous coverage in space and time is not easy to accomplish. Large data gaps exist, especially in the early part of the covered period of investigation, and some regions are observed with larger viewing angles than other regions which introduces some artefacts in the results [
14]. Furthermore, as only one visible and one infrared channel from the 5-channel AVHRR instrument are used in ISCCP, the potential for characterization of microphysical and optical cloud properties is limited. This has led to the compilation of alternative global cloud CDRs based entirely on global AVHRR data. Although the temporal resolution is degraded compared to ISCCP (at least at low latitudes), the additional use of one shortwave infrared channel at 3.7 microns and one infrared channel at 12 microns enables an improved determination of parameters like cloud phase and cloud effective radius. It also improves the detection of thin cirrus clouds and their distinction from thick cirrus clouds [
15,
16]. Two examples of such CDRs are the Pathfinder Atmosphere Extended (PATMOS-x, [
17]) and the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF) Cloud, Albedo and surface Radiation dataset from AVHRR data (first edition denoted CLARA-A1, [
18]).
The different CDR approaches have been evaluated and inter-compared with each other, as well as with other shorter-term datasets in connection with the Global Energy and Water Cycle Experiment (GEWEX) Radiation Panel cloud assessment effort in 2012 [
19]. The study analyzed the spread in the results between the different methods and the global and regional performance of cloudiness and cloud top property retrievals. However, methods have been revised considerably since 2012 after some significant development efforts and especially after utilizing new high-quality reference measurements from active sensors in space. One example is the release of a second improved version of the CLARA CDR (CLARA-A2, [
20]) for which the cloudiness results were improved and examined in detail [
21] by reference to high-quality cloud observations from the Cloud-Aerosol LiDAR and Infrared Pathfinder Satellite Observation (CALIPSO) satellite [
22]. In parallel, a development of a third AVHRR-based CDR has been carried out within the framework of the European Space Agency (ESA) Climate Change Initiative (CCI) program [
23]. This development, with a concept denoted the Community Cloud retrieval for Climate (CC4CL), applied a different methodology in comparison to previous approaches [
24]. Finally, an important development since the GEWEX Radiation Panel evaluation in 2012 is a revision of the ISCCP methodology resulting in the release of the ISCCP-H CDR [
25]. The revision not only included algorithm upgrades, but also a significantly improved spatial resolution.
Remarkable efforts in method development have been noticed during the last years as well as the evolution of independent satellite-based observation datasets with near-climate (decadal-scale) observation capability [
26,
27,
28]. The latter facilitates the evaluation of CDRs which otherwise is difficult using the quite unstable and inhomogeneous (both temporally and spatially) ground-based observation network. Especially the reduction of the surface station network, as well as changes in observation methods (i.e., from manual to automatic observations) over the period of investigation causes problems since these changes are not evenly distributed geographically.
The purpose of this paper is to examine the two most recently upgraded versions of cloud CDRs from the ESA Cloud CCI project and from the International Satellite Cloud Climatology Project (ISCPP, [
25]), both being released in 2017–2018. References will be made to the other mentioned upgraded CDRs and to independent satellite observations from the CALIPSO satellite. Regarding the latter reference, the comparison will also be complemented with Level 2 studies (i.e., comparisons with simultaneously matched observations) for one of the CDRs; the ESA Cloud CCI CC4CL CDR. This was done in order to assess the impact of sampling or other issues affecting the compilation of Level 3 products (averages). The study is limited to the two fundamental cloud parameters Cloud Fraction (CF) and Cloud Top Pressure (CTP) which are provided by all inter-compared CDRs.
Section 2 introduces the two core CDRs under study as well as the referenced CDRs including the observations from the CALIPSO satellite. In addition, the methods of inter-comparing the CDRs are described. Results are given in
Section 3 for both Level 3 and Level 2 versions of the data records. Finally, results are discussed in
Section 4 followed by conclusions in
Section 5.
5. Conclusions
The purpose of this study was to investigate the performance of four available cloud CDRs with special attention to how very recent updates of the ESA Cloud CCI and ISCCP CDRs compared to already established CDRs.
Results for the globally averaged total cloud fraction show that there is in general large agreement between the investigated CDRs, especially from 1995 and onwards. They also all show a weak decreasing trend in global cloudiness although the magnitude differs among them with ISCCP-HGM showing the largest trend (−1.9% per decade) and CLARA-A2 the smallest trend (−0.5% per decade). Despite the overall good agreement, there are interesting differences between the three purely AVHRR-based methods concerning how to deal with orbital drift effects. PATMOS-x shows generally increasing cloud amounts with increasing orbital drift while the other two methods show no or only small changes in cloud amounts. We suggest that these differences are caused by method differences in how to handle the resulting changes in illumination and surface temperature conditions rather than being linked to effects caused by the diurnal cycle in cloudiness which should otherwise have affected all methods in the same way.
The agreement is most pronounced in the tropical region and in the northern mid-latitude region. Remarkable differences are seen over the southern mid-latitude region and over the polar regions. All CDRs show larger cloud fractions than the CALIPSO-CALIOP reference over the southern mid-latitude region which at first sight could appear surprising considering the higher sensitivity of the CALIOP measurement. However, in areas with high cloud amounts (like the southern mid-latitude region) the viewing geometry differences between the nadir looking CALIOP sensor and passive imagery (operating over a wide range of viewing angles) becomes more evident than over other areas. Higher viewing angles lead to higher interpreted cloud amounts for the passive instruments. The polar regions continue to be challenging for all CDRs and cloud detection results vary substantially here. Polar summer and early autumn cloud amounts appear to be reasonably well depicted by most CDRs over the North Pole region when comparing to the CALIPSO-CALIOP reference. However, ISCCP-HGM cloud amounts are clearly too low. Polar winter results differ remarkably between the south and north poles. Over the North Pole all methods underestimate cloud amounts except ISCCP-HGM (producing almost the same cloud amounts over the year) but over the South Pole the pure AVHRR-based methods show diverging results. PATMOS-x shows here a peak in cloudiness in the polar winter while the other two CDRs generally show minimum values. Best agreement with the CALIPSO-CALIOP reference is found for PATMOS-x.
The overall best agreement (seen over all regions) for the total cloud cover with the CALIPSO-CALIOP reference is consequently found for PATMOS-x, mostly because of its good agreement over the poles. However, this conclusion can be questioned after noting that PATMOS-x cloud amounts are often equal or even exceeding the CALIPSO-CALIOP reference values if excluding the tropical region. It is also remarkable to find so much better results for PATMOS-x compared to the others in the polar winter season over Antarctica where traditionally most methods based on passive imagery have always had large difficulties in separating clouds from the cold surface. Further studies are needed here to confirm that this is based on a real ability to separate the cloudy and cloud-free signature or whether it is a sign of having fitted the data too tight to the overall cloud climatology as provided by the training dataset from CALIPSO-CALIOP.
A detailed inter-comparison of simultaneous cloud observations from ESA Cloud CCI V3 and CALIPSO-CALIOP revealed very good agreement (hit rate of 82.9% and Kuipers skill score of 0.699) between the two which is clearly better than what has been reported in previous similar studies. It shows that it is possible to achieve cloud masking results from ANN-based methods which are superior to results from more traditional methods. Considering the rapid development of machine learning methods, this finding could largely influence future strategies for compiling satellite-based CDRs.
ISCCP-HGM cloud amounts agree reasonable well with the other CDRs over all areas except over the polar regions where the existing seasonal variability is not captured at all. The inter-comparison of cloud top pressure results revealed fundamentally different results for the exclusively AVHRR-based CDRs and the combined geostationary and polar CDR in ISCCP-HGM. The latter showed overall about 60 hPa lower global mean cloud top pressures than the others. In addition, ISCCP-HGM shows a negative trend of 5 hPa per decade over the period as opposed to the others showing positive trends ranging from 2–5 hPa. Further studies are needed to understand this different behavior of cloud top retrievals between the CDRs. Otherwise, it seems that ISCCP-HGM results do not deviate much from the previous version of the ISCCP CDR.