1A. Compute Climatologies
Model output and observations must be converted into annual climatologies for use with the mean climate driver. This has already been done for the observations provided by PCMDI. PMP provides a script for generating these input climatologies from other data. For more information about using the script, see the documentation. The basic use is shown here:
python pcmdi_compute_climatologies.py \
-p clim_calc_cmip_inparam.py --options
Examples of parameter files can be found under sample_setups.
Five netcdf files are produced by this script: an annual climatology (AC) and seasonal climatologies (DJF, MAM, JJA, SON).
[1]:
# To open and display one of the graphics
from IPython.display import display_png, Image
Passing parameters via parameter file
First, load custom demo directories:
[2]:
from user_choices import demo_data_directory, demo_output_directory
The parameter file for this demo is shown here:
[3]:
with open("basic_annual_cycle_param.py") as f:
print(f.read())
import os
#
# OPTIONS ARE SET BY USER IN THIS FILE AS INDICATED BELOW BY:
#
#
# VARIABLES TO USE
vars = ['rlut']
# START AND END DATES FOR CLIMATOLOGY
start = '2003-01'
end = '2018-12'
# INPUT DATASET - CAN BE MODEL OR OBSERVATIONS
infile = 'demo_data_tmp/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc'
# DIRECTORY WHERE TO PUT RESULTS
outfile = 'demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc'
In this example, all of the parameters are set in the parameter file shown above. The climatology script is then run from the command line with the parameter file as the sole input.
[4]:
%%bash
pcmdi_compute_climatologies.py -p basic_annual_cycle_param.py
start and end are 2003-01 2018-12
variable list: ['rlut']
ver: v20241111
var: rlut
infile: demo_data_tmp/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
outfile: demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc
outfilename: None
outpath: None
ver: v20241111
infilename: rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
type(d): <class 'xarray.core.dataset.Dataset'>
atts: {'Conventions': 'CF-1.7 ODS-2.1', 'activity_id': 'obs4MIPs', 'contact': 'PCMDI (pcmdi-metrics@llnl.gov)', 'creation_date': '2021-07-27T18:20:26Z', 'curation_provenance': 'work-in-progress', 'data_specs_version': '2.1.0', 'external_variables': 'areacella', 'frequency': 'mon', 'further_info_url': 'https://furtherinfo.es-doc.org/CMIP6.NASA-LaRC.CERES-EBAF-4-1.experiment_idsub_experiment_idPCMDI', 'grid': '1x1 degree latitude x longitude', 'grid_label': 'gn', 'history': '2021-07-27T18:20:26Z; CMOR rewrote data to be consistent with obs4MIPs, and CF-1.7 ODS-2.1 standards', 'institution': 'NASA-LaRC (Langley Research Center) Hampton, Va', 'institution_id': 'NASA-LaRC', 'mip_era': 'CMIP6', 'nominal_resolution': '100 km', 'product': 'observations', 'realm': 'atmos', 'release_year': '2019', 'source': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_description': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_id': 'CERES-EBAF-4-1', 'source_label': 'CERES-EBAF-4-1', 'source_name': 'CERES-EBAF4-1', 'source_type': 'satellite_blended', 'source_version_number': '4.1', 'table_id': 'obs4MIPs_Amon', 'table_info': 'Creation Date:(18 November 2020) MD5:d8a4a72de798e86a999881bdaeb1809e', 'title': 'CERES prepared for obs4MIPs (ODS-v2.1.0)', 'tracking_id': 'hdl:21.14102/d1afa116-4801-4801-b302-43c49b5bb7dc', 'variable_id': 'rlut', 'variant_info': 'Best Estimate', 'variant_label': 'PCMDI', 'license': 'Data in this file processed for obs4MIPs by PCMDI and is for research purposes only.', 'cmor_version': '3.6.1'}
outdir: demo_output_tmp/climo
start_yr_str is 2003
start_mo_str is 01
end_yr_str is 2018
end_mo_str is 12
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20241111.nc
Passing parameters via command line
The following example uses the command line to pass arguments rather than a parameter file. Bash cell magic is used to pass the directory names (referenced as $1 and $2) and run the command as a subprocess.
Similar to the mean climate metrics, filenames can use chain notation to substitute variables into the file name. In this case, the %(variable) placeholder shows the climatology script where to substitute the variable name into the file name. This functionality is useful for model data where there are separate timeseries files to read in for each variable.
[5]:
%%bash -s "$demo_data_directory" "$demo_output_directory"
pcmdi_compute_climatologies.py \
--var rlut \
--start 2003-01 --end 2018-12 \
--outfile $2'/climo/%(variable)_mon_CERES-EBAF-4-1_BE_gn.nc' \
--infile $1/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
start and end are 2003-01 2018-12
variable list: ['rlut']
ver: v20241111
/Users/lee1043/mambaforge/envs/pmp_devel_20241106_xcdat0.7.3/lib/python3.10/site-packages/pcmdi_metrics/io/string_constructor.py:43: UserWarning: Keyword 'variable' not provided for filling the template.
warnings.warn(f"Keyword '{k}' not provided for filling the template.")
var: rlut
infile: demo_data_tmp/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
outfile: demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc
outfilename: None
outpath: None
ver: v20241111
infilename: rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
type(d): <class 'xarray.core.dataset.Dataset'>
atts: {'Conventions': 'CF-1.7 ODS-2.1', 'activity_id': 'obs4MIPs', 'contact': 'PCMDI (pcmdi-metrics@llnl.gov)', 'creation_date': '2021-07-27T18:20:26Z', 'curation_provenance': 'work-in-progress', 'data_specs_version': '2.1.0', 'external_variables': 'areacella', 'frequency': 'mon', 'further_info_url': 'https://furtherinfo.es-doc.org/CMIP6.NASA-LaRC.CERES-EBAF-4-1.experiment_idsub_experiment_idPCMDI', 'grid': '1x1 degree latitude x longitude', 'grid_label': 'gn', 'history': '2021-07-27T18:20:26Z; CMOR rewrote data to be consistent with obs4MIPs, and CF-1.7 ODS-2.1 standards', 'institution': 'NASA-LaRC (Langley Research Center) Hampton, Va', 'institution_id': 'NASA-LaRC', 'mip_era': 'CMIP6', 'nominal_resolution': '100 km', 'product': 'observations', 'realm': 'atmos', 'release_year': '2019', 'source': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_description': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_id': 'CERES-EBAF-4-1', 'source_label': 'CERES-EBAF-4-1', 'source_name': 'CERES-EBAF4-1', 'source_type': 'satellite_blended', 'source_version_number': '4.1', 'table_id': 'obs4MIPs_Amon', 'table_info': 'Creation Date:(18 November 2020) MD5:d8a4a72de798e86a999881bdaeb1809e', 'title': 'CERES prepared for obs4MIPs (ODS-v2.1.0)', 'tracking_id': 'hdl:21.14102/d1afa116-4801-4801-b302-43c49b5bb7dc', 'variable_id': 'rlut', 'variant_info': 'Best Estimate', 'variant_label': 'PCMDI', 'license': 'Data in this file processed for obs4MIPs by PCMDI and is for research purposes only.', 'cmor_version': '3.6.1'}
outdir: demo_output_tmp/climo
start_yr_str is 2003
start_mo_str is 01
end_yr_str is 2018
end_mo_str is 12
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20241111.nc
[6]:
image_path = "demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20241111.png"
img = Image(image_path)
display_png(img)
Mixing inputs and specifying the output directory
Parameters that are specified on the command line override those in the parameter file when both are provided. In this case the output directory and file name are specified separately on the command line using the variables outpath and outfilename. This functionality is usefuly for creating batch climatologies scripts.
[7]:
%%bash -s "$demo_data_directory" "$demo_output_directory"
pcmdi_compute_climatologies.py \
-p basic_annual_cycle_param.py \
--outpath $2/climo/ \
--outfilename rlut_mon_CERES-EBAF-4-1_BE_gn.nc \
start and end are 2003-01 2018-12
variable list: ['rlut']
ver: v20241111
var: rlut
infile: demo_data_tmp/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
outfile: demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc
outfilename: rlut_mon_CERES-EBAF-4-1_BE_gn.nc
outpath: demo_output_tmp/climo/
ver: v20241111
infilename: rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
type(d): <class 'xarray.core.dataset.Dataset'>
atts: {'Conventions': 'CF-1.7 ODS-2.1', 'activity_id': 'obs4MIPs', 'contact': 'PCMDI (pcmdi-metrics@llnl.gov)', 'creation_date': '2021-07-27T18:20:26Z', 'curation_provenance': 'work-in-progress', 'data_specs_version': '2.1.0', 'external_variables': 'areacella', 'frequency': 'mon', 'further_info_url': 'https://furtherinfo.es-doc.org/CMIP6.NASA-LaRC.CERES-EBAF-4-1.experiment_idsub_experiment_idPCMDI', 'grid': '1x1 degree latitude x longitude', 'grid_label': 'gn', 'history': '2021-07-27T18:20:26Z; CMOR rewrote data to be consistent with obs4MIPs, and CF-1.7 ODS-2.1 standards', 'institution': 'NASA-LaRC (Langley Research Center) Hampton, Va', 'institution_id': 'NASA-LaRC', 'mip_era': 'CMIP6', 'nominal_resolution': '100 km', 'product': 'observations', 'realm': 'atmos', 'release_year': '2019', 'source': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_description': 'CERES EBAF (Energy Balanced and Filled) TOA Fluxes. Monthly Averages', 'source_id': 'CERES-EBAF-4-1', 'source_label': 'CERES-EBAF-4-1', 'source_name': 'CERES-EBAF4-1', 'source_type': 'satellite_blended', 'source_version_number': '4.1', 'table_id': 'obs4MIPs_Amon', 'table_info': 'Creation Date:(18 November 2020) MD5:d8a4a72de798e86a999881bdaeb1809e', 'title': 'CERES prepared for obs4MIPs (ODS-v2.1.0)', 'tracking_id': 'hdl:21.14102/d1afa116-4801-4801-b302-43c49b5bb7dc', 'variable_id': 'rlut', 'variant_info': 'Best Estimate', 'variant_label': 'PCMDI', 'license': 'Data in this file processed for obs4MIPs by PCMDI and is for research purposes only.', 'cmor_version': '3.6.1'}
outdir: demo_output_tmp/climo/
start_yr_str is 2003
start_mo_str is 01
end_yr_str is 2018
end_mo_str is 12
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20241111.nc
output file is demo_output_tmp/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20241111.nc