0. Prepare Demos

Download sample data and set up parameter files

This Notebook sets up the data for the other demos. It is recommended that you run this and other demos in a conda environment with xcdat and pcmdi_metrics installed. The files to download are listed in https://pcmdi.llnl.gov/pss/pmpdata/pmp_tutorial_files.v20240201.txt.

Note: All observational datasets used in the PMP analysis are published through the ESGF obs4MIPs project. In addition, several derived climatological products—while not currently available via ESGF—can be accessed directly via download.

To obtain the reference climatologies used in the PMP mean climate metrics, download the list of datasets from: https://pcmdi.llnl.gov/pss/pmpdata/PMP_obs4MIPsClims_v20250228.txt

Instructions for downloading and using these reference climatologies are provided in Demo notebook 1b.

[1]:

# Lets get the file containing the data needed for this demo
import requests
r = requests.get("https://pcmdi.llnl.gov/pss/pmpdata/pmp_tutorial_files.v20240201.txt")
with open("data_files.txt","wb") as f:
    f.write(r.content)

Now let’s retrieve these sample files locally. Please edit the following to match a good location on your system:

demo_data_directory is a path where you would like to download our demo data

demo_output_directory is a path where you would like the demos output to be dumped.

The last cell in this notebook automatically sets up the parameter files for the other demos to use your chosen demo_data and demo_output directories.

[2]:

# This is where you will be downloading the sample_data
demo_data_directory = "demo_data"
# this line is where your output will be stored
demo_output_directory = "demo_output"

In below cell, you are going to download sample input files from PCMDI server. The total size of dataset is about 2 GB, and so please be aware that downloading will take some time to complete.

[3]:

# Let's download the files
from pcmdi_metrics.io.base import download_sample_data_files
try:
    download_sample_data_files("data_files.txt", demo_data_directory)
    print("All files downloaded")
except Exception:
    print("Download failed")

/global/homes/l/lee1043/.conda/envs/pmp_devel_20250305/lib/python3.10/site-packages/esmpy/interface/loadESMF.py:94: VersionWarning: ESMF installation version 8.8.0, ESMPy version 8.8.0b0
  warnings.warn("ESMF installation version {}, ESMPy version {}".format(

Downloading: 'CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.pr.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.pr.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.rlut.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.rlut.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.zg.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.ACCESS1-0.r1i1p1.mon.zg.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.pr.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.pr.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.rlut.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.rlut.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.zg.198101-200512.AC.v20200426.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip5.historical.CanCM4.r1i1p1.mon.zg.198101-200512.AC.v20200426.nc
Downloading: 'CMIP5_demo_clims/cmip6.historical.MCM-UA-1-0.r1i1p1f1.mon.zg.198101-200512.AC.v20201119.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_clims/cmip6.historical.MCM-UA-1-0.r1i1p1f1.mon.zg.198101-200512.AC.v20201119.nc
Downloading: 'CMIP5_demo_data/cmip5.amip.ACCESS1-0.sftlf.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_data/cmip5.amip.ACCESS1-0.sftlf.nc
Downloading: 'CMIP5_demo_data/cmip5.historical.GISS-E2-H.sftlf.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_data/cmip5.historical.GISS-E2-H.sftlf.nc
Downloading: 'CMIP5_demo_timeseries/historical/atmos/day/pr/pr_day_GISS-E2-H_historical_r6i1p1_20000101-20051231.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/CMIP5_demo_timeseries/historical/atmos/day/pr/pr_day_GISS-E2-H_historical_r6i1p1_20000101-20051231.nc
Downloading: 'obs4MIPs_PCMDI_clims/rlut/CERES-EBAF-4-0/v20210804/rlut_mon_CERES-EBAF-4-0_PCMDI_gn.200301-201812.AC.v20210804.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_clims/rlut/CERES-EBAF-4-0/v20210804/rlut_mon_CERES-EBAF-4-0_PCMDI_gn.200301-201812.AC.v20210804.nc
Downloading: 'obs4MIPs_PCMDI_clims/rlut/CERES-EBAF-4-1/v20210804/rlut_mon_CERES-EBAF-4-1_PCMDI_gn.200301-201812.AC.v20210804.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_clims/rlut/CERES-EBAF-4-1/v20210804/rlut_mon_CERES-EBAF-4-1_PCMDI_gn.200301-201812.AC.v20210804.nc
Downloading: 'obs4MIPs_PCMDI_clims/pr/GPCP-2-3/v20210804/pr_mon_GPCP-2-3_PCMDI_gn.200301-201812.AC.v20210804.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_clims/pr/GPCP-2-3/v20210804/pr_mon_GPCP-2-3_PCMDI_gn.200301-201812.AC.v20210804.nc
Downloading: 'obs4MIPs_PCMDI_clims/zg/ERA-INT/v20210804/zg_mon_ERA-INT_PCMDI_gn.200301-201812.AC.v20210804.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_clims/zg/ERA-INT/v20210804/zg_mon_ERA-INT_PCMDI_gn.200301-201812.AC.v20210804.nc
Downloading: 'obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc
Downloading: 'obs4MIPs_PCMDI_daily/NASA-JPL/GPCP-1-3/day/pr/gn/latest/pr_day_GPCP-1-3_PCMDI_gn_19961002-20170101.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/obs4MIPs_PCMDI_daily/NASA-JPL/GPCP-1-3/day/pr/gn/latest/pr_day_GPCP-1-3_PCMDI_gn_19961002-20170101.nc
Downloading: 'misc_demo_data/ocn/ice_conc_nh_ease2-250_cdr-v3p0_198801-202012.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/misc_demo_data/ocn/ice_conc_nh_ease2-250_cdr-v3p0_198801-202012.nc
Downloading: 'misc_demo_data/ocn/ice_conc_sh_ease2-250_cdr-v3p0_198801-202012.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/misc_demo_data/ocn/ice_conc_sh_ease2-250_cdr-v3p0_198801-202012.nc
Downloading: 'misc_demo_data/atm/3hr/pr/pr_3hr_IPSL-CM5A-LR_historical_r1i1p1_5x5_1997-1999.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/misc_demo_data/atm/3hr/pr/pr_3hr_IPSL-CM5A-LR_historical_r1i1p1_5x5_1997-1999.nc
Downloading: 'misc_demo_data/fx/sftlf.GPCP-IP.1x1.nc' from 'https://pcmdiweb.llnl.gov/pss/pmpdata/' in: demo_data/misc_demo_data/fx/sftlf.GPCP-IP.1x1.nc
All files downloaded

The final step is to generate parameter files from templates, using the demo_data_directory and demo_output_directory variables set above.

[4]:

from download_sample_data import generate_parameter_files
generate_parameter_files(demo_data_directory, demo_output_directory)

Preparing parameter file: basic_diurnal_fourier.py
Preparing parameter file: basic_annual_cycle_param.py
Preparing parameter file: basic_monsoon_wang_param.py
Preparing parameter file: basic_enso_param.py
Preparing parameter file: basic_mjo_param.py
Preparing parameter file: basic_monsoon_sperber_param.py
Preparing parameter file: basic_diurnal_compute_daily_mean.py
Preparing parameter file: basic_precip_variability_param.py
Preparing parameter file: basic_diurnal_std_daily_mean.py
Preparing parameter file: basic_diurnal_fourierAllGrid.py
Preparing parameter file: basic_param.py
Preparing parameter file: basic_extremes_param.py
Preparing parameter file: basic_diurnal_std_hourly_mean.py
Preparing parameter file: basic_mov_param.py
Preparing parameter file: basic_diurnal_composite.py
Preparing parameter file: basic_mov_param_sst.py
Saving User Choices

You’re done! Please proceed to the next tutorial