Jblob Documentation

Download

Important: Please read the Terms of use document available here!

Installation instructions

Jblob on DKRZ's mistral system Jblob is already installed on mistral and available as a module, just execute 'module load jblob'.

Prerequisites

Jblob needs an installed Java Runtime Environment (either JRE or JDK of version 8 or above).

Please note: GCJ is NOT supported! If you install an Oracle JDK/JRE or OpenJDK in addition to a pre-installed gcj-based java distribution make sure to change the global JAVA_HOME environment variable to the new setting. You can download the Java Runtime Environment at https://www.oracle.com/java/technologies/downloads/.

Installation

Download the program archive linked above. This file contains all files neccessary for use with Windows and Unix. Extract the contents of the archive to the desired installation directory. Unix/Linux: Edit the lines containing JBLOB_HOME and JAVA_HOME in the jblob script. Windows: Edit the lines containing JBLOB_HOME and JAVA_HOME in the jblob.bat batch script.

Changes for use with HTTP-Proxies This is only necessary if your internet connection requires use of a proxy!

The java options -Dhttp.proxyHost and -Dhttp.proxyPort can be used to enable jblob connections over http-proxies.

For unix/linux systems add the following options to the last line of the jblob script (before the de.dkrz.cera.applications.JblobClient "$@" part)

-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>

For windows systems add the following options to the last line of the jblob.bat batch script (before the de.dkrz.cera.applications.JblobClient %* part

-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>

Synopsis


  jblob --dataset | -d dataset_acronym [ options ]

  jblob --cmip5file drs_name [ options ]

  jblob --help

  jblob --version

Options


`--cdo`	`cdo operator`	CDO processing during data download. See section "CDO processing" for details.
`--config`	`file`	Read username and password information from file instead of ~/.netrc.
`--dateext`		Append timestamp(s) to filename instead of record number(s). Works only for datasets with a "calendrical" temporal structure.
`--dir`	`directory`	The transferred dataset will be placed in the specified directory.
`--file`	`filename`	The transferred dataset will be given the specified name instead of datasetname_rmin-rmax.
`--origin`	`LTA filename`	Transfer a single file from a dataset containing DOKU/LTA data.
`--password`	`password`	Use password for database login. Works only in conjunction with --username.
`--quiet`		Quiet operation. No output except error messages.
`--rmax`	`number`	Last record to transfer. If omitted, the transfer will end with the last available record.
`--rmin`	`number`	First record to transfer. If omitted, the transfer will start with the first available record.
`--tmax`	`date`	Last date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data).
`--tmin`	`date`	First date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data).
`--username`	`username`	Use username for database login. Works only in conjuction with --password.

Supplying username and password

By default the program tries to read the username and password information from the ~/.netrc file. To utilize this you must add an entry for machine 'cera' to your .netrc file (i.e. machine cera login your_username password your_password ). Alternatively you can specify a configuration file from where the login information will be read by using the --config option. This configuration file must follow the same syntax as the .netrc file. As a last resort you can provide username and password with the --username and --password options.

Exit values


0	Success
1	Unspecified error	Please check program output for additional messages (i.e. do not use the -quiet option).
3	Invalid dataset name	The requested dataset name is doesn't exist in the database.
4	Invalid dataset id	The requested dataset id doesn't exist in the database.
5	Insufficient permission to dataset	You do not have the neccessary permission to access the requested dataset. Please contact data@nullwdc-climate.de for further assistance.
8	I/O error	An I/O error was encountered during the data transfer.
12	Temporal structure not supported for tmin/tmax selection	The selected dataset cannot be downloaded by selecting start and end dates, please use selection by record numbers (`--rmax`, `--rmin`) instead.
13	Selected start/end record/time outside of datasets coverage.	The supplied rmin/rmax/tmin/tmax value(s) are outside of the dataset's coverage.
14	Dataset is empty (data_size = 0)	Please contact data@nullwdc-climate.de for further information.

CDO Processing

The following CDO (1.6.9) operators are currently available for use with Jblob:


sellonlatbox	`--cdo "sellonlatbox,<min_lon>,<max_lon>,<min_lat>,<max_lat>"`
selcode	`--cdo "selcode,<code>"`
selgridname	`--cdo "selgridname,<grid_name>"`

CDO processing is usable with these dataset formats: GRIB, NetCDF, IEG

Please note: processing may take some time, please be patient and DO NOT start the same download again if the first one doesn't start downloading immediately!

Examples

Transfer all records of dataset EH4OPYC_SRES_A2_WIND10 to /tmp, reading username and password from ~/.netrc.

jblob --dataset EH4OPYC_SRES_A2_WIND10 --dir /tmp

Transfer data from Jan. 2001 to Dec. 2010 of dataset EH5_OM_A1B_1_MM_TSURF, reading username and password from command line.

jblob --dataset EH5_OM_A1B_1_MM_TSURF --tmin 2001-01 --tmax 2010-12 --username foo --password bar

Download a single file from a LTA dataset.

jblob --dataset DKRZ_LTA_564_ds00001 --origin "/doku/bm0564/DKRZ_LTA_564_ds00001/DMS-paper/dms-paper-all-data-for-fig.tar"

Download CMIP5 data by drs_name.

jblob --cmip5file "cmip5/output1/MPI-M/MPI-ESM-MR/amip/day/atmos/day/r1i1p1/v20120330/clt/clt_day_MPI-ESM-MR_amip_r1i1p1_19790101-20081231.nc"

Select a specific region of a dataset for download.

jblob --dataset EH5_OM_A1B_1_MM_TSURF --cdo "sellonlatbox,0,180,-45,45"

Known Issues

Contents of DOKU/LTA datasets currently can only be downloaded separately due to large filesizes.