Jblob Version 2 Documentation
Download
Important: Please read the Terms of use document available here!
Download Jblob V3.0
Installation instructions
Jblob on DKRZ's mistral system
Jblob is already installed on mistral and available as a module, just execute 'module load jblob'.
Prerequisites
Jblob needs an installed Java Runtime Environment (either JRE or JDK of version 8 or above).
Please note: GCJ is NOT supported! If you install an Oracle JDK/JRE or OpenJDK in addition to a pre-installed gcj-based java distribution make sure to change the global JAVA_HOME environment variable to the new setting. You can download the Java Runtime Environment at http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Installation
Download the program archive linked above. This file contains all files neccessary for use with Windows and Unix.
Extract the contents of the archive to the desired installation directory.
Unix/Linux: Edit the lines containing
JBLOB_HOME
and JAVA_HOME
in the jblob script.
Windows: Edit the lines containing JBLOB_HOME
and JAVA_HOME
in the jblob.bat batch script.Changes for use with HTTP-Proxies
This is only necessary if your internet connection requires use of a proxy!
The java options
-Dhttp.proxyHost
and -Dhttp.proxyPort
can be used to enable jblob connections over http-proxies.For unix/linux systems add the following options to the last line of the jblob script (before the
de.dkrz.cera.applications.JblobClient "$@"
part)-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
For windows systems add the following options to the last line of the jblob.bat batch script (before the
de.dkrz.cera.applications.JblobClient %*
part-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
Synopsis
jblob --dataset | -d dataset_acronym [ options ]
jblob --cmip5file drs_name [ options ]
jblob --help
jblob --version
Options
--cdo | cdo operator | CDO processing during data download. See section "CDO processing" for details. |
--config | file | Read username and password information from file instead of ~/.netrc. |
--dateext | Append timestamp(s) to filename instead of record number(s). Works only for datasets with a "calendrical" temporal structure. | |
--dir | directory | The transferred dataset will be placed in the specified directory. |
--file | filename | The transferred dataset will be given the specified name instead of datasetname_rmin-rmax. |
--origin | LTA filename | Transfer a single file from a dataset containing DOKU/LTA data. |
--password | password | Use password for database login. Works only in conjunction with --username. |
--quiet | Quiet operation. No output except error messages. | |
--rmax | number | Last record to transfer. If omitted, the transfer will end with the last available record. |
--rmin | number | First record to transfer. If omitted, the transfer will start with the first available record. |
--tmax | date | Last date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data). |
--tmin | date | First date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data). |
--username | username | Use username for database login. Works only in conjuction with --password. |
Supplying username and password
By default the program tries to read the username and password information from the ~/.netrc file. To utilize this you must add an entry for machine 'cera' to your .netrc file (i.e. machine cera login your_username password your_password ). Alternatively you can specify a configuration file from where the login information will be read by using the --config option. This configuration file must follow the same syntax as the .netrc file. As a last resort you can provide username and password with the --username and --password options.
Exit values
0 | Success | |
1 | Unspecified error | Please check program output for additional messages (i.e. do not use the -quiet option). |
3 | Invalid dataset name | The requested dataset name is doesn't exist in the database. |
4 | Invalid dataset id | The requested dataset id doesn't exist in the database. |
5 | Insufficient permission to dataset | You do not have the neccessary permission to access the requested dataset. Please contact data@nulldkrz.de for further assistance. |
8 | I/O error | An I/O error was encountered during the data transfer. |
12 | Temporal structure not supported for tmin/tmax selection | The selected dataset cannot be downloaded by selecting start and end dates, please use selection by record numbers (--rmax , --rmin ) instead. |
13 | Selected start/end record/time outside of datasets coverage. | The supplied rmin/rmax/tmin/tmax value(s) are outside of the dataset's coverage. |
14 | Dataset is empty (data_size = 0) | Please contact data@nulldkrz.de for further information. |
CDO Processing
The following CDO (1.6.9) operators are currently available for use with Jblob:
sellonlatbox | --cdo "sellonlatbox,<min_lon>,<max_lon>,<min_lat>,<max_lat>" |
selcode | --cdo "selcode,<code>" |
selgridname | --cdo "selgridname,<grid_name>" |
CDO processing is usable with these dataset formats: GRIB, NetCDF, IEG
Please note: processing may take some time, please be patient and DO NOT start the same download again if the first one doesn't start downloading immediately!
Examples
Transfer all records of dataset EH4OPYC_SRES_A2_WIND10 to /tmp, reading username and password from ~/.netrc.
jblob --dataset EH4OPYC_SRES_A2_WIND10 --dir /tmp
Transfer data from Jan. 2001 to Dec. 2010 of dataset EH5_OM_A1B_1_MM_TSURF, reading username and password from command line.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --tmin 2001-01 --tmax 2010-12 --username foo --password bar
Download a single file from a LTA dataset.
jblob --dataset DKRZ_LTA_564_ds00001 --origin "/doku/bm0564/DKRZ_LTA_564_ds00001/DMS-paper/dms-paper-all-data-for-fig.tar"
Download CMIP5 data by drs_name.
jblob --cmip5file "cmip5/output1/MPI-M/MPI-ESM-MR/amip/day/atmos/day/r1i1p1/v20120330/clt/clt_day_MPI-ESM-MR_amip_r1i1p1_19790101-20081231.nc"
Select a specific region of a dataset for download.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --cdo "sellonlatbox,0,180,-45,45"
Known Issues
Contents of DOKU/LTA datasets currently can only be downloaded separately due to large filesizes.