Installation instructions
Jblob on DKRZ's mistral system
Jblob is already installed on mistral and available as a module, just execute 'module load jblob'.
PrerequisitesJblob needs an installed Java Runtime Environment (either JRE or JDK of version 6 or above).
Please note: GCJ is NOT supported! If you install a Sun JDK/JRE in addition to a pre-installed gcj-based java distribution make sure to change the global JAVA_HOME environment variable to the new setting.
You can download the Java Runtime Environment at
http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Installation
- Download the program archive linked above. This file contains all files neccessary for use with Windows and Unix.
- Extract the contents of the archive to the desired installation directory.
-
Unix/Linux: Edit the lines containing 'JBLOB_HOME' and 'JAVA_HOME' in the jblob script.
Windows: Edit the lines containing 'JBLOB_HOME' and 'JAVA_HOME' in the jblob.bat batch script.
Changes for use with HTTP-Proxies
This is only necessary if your internet connection requires use of a proxy!The java options "
-Dhttp.proxyHost" and "
-Dhttp.proxyPort" can be used to enable jblob connections over http-proxies.
For unix/linux systems add the following options to the last line of the
jblob script (before the '
de.dkrz.cera.applications.JblobClient "$@"' part)
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
For windows systems add the following options to the last line of the
jblob.bat batch script (before the '
de.dkrz.cera.applications.JblobClient %*' part
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
Synopsis
|
| jblob |
--dataset | -d |
dataset_acronym |
[ options ] |
|
| jblob |
--cmip5file |
drs_name |
[ options ] |
|
| jblob |
--help |
|
|
|
| jblob |
--version |
|
|
|
Options
| --cdo |
cdo operator |
CDO processing during data download. See section "CDO processing" for details. |
| --config |
file |
Read username and password information from file instead of ~/.netrc.
|
| --dateext |
|
Append timestamp(s) to filename instead of record number(s). Works only for datasets with a "calendrical" temporal structure. |
| --dir |
directory |
The transferred dataset will be placed in the specified directory. |
| --file |
filename |
The transferred dataset will be given the specified name instead of datasetname_rmin-rmax.
|
| --origin |
LTA filename |
Transfer a single file from a dataset containing DOKU/LTA data. |
| --password |
password |
Use password for database login. Works only in conjunction with --username.
|
| --quiet |
|
Quiet operation. No output except error messages. |
| --rmax |
number |
Last record to transfer. If omitted, the transfer will end with the last available record. |
| --rmin |
number |
First record to transfer. If omitted, the transfer will start with the first available record. |
| --tmax |
date |
Last date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data). |
| --tmin |
date |
First date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data). |
| --username |
username |
Use username for database login. Works only in conjuction with --password.
|
Supplying username and password
By default the program tries to read the username and password information from the ~/.netrc file. To utilize this you must add an entry for machine 'cera' to your .netrc file (i.e.
machine cera login your_username password your_password
). Alternatively you can specify a configuration file from where the login information will be read by using the --config option. This configuration file must follow the same syntax as the .netrc file. As a last resort you can provide username and password with the --username and --password options.
Exit values
| 0 |
Success |
|
| 1 |
Unspecified error |
Please check program output for additional messages (i.e. do not use the -quiet option). |
| 3 |
Invalid dataset name |
The requested dataset name is doesn't exist in the database. |
| 4 |
Invalid dataset id |
The requested dataset id doesn't exist in the database. |
| 5 |
Insufficient permission to dataset |
You do not have the neccessary permission to access the requested dataset. Please contact
for further assistance.
|
| 8 |
I/O error |
An I/O error was encountered during the data transfer. |
| 12 |
Temporal structure not supported for tmin/tmax selection |
The selected dataset cannot be downloaded by selecting start and end dates, please use selection by record numbers (--rmax, --rmin) instead.
|
| 13 |
Selected start/end record/time outside of datasets coverage. |
The supplied rmin/rmax/tmin/tmax value(s) are outside of the dataset's coverage. |
| 14 |
Dataset is empty (data_size = 0) |
Please contact
for further information.
|
CDO Processing
The following
CDO (1.6.9) operators are currently available for use with Jblob:
| sellonlatbox |
--cdo "sellonlatbox,<min_lon>,<max_lon>,<min_lat>,<max_lat>" |
| selcode |
--cdo "selcode,<code>" |
| selgridname |
--cdo "selgridname,<grid_name>" |
CDO processing is usable with these dataset formats: GRIB, NetCDF, IEG
Please note: processing may take some time, please be patient and DO NOT start the same download again if the first one doesn't start downloading immediately!
Examples
- Transfer all records of dataset EH4OPYC_SRES_A2_WIND10 to /tmp, reading username and password from ~/.netrc.
jblob --dataset EH4OPYC_SRES_A2_WIND10 --dir /tmp
- Transfer data from Jan. 2001 to Dec. 2010 of dataset EH5_OM_A1B_1_MM_TSURF, reading username and password from command line.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --tmin 2001-01 --tmax 2010-12 --username foo --password bar
- Download a single file from a LTA dataset.
jblob --dataset DKRZ_LTA_564_ds00001 --origin "DMS-paper/dms-paper-all-data-for-fig.tar"
- Download CMIP5 data by drs_name.
jblob --cmip5file "cmip5/output1/MPI-M/MPI-ESM-MR/amip/day/atmos/day/r1i1p1/v20120330/clt/clt_day_MPI-ESM-MR_amip_r1i1p1_19790101-20081231.nc"
- Select a specific region of a dataset for download.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --cdo "sellonlatbox,0,180,-45,45"
Known Issues
- Contents of DOKU/LTA datasets currently can only be downloaded separately due to large filesizes.