Installation instructions
Jblob on DKRZ's mistral system
Jblob is already installed on mistral and available as a module, just execute
'module load jblob'.
PrerequisitesJblob needs an installed Java Runtime
Environment (either JRE or JDK of version 6 or above).
Please note:
GCJ is NOT supported! If you install a Sun
JDK/JRE in addition to a pre-installed gcj-based java distribution make sure
to change the global JAVA_HOME environment variable to the new setting.
You
can download the Java Runtime Environment at
http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Installation
-
Download the program archive linked above. This file contains all files
neccessary for use with Windows and Unix.
-
Extract the contents of the archive to the desired installation directory.
-
Unix/Linux: Edit the lines containing 'JBLOB_HOME' and 'JAVA_HOME' in the
jblob script.
Windows: Edit the lines containing
'JBLOB_HOME' and 'JAVA_HOME' in the jblob.bat batch script.
Changes for use with HTTP-Proxies
This is only necessary if your internet connection requires use of a
proxy!The java options "
-Dhttp.proxyHost" and
"
-Dhttp.proxyPort" can be used to enable jblob connections over
http-proxies.
For unix/linux systems add the following options to
the last line of the
jblob script (before the '
de.dkrz.cera.applications.JblobClient "$@"' part)
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
For windows systems add the following options to the last line of
the
jblob.bat batch script (before the '
de.dkrz.cera.applications.JblobClient %*' part
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
Synopsis
|
| jblob |
--dataset | -d |
dataset_acronym |
[ options ] |
|
| jblob |
--cmip5file |
drs_name |
[ options ] |
|
| jblob |
--help |
|
|
|
| jblob |
--version |
|
|
|
Options
| --cdo |
cdo operator |
CDO processing during data download. See section "CDO processing" for
details.
|
| --config |
file |
Read username and password information from file instead of
~/.netrc.
|
| --dateext |
|
Append timestamp(s) to filename instead of record number(s). Works
only for datasets with a "calendrical" temporal structure.
|
| --dir |
directory |
The transferred dataset will be placed in the specified directory.
|
| --file |
filename |
The transferred dataset will be given the specified name instead of
datasetname_rmin-rmax.
|
| --origin |
LTA filename |
Transfer a single file from a dataset containing DOKU/LTA data.
|
| --password |
password |
Use password for database login. Works only in conjunction with
--username.
|
| --quiet |
|
Quiet operation. No output except error messages. |
| --rmax |
number |
Last record to transfer. If omitted, the transfer will end with the
last available record.
|
| --rmin |
number |
First record to transfer. If omitted, the transfer will start with the
first available record.
|
| --tmax |
date |
Last date to transfer. The standard date format is yyyy-mm-ddThh:mm,
however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient
when working with monthly data).
|
| --tmin |
date |
First date to transfer. The standard date format is yyyy-mm-ddThh:mm,
however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient
when working with monthly data).
|
| --username |
username |
Use username for database login. Works only in conjuction with
--password.
|
Supplying username and password
By default the program tries to read the username and password information
from the ~/.netrc file. To utilize this you must add an entry for machine
'cera' to your .netrc file (i.e.
machine cera login your_username password your_password
). Alternatively you can specify a configuration file from where the login
information will be read by using the --config option. This configuration file
must follow the same syntax as the .netrc file. As a last resort you can
provide username and password with the --username and --password options.
Exit values
| 0 |
Success |
|
| 1 |
Unspecified error |
Please check program output for additional messages (i.e. do not use
the -quiet option).
|
| 3 |
Invalid dataset name |
The requested dataset name is doesn't exist in the database.
|
| 4 |
Invalid dataset id |
The requested dataset id doesn't exist in the database. |
| 5 |
Insufficient permission to dataset |
You do not have the neccessary permission to access the requested
dataset. Please contact
for further assistance.
|
| 8 |
I/O error |
An I/O error was encountered during the data transfer. |
| 12 |
Temporal structure not supported for tmin/tmax selection |
The selected dataset cannot be downloaded by selecting start and end
dates, please use selection by record numbers (--rmax,
--rmin) instead.
|
| 13 |
Selected start/end record/time outside of datasets coverage.
|
The supplied rmin/rmax/tmin/tmax value(s) are outside of the dataset's
coverage.
|
| 14 |
Dataset is empty (data_size = 0) |
Please contact
for further information.
|
CDO Processing
The following
CDO (1.6.9) operators are
currently available for use with Jblob:
| sellonlatbox |
--cdo
"sellonlatbox,<min_lon>,<max_lon>,<min_lat>,<max_lat>"
|
| selcode |
--cdo "selcode,<code>" |
| selgridname |
--cdo "selgridname,<grid_name>" |
CDO processing is usable with these dataset formats: GRIB, NetCDF, IEG
Please note: processing may take some time, please be patient and DO NOT
start the same download again if the first one doesn't start downloading
immediately!
Examples
-
Transfer all records of dataset EH4OPYC_SRES_A2_WIND10 to /tmp, reading
username and password from ~/.netrc.
jblob --dataset EH4OPYC_SRES_A2_WIND10 --dir /tmp
-
Transfer data from Jan. 2001 to Dec. 2010 of dataset
EH5_OM_A1B_1_MM_TSURF, reading username and password from command
line.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --tmin 2001-01 --tmax 2010-12
--username foo --password bar
-
Download a single file from a LTA dataset.
jblob --dataset DKRZ_LTA_564_ds00001 --origin
"DMS-paper/dms-paper-all-data-for-fig.tar"
-
Download CMIP5 data by drs_name.
jblob --cmip5file
"cmip5/output1/MPI-M/MPI-ESM-MR/amip/day/atmos/day/r1i1p1/v20120330/clt/clt_day_MPI-ESM-MR_amip_r1i1p1_19790101-20081231.nc"
-
Select a specific region of a dataset for download.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --cdo
"sellonlatbox,0,180,-45,45"
Known Issues
-
Contents of DOKU/LTA datasets currently can only be downloaded separately
due to large filesizes.