Systems  
Status displays
System status
Retired systems
 
 
 
 
 
 
 
 
 
 
 
 

MARS User Guide at NSC

/!\ This page is under major revision.


Introduction

Meteorological Archival and Retrieval System (MARS) is an archiving system for meteorological data developed at ECMWF and used at SMHI to archive both operational and research data. MARS accepts meteorological data in GRIB1, GRIB2 format and in principle BUFR and ODB. For further (somewhat outdated) reference, see the ECMWF documentation at ECMWF.

The SMHI MARS archive is accessible from all nodes on krypton, byvind and vagn after loading its module (see below). It will be soon available directly from SMHI computers through a proxy server.

Support

For support please use mars-support@nsc.liu.se.

Before using MARS

First make sure you have all MARS components loaded by

	module load mars/2.6

The latest version of the MARS client uses grib_api 1.11.0 to read/validate GRIB messages and emoslib 392 to manipulate/transform them (interpolate, crop, etc.). You also need to load the python module (currently 2.7.4-smhi1) which is used in various MARS subcomponents. If you did not load the grib_api, emos and python modules, the mars module will do it for you, making sure that the correct version of these libraries are used.

If it is the first time you are using MARS, you will need to generate a key (SSH key) to send to the MARS administrator when you apply for an account. This is described in the following section.

Getting an account

The first time you use one of the MARS components, a pair of SSH keys will be generated. The pair of keys is used for authentication on the MARS servers. Note that these are not the common SSH keys you may have in your ~/.ssh directory on krypton, but a separate pair used solely to contact MARS and stored in ~/.mars directory.

During the first attempt to contact MARS, you will get on-screen instructions on how to proceed to get an account on the MARS servers. Once you have an account, the next step is to create an "experiment id" (expid) to tag your data as being your data.

Note that if you have an account on the old mars1 and mars2 servers, you will have to go through the registration process again (new SSH keys will not be created but you will still have to contact us to get access to marsre and marsop.

Using MARS Operational production

On the server for operational data, marsop, one can only list or retrieve data, but not archive anything.

Available Operational Model

Currently, marsop contains the following datasets:

EXPVERDescription
C11AHirlam C11 11km operational run
C22AHirlam C22 22km operational run
E11AHirlam E11 11km operational run
E05AHirlam E05 5km operational run
G05AHirlam G05 5km operational run
H01AHiromb BS01 forecast operational run
H03AHiromb NS03 short forecast operational run
HL3AHiromb NS03 long forecast operational run

Retrieving

For general information on how to retrieve data from MARS read the ECMWF user guide: http://www.ecmwf.int/publications/manuals/mars/

A typical request to retrieve initial 2m temperatures at synoptic time 00/06/12/18 +00H for last summer (june-july-august 2013) would be:

RETRIEVE,
	CLASS    = OP,
	STREAM   = OPER,
	EXPVER   = C11A,
	MODEL    = HIRLAM,
	TYPE     = FC,
	DATE     = 20130601/TO/20130831/BY/1,
	TIME     = 00/06/12/18,
	STEP     = 0,
	LEVTYPE  = HL,
	LEVELIST = 2,
	PARAM    = 11.1,   
	TARGET   = "T2m.grb"
Save this request in a file, for instance T2m.retrieve, and submit it using the mars command:
[sm_sebvi@krypton ~]$ mars T2m.retrieve 
tun  - INFO   - 20131106.095047.317730 - Using SSH tunnel configuration within /tmp/mars.tunnel.3369.UYWh8A
mars - INFO   - 20131106.095047 - Welcome to MARS with grib_api
mars - INFO   - 20131106.095047 - MARS_HOME=/software/apps/mars_client/client/2.6/gcc446
mars - INFO   - 20131106.095047 - MARS Client version: 20110905
mars - INFO   - 20131106.095047 - EMOSLIB version: 392
mars - INFO   - 20131106.095047 - Using grib_api version 1.11.0
	RETRIEVE,
		CLASS    = OP,
		STREAM   = OPER,
		EXPVER   = C11A,
		MODEL    = HIRLAM,
		TYPE     = FC,
		DATE     = 20130601/TO/20130831/BY/1,
		TIME     = 00/06/12/18,
		STEP     = 0,
		LEVTYPE  = HL,
		LEVELIST = 2,
		PARAM    = 11.1,   
		TARGET   = "T2m.grb"

mars - INFO   - 20131106.095047 - Processing request 1

RETRIEVE,
    CLASS      = OP,
    TYPE       = FC,
    STREAM     = OPER,
    EXPVER     = c11a,
    MODEL      = HIRLAM,
    LEVTYPE    = HL,
    LEVELIST   = 2,
    PARAM      = 11.1,
    DATE       = 20130601/20130602/20130603/20130604/20130605/20130606/20130607/20130608/20130609/20130610/20130611/20130612/20130613/20130614/20130615/20130616/20130617/20130618/20130619/20130620/20130621/20130622/20130623/20130624/20130625/20130626/20130627/20130628/20130629/20130630/20130701/20130702/20130703/20130704/20130705/20130706/20130707/20130708/20130709/20130710/20130711/20130712/20130713/20130714/20130715/20130716/20130717/20130718/20130719/20130720/20130721/20130722/20130723/20130724/20130725/20130726/20130727/20130728/20130729/20130730/20130731/20130801/20130802/20130803/20130804/20130805/20130806/20130807/20130808/20130809/20130810/20130811/20130812/20130813/20130814/20130815/20130816/20130817/20130818/20130819/20130820/20130821/20130822/20130823/20130824/20130825/20130826/20130827/20130828/20130829/20130830/20130831,
    TIME       = 0000/0600/1200/1800,
    STEP       = 0,
    DOMAIN     = G,
    TARGET     = "T2m.grb",
    DATABASE   = marsop

mars - INFO   - 20131106.095047 - Requesting 368 fields
mars - INFO   - 20131106.095047 - Calling mars on 'localhost', callback on 4207
mars - INFO   - 20131106.095047 - Callback at address 127.0.0.1, port 4207
mars - INFO   - 20131106.095048 - Mars client is on localhost (127.0.0.1) 4207
mars - INFO   - 20131106.095048 - Mars server is on localhost (127.0.0.1) 54156
mars - INFO   - 20131106.095048 - Server task is 562 [marsop]
mars - INFO   - 20131106.095049 - Request cost: 368 fields, 111.882 Mbytes online, nodes:  [marsop]
mars - INFO   - 20131106.095050 - Transfering 117316914 bytes
mars - INFO   - 20131106.095053 - Request time:  wall: 6 sec 
mars - INFO   - 20131106.095053 -   Processing in marsop: wall: 3 sec 
mars - INFO   - 20131106.095053 -   Visiting marsop: wall: 5 sec 
mars - INFO   - 20131106.095053 -   Writing to target file: 111.89 Mbyte(s) in 2 sec  [70.15 Mbyte/sec]
sm_sebvi@krypton ~]$

Listing

It is possible to querry the MARS servers using a "LIST" request. For instance, one can ask if marsop contains 10 meters wind components u and v for the model C11, June 2013:

LIST,
    CLASS     = OP,
    MODEL     = HIRLAM,
    TYPE      = FC,
    STREAM    = OPER,
    EXPVER    = C11A,
    DATE      = 20130601/TO/20130630,
    TIME      = ALL,
    STEP      = ALL,
    LEVTYPE   = HL,
    LEVELIST  = 10,
    PARAM     = 33.1/34.1,
    TARGET    = "list.out"
Save this request in a file, for instance list.request and use submit it with the mars command:
[sm_sebvi@krypton ~]$ mars list.request 
tun  - INFO   - 20131106.095726.589504 - Using SSH tunnel configuration within /tmp/mars.tunnel.3369.UYWh8A
mars - INFO   - 20131106.095726 - Welcome to MARS with grib_api
mars - INFO   - 20131106.095726 - MARS_HOME=/software/apps/mars_client/client/2.6/gcc446
mars - INFO   - 20131106.095726 - MARS Client version: 20110905
mars - INFO   - 20131106.095726 - EMOSLIB version: 392
mars - INFO   - 20131106.095726 - Using grib_api version 1.11.0
LIST,
    CLASS     = OP,
    MODEL     = HIRLAM,
    TYPE      = FC,
    STREAM    = OPER,
    EXPVER    = C11A,
    DATE      = 20130601/TO/20130630,
    TIME      = ALL,
    STEP      = ALL,
    LEVTYPE   = HL,
    LEVELIST  = 10,
    PARAM     = 33.1/34.1,
    TARGET    = "list.out"

mars - INFO   - 20131106.095726 - Processing request 1

LIST,
    CLASS      = OP,
    TYPE       = FC,
    STREAM     = OPER,
    MODEL      = HIRLAM,
    EXPVER     = c11a,
    LEVTYPE    = HL,
    LEVELIST   = 10,
    DUPLICATES = KEEP,
    PARAM      = 33.1/34.1,
    DATE       = 20130601/20130602/20130603/20130604/20130605/20130606/20130607/20130608/20130609/20130610/20130611/20130612/20130613/20130614/20130615/20130616/20130617/20130618/20130619/20130620/20130621/20130622/20130623/20130624/20130625/20130626/20130627/20130628/20130629/20130630,
    TIME       = ALL,
    STEP       = ALL,
    HIDE       = missing,
    TARGET     = "list.out",
    OUTPUT     = TABLE,
    DATABASE   = marsop

mars - INFO   - 20131106.095726 - Calling mars on 'localhost', callback on 4207
mars - INFO   - 20131106.095726 - Callback at address 127.0.0.1, port 4207
mars - INFO   - 20131106.095726 - Mars client is on localhost (127.0.0.1) 4207
mars - INFO   - 20131106.095726 - Mars server is on localhost (127.0.0.1) 54550
mars - INFO   - 20131106.095726 - Server task is 562 [marsop]
mars - WARN   - 20131106.095744 - Closing connection with 1 byte outstanding
mars - INFO   - 20131106.095744 - Request performed on database 'marsop'
mars - INFO   - 20131106.095744 - Request time:  wall: 18 sec 
mars - INFO   - 20131106.095744 -   Processing in marsop: wall: 18 sec 
mars - INFO   - 20131106.095744 - No errors reported
[sm_sebvi@krypton ~]$
The list requests tend to output quite a lot of cryptic informations. It is usually a good idea to include the "HIDE" and "OUTPUT" keywords in the request:
LIST,
   CLASS     = OP,
   MODEL     = HIRLAM,
   TYPE      = FC,
   STREAM    = OPER,
   EXPVER    = C11A,
   DATE      = 20130601/TO/20130630,
   TIME      = ALL,
   STEP      = ALL,
   LEVTYPE   = HL,
   LEVELIST  = 10,
   PARAM     = 33.1/34.1,
   HIDE      = GRAND-TOTAL/CUBES/DATE_INTERVAL,
   OUTPUT    = TREE,
   TARGET    = "list.out"
the content of "list.out" indicates that the data is available:
[sm_sebvi@krypton ~]$ more list.out 
class=op,date=2013-06-01,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-02,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-03,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-04,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-05,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-06,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-07,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-08,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-09,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-10,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-11,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-12,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-13,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-14,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-15,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-16,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-17,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-18,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-19,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-20,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-21,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-22,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-23,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-24,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-25,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-26,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-27,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-28,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-29,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-30,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
[sm_sebvi@krypton ~]$

Using MARS Research

On the server containing research data, marsre, users are allowed to archive their personal experiments, provided it is in GRIB format. We cover here, how to get one or more experiments IDs to "tag" your data as yours and how to preprocess your data so MARS understands it.

Getting an experiment ID

Since MARS does not store any ownership details within its database, an experiment id (expid), specified in the request through the "EXPVER" keyword, is used to distinguish between the data owned by different users. This experiment id is exactly 4 characters long and may consist of any combination of the digits 0-9 and the ASCII character set a-z (or A-Z, MARS does not make a distinction between upper and lower case). The experiment id's are stored in a separate database, called expid database and are used to genereate MARS permissions on the fly to protect your data.

There are few things especially noteworthy regarding the experiment id's:

  • An experiment id is only unique to a certain MARS class (research or operational), therefore the MARS class must always be specified as a mandatory argument when creating an id. In practice, you will never create an id on the operational server, marsop, for safety reasons. So the MARS class will always be "re" for all regular users. This will change in the future if we decide to open a third MARS server, for special usage or datasets.
  • An allocated id is stored on a MARS server together with the user (you, called "owner"), with optionally other users with archiving permissions (called "writers", useful for collaborative project) and with a supplied (up to 43 character long) description of the id.
  • You can create as many ids as you like, you are not limited to only one. Ideally, you should create a new id for each project or dataset you are working on.
  • There are no restrictions/permissions regarding retrieving/listing your data. If you have an account on a MARS server, you can retrieve data from whatever id on that server. Your data will be retrievable by anyone having a MARS account.
  • An allocated id can not be updated/removed by any other than the owner of the id. A "writer" can not modify an id.
  • Removing an id is a two-stage process. First you "de-allocate" the id, then a MARS administrator physically remove the data and the id from the database. In between, you have the possibility to change your mind and "restore" that id (and get access back to your data). You can restore the id as long as you can see it listed preceded by the mention "DELETED".
  • If you are tempted to re-use one of your id because you no longer need its content, don't!!! In three years time, you will forget what belongs to which project in that id, and it is more complicated for a MARS administrator to selectively delete some of the data than wiping away the entire content of the id. If you really want to re-use an id, contact us for a fast delete.

Here are few examples on how to manage your expids:

  1. List configured MARS classes:
    expid --list--classes
  2. use the research id database in interactive mode:
    expid --class=re --interactive
  3. Allocate a new experiment id for the server marsre (research):
    expid --class=re --allocate -id A007 --description="This is James Bond's id"
  4. Update a previously allocated id to add writers:
    expid --class=re --update --id=A007 --writers=m,q,moneypenny
  5. Delete (de-allocate) a previously allocated id:
    expid --class=re --delete --id=A007
  6. Restore a previously deleted id (valid if not processed by the MARS administrator yet):
    expid --class=re --restore --id A007
  7. List all allocated id:s on marsop (operational):
    expid --class=op --list
  8. List all allocated id:s on marsre by the user bond:
    expid --class=re --list --user=bond

The expid command is available to you once you loaded the MARS module.

Preprocessing

Before archiving in marsre, you need to prepare your data so MARS understands it and decides where to keep it and how to organize it. To do so, you need to encode within the local section of your GRIB messages (octet 40-80 in section 1 for GRIB edition 1) a set of keys that MARS understands (CLASS, STREAM, TYPE, MODEL and EXPVER) and sometimes some extra ones (for the "MATCH" model for instance or for the stream "EXPR"). Here is a table with the MARS keywords that should be encoded in the GRIB local section:

EXPVERid as generated by expid, see previous section
CLASSalways "re" on marsre (research)
STREAMtype of data, usually "OPER" for marsop or "EXPR" for marsre
TYPEtype of integration, typically "AN" for analysis or "FC" for forecast
MODELtype of model: HIRLAM, HIROMB, AROME, SURFEX, etc...
EXPOFFSETonly needed if STREAM=EXPR, usually 0
TIMEREPRESonly used with MATCH model
SORTonly used with MATCH model
LANDTYPEonly used with MATCH model
AEROSOLBINNUMBERonly used with MATCH model

Most of the keywords are self-explained.
On marsre, you will always use CLASS=RE. Depending on the type of data you are archiving, you will have to choose the right STREAM. It is recommended that by default you use STREAM=EXPR, which is the intended default STREAM on marsre. This is typically for a one-time experiment over a limited range of time, typically one model run. When you use that STREAM, you should use it combined with the EXPOFFSET keyword. The offset allow you to store the same experiment a second, a third, a fourth, N times while changing few settings in your model. This way you get all your related experiments under the same expid, with expoffset=0 or 1 or 2, etc. This is ideal while you test several values of a parameter in your model.

adding local sections

The easiest way to add local sections in your GRIB files is to use grib_api:

	grib_set -ssetLocalDefinition=82,localDefinitionNumber=82,class=RE,expver=MYID,type=FC,model=HIRLAM,stream=EXPR,expoffset=0 input.grb output.grb
This command reads GRIB messages from the file input.grb and writes them in output.grb, after including the local definition 82 of local centre 82. In details, "setLocalDefinition=82" refers to SMHI (with WMO CCCC code being "eswi"), "localDefinitionNumber=82" refers specifically to local definition 82 (within the range of local section defined by SMHI), and the other keywords are pretty self-explaining.

In practice, "setLocalDefinition" is usually set to 82 (at least for now) and "localDefinitionNumber" can be either 82 (all models but MATCH) or 83 (MATCH model). In the future, we will probably create new local sections needed to describe other type of data.

Note that the whole procedure assumes that your GRIB messages are encoded with centre=82 exclusively (maybe in the future, metcoop and harmonie centre will be added) and not centre=96 (old coding used at SMHI for hirlam data) or centre=98 (ecmwf). To set "centre" to 82 before including the local section, use:

        grib_set -s centre=82 grib_without_centre_set_to_82.grb grib_with_centre_set_to_82.grb
However, you need to be very careful with what you are doing. For instance, suppose you have a GRIB message from ECMWF (centre=98) containing temperature (coded with ECMWF definitions indicatorOfParameter=130, table2Version=128). If you simply set centre=82 without changing indicatorOfParameter and table2Version to SMHI definitions for temperature, you create a very confusing and dangerous situation because parameter 130 of table 128 at SMHI is undefined. In the worst case scenario, the parameter could be defined but to something completely different!

organizing GRIB messages for efficient archiving

MARS sorts and organizes data logically for efficient retrieving and compact storage. A typical model output is usually organized in files per steps per synoptic hours because the model usually writes data to disk as soon as a specific step is finished. However, common retrieving patterns are "all temperatures for all the steps of a specific day or month", or "pressure levels over a period of time", or "precipitations for a specific season", not "give me all the GRIB messages of a specific step from a specific synoptic hours". For this reason, MARS groups your data in a different way than you have it in files. Previously, this was sorted out by a tool called "himar.x" but now it is replaced by a set of grib_api tools.

It is maybe complicated at first, but in practice, it is rather simple. MARS uses a tree-branches-leaves scheme to organize GRIB files. The MARS keywords can either be the nodes of the branches or axis within the leaves.

If a MARS keyword is used as a node of the MARS tree, that keyword must have a unique value during archiving KEYWORD=VALUE (but you have total freedom during listing, staging or retrieving). MARS keywords that are always used as node are CLASS, STREAM, EXPVER, MODEL, TYPE, and LEVTYPE:

    CLASS    = op,
    STREAM   = oper,
    EXPVER   = a007,
    MODEL    = hirlam,
    TYPE     = fc,
    LEVTYPE  = MSL,

If a MARS keyword is used within a MARS leaf (also called hypercube, because values of the leaf keywords become axis of a n-dimensional array or cube), several values of that keyword can be listed, separated by "/":

    TIME     = 00/06/12/18,   # could be shortened with 00/TO/18/BY/6
    STEP     = 00/TO/48/BY/3, # this is equivalent to /00/03/06/.../42/45/48
    LEVELIST = 01/TO/60/BY/1, # this equivalent to /01/02/.../59/60
    PARAM    = 11.1/91.1,     # no compact way to specify parameters
MARS keywords that are always hypercube axis are TIME, STEP, PARAM, and LEVELIST.

Currently, only the DATE keyword needs special attention. MARS uses the DATE keyword to extract the meta keywords YEAR, MONTH and DAY. If data is compacted by DAY, you can only archive one DATE at a time. If data is compacted by MONTH, you can archive several dates belonging to the same month. If data is compacted by YEAR, you can archive several dates belonging to the same year.

    DATE     = 20130903,             # only one date: September 3rd 2013
    ...
    DATE     = 20130901/TO/20130930, # one month: September 2013
    ...
    DATE     = 20130101/TO/20131231, # one year: 2013
How do you know if you can group data per DAY, MONTH or YEAR? Well there is no easy answer. A new tool will soon be available to do this for you.

If you decide to do things yourself, the easiest way to sort/group your GRIB messages is to use the grib_api tool called grib_filter:

    grib_filter filter_rules in_grib_file.grb in_grib_file.grb ....
grib_filter applies to each GRIB messages a set of rules described in an input file "filter_rules". Typical filter files can be found in /software/apps/mars_client/filter, covering the most common user cases. If you are dealing with unusual or more complex case, have a look at the official grib_api documentation or contact the mars support.

generating MARS archive requests

Once you have your GRIB files ready, you need to prepare your MARS requests. The easiest way to achieve this is to use the "grib2request" tool (provided by the mars module). grib2request does not actually produce a valid MARS request stricto sensu but will give you very good hints on what it should look like.

[sm_sebvi@krypton ~]$ grib2request 
Usage: grib2request -1 -f 
          -1 : print a single request
          -f : file to process
[sm_sebvi@krypton ~]$ grib2request -1 -f e4od_19951012_FC_ML.grb
GRIB,
    DOMAIN     = G,
    LEVTYPE    = ML,
    LEVELIST   = 1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60,
    DATE       = 19951012,
    TIME       = 0000/0600/1200/1800,
    STEP       = 3/6/9/12/18/24/30/36/42/48,
    PARAM      = 11.1/33.1/34.1/51.1/58.1/71.1/76.1/200.1/61.1/115.1/116.1,
    CLASS      = RE,
    TYPE       = FC,
    STREAM     = REAN,
    EXPVER     = e4od,
    MODEL      = HIRLAM

grib2request - INFO   - 20131106.092323 - Decoded 21720 GRIB messages
grib2request - INFO   - 20131106.092323 - Request time:  wall: 35 sec  cpu: 11 sec 
The "-1" option is almost always needed. the "-f" is used to specify which file to process. What is left for you to do is to change "GRIB" for "ARCHIVE", and add the "TARGET" keyword . Save this (remove of course the last two lines starting with "grib2request - INFO") in a file called for instance "archive.request"

Using a compact tool to add local definitions, sort GRIB messages and create archive requests.

not available yet.

Archiving

Once you have preprocessed your data, archiving is a pretty simple task:

	mars archive.request

Removing data

It is not possible for a user to remove data from the data himself.

Retrieving EURO4M data

The Euro4M Reanalysis datatset is currently stored on marsre. You can use the following keyword to retrieve it:

DATABASEMARSRE
EXPVERE4OD
CLASSRE
STREAMREAN
TYPEAN or FC
MODELHIRLAM
LEVTYPEHL,ML,PL,etc.
LEVELISTinteger, depends on LEVTYPE keyword
PARAM1.1,11.1,etc.
DATEfrom 19900101 to 20090930
TIME00, 06, 12 or 18
STEP 0 with TYPE=AN, 3,6,9,etc. with TYPE=FC
TARGETfilename to write your data in.

Have a look at keywords tables at the bottom of this page if you wonder what is HL, ML, PL, etc. Don't hesitate to contact us at mars-support@nsc.liu.se if you have any questions/requests regarding Euro4M dataset.

MARS from non-NSC sites (MISU, etc.)

For MARS to work the username supplied from the client must match that recorded on the server, meaning that from non-NSC sites you will have to supply your NSC username on the command line when the mars client is invoked.

Differences between MARS at ECMWF and SMHI/NSC

There are some syntax differences between ECMWF and SMHI/NSC on how a MARS request is written. These differences reflects the adaptation of the MARS system to SMHI needs in the form of new keywords or extra values for the existing keywords.

The main difference is probably the introduction of the MODEL keyword which does not exist at ECMWF but which is mandatory here at SMHI.

Leveltypes

At ECMWF all surface data are archived as LEVTYPE=SFC. At SMHI, we don't group surface data into one single leveltype but keep a one-to-one relationship between GRIB code for leveltype and their labels.

APPENDIX

User groups and corresponding mars servers

Server Database Identification User group
Marsop marsop CLASS=op Operational data
Marsre marsre CLASS=re Research data

MARS@NSC GRIB tables

CLASS

LABELGRIB CODEDescription
OP 80 SMHI operations
RE 81 SMHI research

STREAM

LABELGRIB CODEDescription
OPER 1025 Operational deterministic model daily archive
MNTH 1043 Monthly means
MODA 1071 Monthly means of daily means
MDFA 1075 Monthly means of daily forecast accumulations
DAME 4000 Daily means
SEME 4001 Seasonal means
SECY 4002 Seasonal daily cycle
EXPR 8200 Research Experiments
REAN 8299 Reanalysis Datasets (EURO4M)

MODEL

LABELGRIB CODEDescription
HIRLAM 30 NWP LAM
RCA 31 Climate model
HIROMB 40 Ocean model
MATCH 50 Atmospheric dispersion model
ALARO 20 ALARO part of Harmonie
SURFEX 21 SURFEX part of Harmonie
AROME 22 AROME part of Harmonie
IFS 10 Global NWP
MESAN 50 Mesoscale analysis system
SWAN 60 Wave model

LEVTYPE

ECMWFSMHIGRIB1 CODEDescription
SFCSURF 1SURFace level
SFCTATM 8Top of ATMosphere level
PL PL100Pressure Level
SFC MSL102Mean Sea Level
HMSL103Height above Mean Sea Level
HL HL105Height (above ground) Level
LHL106Layer between Height Level
ML ML109Model Level
LDL112Layer between Depth below Land surface
PT PT113Potential Temperature
PV PV117Potential Vorticity
DP DP160DePth (below sea) level
ND191North Direction tile
NED192North East Direction tile
ED193East Direction tile
SED194South East Direction tile
SD195South Direction tile
SWD196South West Direction tile
WD197West Direction tile
NWD198North West Direction tile
SFC ATM200entire ATMosphere level

TYPE

LABELGRIB CODEMeaning
AN 02 Analysis
3V 05 3DVAR analysis
4V 06 4DVAR analysis
FC 09 Forecast
SI 21 Climate simulation

GRIB for MATCH Model

For further information about MATCH conversions please contact Lennart Robertson at SMHI.

Author: Sebastien Villaume






Page last modified: 2014-02-11 14:23
For more information contact us at info@nsc.liu.se.