History

MeteoIO library was initially developed at SLF http://www.slf.ch/ in support to research activities, then, thanks to the HYDROSYS project http://www.hydrosysonline.eu/, it was further developed in collaboration with Thomas Egger and the EPFL-EFLUM team http://eflum.epfl.ch/index.en.php to be connected to GEOtop with the aim of providing an automatic tools to deal with a large amount of data and real time data (data cleaning, formatting and interpolation).

Features

MeteoIO library embedded in GEOtop enables the model to easily access a variety of input data formats (meteo time series and gridded data) and work with them at two different levels: handling meteorological data and performing hydrological simulations. Using MeteoIO thought GEOtop provides three several features:

  1. Reading of meteo data and station data from different sources (e.g. GEOtop _meteo files, GSN, A3D meteo files) with all necessary filtering upon that data (e.g. hourly accumulations, averaging, min/max filtering, etc)
  2. Using GEOtop in a real time mode, reading data from a Global Sensor Network (Currently implemented only for the study area of La Fouly, in the Swiss Valais; this is one test sites selected for HYDROSYS project).
  3. Computing morphological features, such as aspect, slope, curvature and drainage direction at runtime, as possible alternative to JGRASS Horton Machines (in this case the user is not required to provide these input files to GEOtop)
  4. Spatial interpolations for the meteorological parameters based on Liston et al., 2006.
  5. It provides the framework to create your own plugins, filter and interpolation algorithms and add them to the library.

In addition to proving a useful tool for handling data, this library aims to be a robust structure to which people can easily contribute by writing application specific algorithms.

How to

Before you get started on GEOtop-MeteoIO you need to download and install the MeteoIO library and the GSOAP library to connect to GSN (see below for details). MeteoIO can be downloaded from the following web page http://slfsmm.indefero.net/p/meteoio/page/Getting-started/ and compiles under Linux, OSX and Windows with gcc or mingw. You'll need to create your account if have none; then, once you are logged in, you can navigate to the meteoIO page and download the source code. To get MeteoIO you'll need to install an SVN client if you don't have one already installed.

A Mac version of GSOAP can be downloaded at http://gsoap.darwinports.com/ or http://www.macports.org/install.php (see Note below for details). The first link also provides clear instructions on how to install gsoap after the dmg is downloaded, the second provides the up-to-date download links for all platform.

Snow Leopard users might have troubles with the macports, alternatively they can download the version from http://www.cs.fsu.edu/~engelen/soap.html and compile/install it themselves.

MeteoIO is also designed for working with different Reference Systems; to enable this feature you need to install PROJ4 libraries. You can easily install it with macports using the following command: sudo port install proj

If you don't have cmake installed on your machines, you can download it at the following link:

Note for MacOS users

After having installed gSOAP you have to copy the file /opt/local/include/stdsoap2.h which has just been installed on your computer to your /usr/include folder. Alternatively you can create a symlink of stdsoap2.h file in /usr/include folder (sudo ln -s /opt/local/include/stdsoap2.h /usr/include/stdsoap2.h) or add the /opt/local/include/ path to your .profile file.

In order to compile MeteoIO you need to properly configure the makefile by selecting all the plugins you need. Navigate to the source directory and type "ccmake ." Then navigate the interface to change the configuration; when you are done, press c to configure and g to generate the makefile. Once you have configured your options and generated the makefile, just type make and later make install (on Linux and OSX with sudo or root rights) and the library with its plugins will be installed system wide.

Pay attention on the cmake version, some users experiences problem with version 2.8 using the arrows to navigate the interface. A work-around is to use Cntr n for moving to the next lines and Cntr p for moving back to the previous line.

The Figure below shows a screenshot of a sample configuration.


You can also automatically generate the documentation for meteoIO; you just need to have doxygen installed http://www.doxygen.nl/download.html and type make documentation from meteoio source folder. A html folder containing the documentation will be either created or updated, navigate to it and open the index.html file.

PLUGINS

Plugins are functionalities that can be added on demand to the main library. Available plugins are:

  • A3DIO enables MeteoIO reading Alpine3D meteo files (no extra requirements)
  • ARCIO enables MeteoIO reading ESRII/ARC DEM files (no extra requirements)
  • ARPSIO enables MeteoIO reading ARPS grid files (no extra requirements)

  • GEOtopIO enables MeteoIO reading GEOtop meteofiles and feeding GEOtop with other source of data such as GSN (no extra requirements)

  • GrassIO enables MeteoIO reading GRASS DEM file (no extra requirements)
  • GSNIO enables MeteoIO working over GSN - Global Sensor Network;(prerequisite is the gsoap++ library)
  • IMISIO enables MeteoIO reading IMIS data (require Oracle's OCCI library)

For further details see MeteoIO documentation.

NOTE on Global Sensor Network (GSN)

GSN is a software middleware designed to manage sensor networks and data real time acquisition. A comprehensive description can be found here http://sourceforge.net/apps/trac/gsn. Where a wireless sensor network is installed and data is delivered to a server in real time, GSN makes that data available for post processing and model applications. When using GEOtop with the GSN plugins, MeteoIO requires a library called gSOAP, to connect to web services. For more information on gSOAP http://www.cs.fsu.edu/~engelen/soap.html.


DOWNLOADING GEOtop with MeteoIO

Due to several projects in which GEOtop is involved, there are several versions with different functionalities at the moment. Developers are currently working on merging all versions in one single version, meantime a temporary branch called hydrosys has been created on the SNV. It contains the version currently enabled to work with MeteoIo. You can check out that folder which also contains a TESTGSN folder. This forder contains an io.ini file for the test simulation and an additional folder (with respect to the trunk version) called mymeteo; this folder is where GEOtop will output data read from GSN (if GSN plugin is enabled).

CONFIGURATION FILE: io.ini

To run GEOtop with MeteoIO you need to add one file to your simulation directory called "io.ini". This file holds all relevant information for MeteoIO on where to retrieve meteo and station data, which formats to output data into, how to deal with the filtering and which algorithms to use for spatial interpolations. Your io.ini file might look very similar to this one:

[General]

PLUGINPATH = ../../meteoio/trunk/lib

[Input]

COORDSYS = CH1903

DEM = ARC

DEMFILE = morpho/_dem.asc

GRID2D = GRASS

STATION = GSN

METEO = GSN

NROFSTATIONS = 6

STATION1 = lafouly_st_1033

STATION2 = lafouly_st_1036

STATION3 = lafouly_st_1037

STATION4 = lafouly_st_1040

STATION5 = lafouly_st_1042

STATION6 = lafouly_st_1043

#STATION = GEOTOP

#METEO = GEOTOP

#METEOPATH = meteo

#METEOPREFIX = _meteo

[OUTPUT]

METEO = GEOTOP

METEOPATH = mymeteo

METEOSEQ = WindS WindDir RelHum AirT SWglobal Iprec

[Filters]

HNW::filter1 = accumulate

HNW::arg1 = 3600

TA::filter1 = mean_avg

TA::arg1 = soft center 1 3600

RH::filter1 = mean_avg

RH::arg1 = soft center 1 3600

ISWR::filter1 = mean_avg

ISWR::arg1 = soft center 1 3600

VW::filter1 = wind_avg

VW::arg1 = soft center 1 3600

[Interpolations2D]

HNW::algorithms = idw

RH::algorithms = Rh idw

VW::algorithms = wind

DW::algorithms = idw

TA::algorithms = idw

TA::cst_lapse = -0.0078

The io.ini file is divided into 5 sections in this example, namely [Input], [Output], [Filters] and [Interpolations2D].

GENERAL

The general section contains the path to MeteoIO plugins, GEOtop plugins, GRASS plugins etc... which are needed to read different types of data sources and loaded at run time. It is optional for MacOS and Linux OS, but it's mandatory for Windows.

INPUT

The input section deals with specifying where MeteoIO can retrieve meteo data, station metadata, DEM (Digital Elevation Model), other 2D grids (such as drainage directions, gradient, etc...) and which coordinate system to use. The keys, DEM, GRID2D, STATION and METEO tell MeteoIO which plugin to load when a request for a DEM, a 2D grid (here in ARC format), station metadata or meteo data is issued (in the example the GEOtop plugin will be invoked). Each plugin can have different keys that need to be configured for it to run properly; e.g. the ARC plugin needs to know the name of the DEMFILE, the GEOTOP plugin needs to know the METEOPATH and the METEOPREFIX for the files in METEOPATH, the GSN plugin needs the number of stations and their name on the GSN server.

COORDSYS

As to the key COORDSYS, available reference systems are:

When UTM and PROJ4 keywords are used, an additional COORDPARAM keyword needs to be specified as follows.

E.g.

COORDIN = UTM

COORDPARAM = 32N

COORDSYS = PROJ4

COORDPARAM = +init=epsg:21781

COORDSYS = PROJ4

COORDPARAM = +proj=utm +ellps=WGS84 +zone=10

STATION/METEO

These keywords provide MeteoIO with the source of data; if set to GSN, then MeteoIO retrieves data through the GSN server, if set to GEOTOP, MeteoIO retrieves data through GEOtop meteo files (manually generated) or files previously generated by running GEOtop over GSN. If GSN is used, the user has to specify the number of stations used for the simulation (6 in the example) and the name of the stations, as they are on GSN. If GEOTOP is used, the user has to specify the PATH to the folder containing meteo files and the meteo prefix which is usually _meteo.

For further details see MeteoIO documentation.

OUTPUT

Similarly the Output section configures how MeteoIO deals with outputting data. In this instance the GEOtop plugin is used and it requires to know the path to output the data retrieved from GSN and formatted according to GEOtop standard (the sequence of parameters specifies how to format and fill the file).

FILTERS

The Filters section deals with filtering the meteo data that is read from a given data source. This data can come in timesteps of 2 minutes, or 2 hours; since GEOtop and other models mostly operate on discrete timesteps there needs to be some sort of accumulation, averaging and data cleaning performed before handing the data over to the model. This section deals with configuring these filters (for details please check out the documentation for MeteoIO). In the above example HNW (height new water == precipitation) is cumulated over 3600 seconds and several other parameters are averaged over 3600 seconds. The measured wind speed and direction are dealt with a special filter called "wind_avg" that averages wind speed and direction with a vector based approach http://www.webmet.com/met_monitoring/622.html. Filters usually come with an argument (arg1) e.g.

HNW::arg1 = 3600

It indicates the time interval in minutes over which precipitation is cumulated.

RH::filter1 = mean_avg

RH::arg1 = soft center 1 3600

It indicates that Relative Humidity is a centered average within the interval HH:01 HH:60=H+1. The soft keyword means that if a center average is required, but there are no enough values to the right of the center (i.e. after your measurement) then the time window is allowed to be shifted to the left.

Available filters are:

  • rate: calculates the derivative at the current point and reject the point if the derivative is above a given threshold.
  • min_max: filter all values outside of the interval [min, max], soft operation supported
  • min: filter out all values less than the given parameter, soft operation supported
  • max: filter out all values greater than the given parameter, soft operation supported
  • resample: resamples (if necessary) the data so that if a required time stamp is between two data points, an interpolated value will be returned (instead of nodata).
  • accumulate: accumulates the data on a given interval (Seconds).
  • median_avg: computes median average over a given window
  • mean_avg: computes mean average over a given window
  • wind_avg: vector average over a given window

INTERPOLATION2D

The Interpolations2D section deals with the spatial interpolations. Different algorithms have been implemented. The user may specify any number of algorithms for a parameter (in the example air temperature TA has four algorithms configured) and at runtime the best one will be chosen. Please turn to the documentation of MeteoIO for further details. Note: in order to use spatial interpolations in connection with MeteoIO this section needs to be configured!

Available interpolation algorithms are:

VARIABLE NAMES

TA Air temperature [Celsius]

VW Wind velocity [m s-1]

DW: Wind direction [degrees]

RH: Relative humidity (0 - 1)

HNW: Precipitations [mm h-1]

ISWR: Incoming shortwave radiation [W m-2]

RSWR: Reflected Short Wave Radiation [W m-2]

ILWR: Long wave radiation [W m-2]

TGS: Soil or snow bottom temperature [Kelvin]

TSS: Soil or snow surface temperature [Kelvin]

HS: Snow height [cm]

P: Atmospheric pressure [Pa]

COMPILING and RUN GEOtop with MeteoIO

Once you have compiled the version of GEOtop that offers MeteoIO support and configured the io.ini file in the base directory of your GEOtop simulation, you can run GEOtop with MeteoIO by invoking GEOtop with the command line option

./geotop ./ --meteoio

Also you might want to specify an enddate for your simulation:

./geotop ./ --meteoio --enddate=20090714T0100

Now GEOtop will fetch the meteo data through MeteoIO and return the data to the model after executing the filters. Depending on which version of GEOtop MeteoIO you are using the spatial interpolations are also performed through MeteoIO (bypassing the inbuilt micromet solution of GEOtop).