GoForMet-Climatology (G4M-Climatology)

The G4M-Climatology includes two components presented below:

• Web-Screens: a friendly unique Graphical User Interface (GUI).
• CDMS: a Climatological Database Management System fully compliant with WMO Resolution 40 (Cg-XII) and Resolution 25 (Cg-XIII),

Web Screens

It is the unique Graphical User Interface (GUI) of the system.

A characteristic of this module is a fine granularity at a functional level which allows users to create custom menus and group different features within their own needs.

Being o fully web-based interface, each specific functionality is presented as a different page, together with all the contextual tools the user could need to perform different operations (overlay information, change palettes and representations, make annotations, etc.).

Main Functionalities

Display of standard climatological products

Each product from the list below can be obtained from a dedicated page using specific graphical panels. Users can fully configure the request and the layout for data presentation in a very simple way using the graphical bespoke interface. Contextual help is available on almost all panels of the application allowing users to find immediately the desired information without being forced to search for it in the documentation (available in an online format as well).

• Climatological tables & summaries

• Climatological reports
• Time-series
• Area maps

• Wind roses

• Climatological charts

• Statistics
• Custom products

There is the possibility of putting together several products on a page for analysis or dissemination purposes. This goal can be achieved using the Workspace functionality.

Manually data ingestion

Few customized graphical panels are available for manual data ingestion. Basic data consistency checks and basic quality control are performed on the fly not to allow wrong data to be introduced into the database.

Data saving and export

• Different climatological data can be retrieved by users by requests made from user-friendly panels. All the data retrieved can be saved/exported in graphical formats (charts, maps, etc.) as raster or vector files and tabular format (CSV, excel tables).

Climatological Database Management System

The CDMS is the main Climatological Database Management System module of GoForMet. It is responsible for specific climatological functionality as listed below, being interconnected with a few other GoForMet modules as follows:

The Web-GUI (Web-based Graphical User Interface) for:

• users, roles, and permissions management
• data visualization and analysis
• climatological product elaboration
• system setup and monitoring

The COMET for:

• data ingestion from both real-time and archived data
• data normalization
• level one of data Quality Control

The Storage System for:

• store the data and metadata according to the approved policies
• maintaining the Data Catalogue
• providing the Data Access layer for the Storage System
• data exchanges

The communication between different modules is realized using REST-secured API

Main Functionalities

Data ingestion using its configurable data flows

• From the Operational Database (pull interface)
• From archives (pull & push interfaces) either manually using the Web-GUI module or automatically from other existent databases.
• Data Quality Control (QC) - level 2
• Data consistency

Climatological computations

• The CDMS server performs the climatological data analysis using a series of techniques, including statistical, spatial, and image analysis to generate a wide range of derived data and products.

Task scheduler

The authorized users can configure and schedule using The Web Screens UI panels the operational tasks as follows:

• Computation of derived data
• Compute CLIMAT and CLIMAT TEMP messages encoded in both TAC and TDCF format (BUFR)
• Compute yearly record reports
• Compute monthly aerodrome climatology summary (A-E)
• Compute ETCCDI Climate Change Indices (only after the climatological time-series data has been homogenized)

Data homogenization

The CDMS server embeds homogenization tools to qualify climate trends. These tools follow WMO 1245 recommendations and HOMER (COST) specifications.

These tools can be used by authorized users, and after the process is terminated, the climatological computations could be re-started from the new homogenized dataset.

• Long range forecast
• Seasonal forecast downscaling

Climatological definitions setup

According to WMO specifications for the Climate Data Management System, there are currently many inconsistencies observed between climatological data from different providers related to the basic concepts for:

• the generation of hourly observation data from observations data recorded at a minute frequency
• the climatological day definitions (some use 9 a.m. to 9 a.m., others 12 a.m. to 12 a.m., and so on)
• how daylight saving is treated
• how the standard normals are computed (using WMO standard period 1961-1990, or others)
• The customer can set up the system to use his climatological definitions instead the default ones by the dedicated panels from the Web-GUI module.
• System logging

Data Policy

The GoForMet system facilitates users to comply with WMO Resolution 40 (Cg-XII) and Resolution 25 (Cg-XIII), which stipulates for nations the obligations provide on a free and unrestricted basis essential data and products which are necessary for the provision of services in support of the protection of life and property and the well-being of all nations. For these “Essential data” defined in Annex 1 of the above resolution, the system allows unrestricted access to Authenticated Users, which means users whose identities were authenticated when they logged on. They have no authorized users and they don’t have any role assigned. They only have access to the public domain.

For all the “Additional data”, which are not included in Annex 1, the system allows restricted access only.

Metadata Policy

Regarding Metadata, we make a distinction between Static and Dynamic metadata. Static metadata are information not related to the data themselves. They are modified very rarely and generally by authorized users when physical changes are produced in the observational networks. Static metadata includes information related to observational stations and measurement instruments, information about physical parameters and measurement units, etc.

Data Catalog

The data catalog is built from the dynamical metadata registered at the same time as data in the databases. It allows users to check the availability of different types of data in the database. It contains hierarchical information about each data including:

• type
• category
• originator
• data source
• Geo-location
• temporal validity
• data quality

Authentic user has access only to the public selection of metadata, but authorized users can consult the entire catalog, including essential and additional data.

The data catalog contains information about all types of data existent in the Storage System, including both data and elaborated products. In addition to the above-listed information, for elaborated products, the Data Catalogue contains information about:

• the author of the product
• domain of applicability
• elaboration date

Climate Storage

Climate Storage or Climatological Database (CDB) is designed to store climatological data and metadata according to WMO CDMS specifications (WMO- No. 1131).

Climatological data are processed and stored on CDB by The COMET and The Climatology modules.

Data Access Layer (DAL)

The Data Access Layer offers role-based access to the information from the Storage System. Roles make it easy to assign the same set of permissions to multiple users based on job function. Permission to perform certain operations is assigned to specific roles. In turn, these permissions apply to the users who belong to that role.

The GoForMet system is very complex and has different workflows, and each workflow could be associated with a specific user role.

The Web Screens have special panels that allow administrators to configure role-based access for different users at different functionalities of the application and different kinds of data. They can:

• create and remove roles,
• configure users in roles,
• configure permissions for roles,
• assign different ownership on certain items to other users

Data Exchange

This functionality allows administrators to use Web-GUI panels:

• assign users access roles for certain types of data, for different periods, in certain regions on a contractual basis
• attach customized data licenses to certain types of data, grant end-user permission to use them on a contractual basis
• configure and schedule export tasks to retrieve the data or products from archives and send them to end-users in standard formats. All the data transmissions are logged by the system.
• on a commercial basis, end-users can receive authorized access to pull data themselves using the DAL API. The access will be limited in time and will grant access from the Data Catalogue only to the kind of data agreed upon (period, region, type). All the requests are logged by the system.

Data Backup and Recovery

Any transaction effectuated on the PostgreSQL database is rolled back if not successful. Supposing that something goes wrong due to a human mistake, we strongly recommend performing a manual backup of the database before any sensitive change.

Before any change to any database, a manual backup of the concerned database is advised. This will easily allow us to recover the previous running state in the rare case when something goes wrong.

If such a backup was not performed, anyway the corrupted database can be safely recovered from the last automatic backup made according to the scheduling.