Skip navigation links
(CGNS Documentation Home Page) (Steering Committee Charter) (Overview and Entry-Level Document) (A User's Guide to CGNS) (Mid-Level Library) (Standard Interface Data Structures) (SIDS File Mapping Manual) (CGIO User's Gudie) (Parallel CGNS User's Guide) (ADF Implementation) (HDF5 Implementation) (Python Implementation) (CGNS Tools and Utilities)

(Implementation Summary) (General HDF5 Mapping Concepts)

Implementation Summary

The purpose of the current document is to describe the way in which CFD data is to be represented in an HDF5 data tree. To do this, it is necessary to first describe the HDF5 data structure itself in some detail. Therefore, a conceptual summary of HDF5 is given here in order to make the current document relatively independent, and to allow the reader to focus on those aspects of HDF5 which are essential to understanding the file mapping.

Any HDF5 file with a conformant mapping is CGNS/HDF5 compliant. The mapping has been made using a per-node basis. Instead of having a new mapping dedicated to HDF5, we have made a mapping from the ADF nodes to a set of HDF5 groups. While this is not an optimal mapping, the use of such an HDF5 node allows us to re-use the ADF API without change.

The HDF5 documentation external link should be used as the authoritative references to resolve any issues not covered by this summary.

General Description of HDF5

The HDF5 library provides CGNS with a low level storage system. This library is developed and maintained by the National Center for Supercomputing Applications external link at the University of Illinois at Urbana-Champaign. It offers a free storage system, with support for a very large number of platforms used in the scientific world. This system is in charge of all physical features such as storage, compression, virtual access to data and many other services also useful for the CGNS users.

HDF5 allows a fine tuning of the physical devices, however, the SLL use of HDF5 forces all property lists to DEFAULT values.

Refer to the HDF5 web site external link for further information about HDF5.

Dedicated HDF5 Structures

The CGNS node is an HDF5 group. It contains attributes, an optional dataset and optional child groups. This structure is required and can be extended by other implementors as long as the following requirements are fullfilled.

In a CGNS tree, most of the nodes are normal nodes. These nodes can optionally contain data and have other nodes as children.

There are special nodes, such as the root node and the nodes managing the links. These special link structures use the HDF5 mount system. It is not necessary for either the MLL or SLL users to understand the HDF5 mount system, but it should be understood by SLL implementors.

The CGNS Node Mapping

The CGNS node uses groups and attributes elements of HDF5. A basic CGNS node is composed of elements listed in the following table. The attribute names have been enclosed in square brackets, because the SLL uses private attributes with a leading space character. These brackets are not part of the names.

Basic Node


GROUP     

REQUIRED ATTRIBUTE [name] U8LE[33]
REQUIRED ATTRIBUTE [label] U8LE[33]
REQUIRED ATTRIBUTE [type] U8LE[3]
REQUIRED ATTRIBUTE [ order] H5T_NATIVE_INT
OPTIONAL DATASET [ data]

All the children groups of a node are understood as SLL children nodes of this node, unless the group name starts with a space character. The dataset contains the actual data of a node. In the case of a node without data (i.e. MT type), the dataset does not exist.

In addition to the attributes of a normal node, special attribute names are reserved for the root node, shown in the following table. The name of the root node is [HDF5 MotherNode], its label is [Root Node of HDF5 File], type is MT.

Root Node


GROUP   [/]  

REQUIRED ATTRIBUTE [ version] U8LE[33]
REQUIRED ATTRIBUTE [ format] U8LE[33]

The node hosting the link entries is a special node named [ mount] without attributes nor data. This group should be a child of the root node, its absolute path is [/ mount]. The group [ mount] contains one group for each mounted file refered to by a link. Each entry has the attributes described in the table below. The link node itself is a group with the special name [ link]. This implementation is discussed further in the section describing the link mechanism.

Link Node


GROUP   [/ mount/ *]  

REQUIRED ATTRIBUTE [ refcnt] H5T_NATIVE_INT
REQUIRED ATTRIBUTE [ file] H5T_NATIVE_CHAR

The Node ID

The node ID is a unique identifier assigned to each existing node by HDF5 when the file containing it is opened, and to new nodes as they are created. SLL inquiries generally return node IDs as a result and accept node IDs as input. By building a table of IDs, calling software can subsequently access specific nodes without further search. The Node ID is real and is not under user control, its lifetime is ended by the closing of the HDF5 file.

The Node Name

The node Name is a 32-byte character field which is user controllable. Its general use is to distinguish among the children of a given node; consequently, no two children of the same parent may have the same Name. Constraints related to the node name are detailed in the General CGNS SLL Mapping Concepts section, under Node Name.

The Label

The Label is a 32-byte character field which is user controllable. SLL assigns no formal role to the Label, but the intent was to identify the structure of the included data. It is common for the various children of a single parent to store different instances of the same structure. Therefore, there is no prohibition against more than one child of the same parent having the same Label.

The Data Type

The Data Type is a 32-byte character field which specifies the type and precision of any data which is stored in the data field. Types provided by HDF5 are:

Data Types


Data Type   Notation   HDF5 Type

No Data MT -
Integer 32 I4 H5T_NATIVE_INT32
Integer 64 I8 H5T_NATIVE_INT64
Unsigned Integer 32 U4 H5T_NATIVE_UINT32
Unsigned Integer 64 U8 H5T_NATIVE_UINT64
Real 32 R4 H5T_NATIVE_FLOAT
Real 64 R8 H5T_NATIVE_DOUBLE
Complex 64 X4 -
Complex 128 X8 -
Character C1 H5T_NATIVE_CHAR
Byte B1 H5T_NATIVE_UCHAR
Link LK -

There is no mapping to HDF5 MT and LK types, because there is no actual data space associated with the nodes. The type information itself is stored in the node as the strings "MT" and "LK". The types X4 and X8 are also not mapped.

The data storage format is translated as described below.

Native Formats


Native Format   HDF5 Type

H5T_IEEE_F32BE IEEE_BIG_32
H5T_IEEE_F32LE IEEE_LITTLE_32
H5T_IEEE_F64BE IEEE_BIG_64
H5T_IEEE_F64LE IEEE_LITTLE_64

The Number of Dimensions

The Data portion of a node is designed to store multi-dimensional arrays of data, each element of which is presumed to be of the Data Type specified. The Number of Dimensions specifies the number of integers required to reference a single datum within the array.

The Dimension Values

The Dimension Values are a list of integers expressing the actual sizes of the stored array in each of the dimensions specified. These dimensions are stored by the dataspace associated with the dataset; no other attributes contains these values.

The Data

In an HDF5 node, the portion of the node holding the actual stored data array is a dataset.

The Child Table

All groups of the current group are said to be the children of this group, except the groups with a name starting with a space character. Each of these children groups is a normal node (i.e. group).

Children may be identified by their names and labels, and, thence, by their node IDs once these have been determined. HDF5 provides no notion of order among children, but the SLL layer adds a creation order stored in the [ order] attribute. This order is guaranteed to be the same from call to call, even after the file has been closed and re-opened.

Note that there is no parent table; that is, a node has no direct knowledge of its parent. Since calling software must open the file from the root, it presumably cannot access a child without having first accessed the parent. It is the responsibility of the calling software to record the node ID of the parent if this information will be required.

The link mechanism

A LK typed node is a link. Such a node refers to another node elsewhere. In other words, the link has no child or contents, but is a name of a node somewhere in the current file or in another file. The LK typed node is said to be the emph node, while the node elsewhere is said to be the destination node. Is is the role of the SLL layer to insure consistency [in particular in order to avoid an acyclic graph] and transparency of the link mechanism, so that any normal node request to the link node is performed as if it is performed on the destination node. A link destination can be in the same file as the link source or in another file. In both cases, the link is made using an HDF5 soft link from source to destination. In the case of a destination in another file, the destination file is mounted in the [/ mount] [absolute path] group and the soft link is made from the source to the destination now present in this mounted file. Each time a file is mounted, the [/ refcnt] [absolute path] attribute is incremented. The file is unmounted if there is no reference to itself.