YAML Parser

From WikiROMS
Jump to navigationJump to search
YAML Parser

Starting with svn revision -r 1902 released on March 1, 2022, the ROMS metadata is managed with a YAML file, and the regular text file varinfo.dat is deprecated. The YAML files are simple, easy to follow, elegant, portable, and expandable. ROMS can now process YAML files with its parser module, yaml_parser.F. Therefore, there is no need to use third-party YAML parsers.

The ROMS YAML parser source code can be found in ROMS/Utility. It is written in Fortran 2003 and includes a CLASS of type yaml_tree for parsing input YAML files. You may download our YAML parser here.


Although several YAML parsers for Fortran exist, a more straightforward and uncomplicated parser with substantial capabilities was coded in ROMS, yaml_parser.F. It is a hybrid between standard and Object-Oriented Programming (OOP) principles but without the need for recurrency, polymorphism, and containers (another library).

The only constraint in the ROMS parser is that the YAML file is read twice for simplicity and to avoid containers. The container is a Fortran vector! The first read determines the indentation policy and the length of the collection vector, list(:) pairs object (derived-type structure yaml_pair). The first reading is quick.

TYPE, PUBLIC :: yaml_pair

logical :: has_alias ! alias * token
logical :: has_anchor ! anchor & token
logical :: is_block ! block - list
logical :: is_sequence ! sequence [] tokens
logical :: is_logical ! logical value
logical :: is_integer ! integer value
logical :: is_real ! floating-point value
logical :: is_string ! string value

integer :: id ! key/value ID
integer :: parent_id ! parent ID
integer :: left_padding ! indent level: 0,1,..

character (len=:), allocatable :: line ! YAML line
character (len=:), allocatable :: key ! YAML keyword:
character (len=:), allocatable :: value ! YAML value(s)
character (len=:), allocatable :: anchor ! anchor keyword

END TYPE yaml_pair

The YAML file dictionary CLASS yaml_tree is defined as:

TYPE, PUBLIC :: yaml_tree

logical :: IsCreated = .FALSE. ! YAML object creation switch

integer :: Nbranches ! total number of branches
integer :: Npairs ! total number of pairs
integer :: indent ! blank indentation policy

character (len=:), allocatable :: filename ! YAML file name

TYPE (yaml_pair), pointer :: list(:) ! collection pairs


PROCEDURE :: create => yaml_tree_create
PROCEDURE :: destroy => yaml_tree_destroy
PROCEDURE :: dump => yaml_tree_dump
PROCEDURE :: extract => yaml_tree_extract
PROCEDURE :: fill => yaml_tree_fill
PROCEDURE :: fill_aliases => yaml_tree_fill_aliases
PROCEDURE :: has => yaml_tree_has
PROCEDURE :: read_line => yaml_tree_read_line

END TYPE yaml_tree

The yaml_tree object stores all the data contained in a specific YAML file. For Example, in ROMS the input YAML metadata dictionary is created and initialized as follows:

USE yaml_parser_mod, ONLY : yaml_initialize

logical :: Lreport ! verbose report switch
integer :: ErrorFlag ! processing error flag

TYPE (yaml_tree) :: self ! declare a dummy YAML object

IF (.not.self%IsCreated) THEN ! process input YAML file
Lreport = .TRUE.
ErrorFlag = yaml_initialize (self, 'varinfo.yaml', Lreport)

The error management is omitted for clarity. Then, the needed data is extracted from the self object and loaded into the internal ROMS variables using the overloaded yaml_get API:

INTERFACE yaml_get

MODULE PROCEDURE yaml_Get_i_struc ! Gets integer structure
MODULE PROCEDURE yaml_Get_l_struc ! Gets logical structure
MODULE PROCEDURE yaml_Get_r_struc ! Gets real structure
MODULE PROCEDURE yaml_Get_s_struc ! Gets string structure

MODULE PROCEDURE yaml_Get_ivar_0d ! Gets integer value
MODULE PROCEDURE yaml_Get_ivar_1d ! Gest integer values
MODULE PROCEDURE yaml_Get_lvar_0d ! Gets logical value
MODULE PROCEDURE yaml_Get_lvar_1d ! Gets logical values
MODULE PROCEDURE yaml_Get_rvar_0d ! Gets real value
MODULE PROCEDURE yaml_Get_rvar_1d ! Gets real values
MODULE PROCEDURE yaml_Get_svar_0d ! Gets string value
MODULE PROCEDURE yaml_Get_svar_1d ! Gets string values


Additionally, the YAML parser module has the following public and private routines/functions used during processing:

PUBLIC  :: yaml_AssignString
PUBLIC  :: yaml_Error
PUBLIC  :: yaml_get
PUBLIC  :: yaml_initialize

PRIVATE :: yaml_CountKeys
PRIVATE :: yaml_LowerCase
PRIVATE :: yaml_UpperCase
PRIVATE :: yaml_ValueType

The parser module is self-contained with minimal dependencies on other ROMS modules:

USE mod_kinds, ONLY : kind_real => dp ! double-precision
USE mod_parallel, ONLY : yaml_Master => Master ! master PET
USE mod_scalars, ONLY : yaml_ErrFlag => exit_flag ! error flag
USE mod_iounits, ONLY : yaml_stdout => stdout ! standard ouput

Overall, the parser is very fast and works in parallel. All PETs are involved in their dictionary copy to avoid overhead from collective MPI calls.


Currently, the YAML parser supports the following features:

  • Single or multiple line comments start with a hash #. Also, comment after a key/value pair is allowed. All comments are skipped during processing.
  • It has an unlimited nested structure (lists, mappings, hierarchies). Indentation of whitespace is used to denote structure.
  • It has an unrestricted schema indentation. However, some schema validators recommend or impose two whitespaces indentations.
  • A colon follows a key to denote a mapping value like:
    ocean_model: ROMS
  • It supports Anchors and Aliases.
    ATM_component: &ATM WRF


    - standard_name: surface_eastward_wind
    long_name: surface eastward wind
    short_name: Uwind
    data_variables: [uwind, time]
    source_units: m s-1
    destination_units: m s-1
    source_grid: cell_center
    destination_grid: cell_center
    add_offset: 0.0d0
    scale: 1.0d0
    debug_write: false
    connected_to: *ATM # u10
    regrid_method: bilinear
    extrapolate_method: none
  • It supports blocking lists: members are denoted by a leading hyphen-and-space, which is considered part of the indentation.
  • It supports a flow sequence: a vector list with values enclosed in square brackets and separated by a comma-and-space, like a keyword: [val1, ..., valN].
  • The keyword value(s) is (are) processed and stored as strings but converted to a logical, integer, floating-point, or derived-type when appropriate during extraction. If particular derived-type values are needed, the caller can process such a structure outside the parser.
  • It removes unwanted control characters like tabs and separators (ASCII character code 0-31).
  • It is restricted to the English uppercase and lowercase alphabet but can be expanded to other characters (see yaml_ValueType routine).
  • Multiple or continuation lines are supported. So, for example, we can have:
    state variables: [sea_surface_height_anomaly,


Several derived-type structures are declared to facilitate the extraction of similar data blocks compactly from the YAML dictionary (CLASS yaml_tree) in a compact way.

TYPE, PUBLIC :: yaml_Ivec ! integer structure
integer, allocatable :: vector(:) ! vector values
END TYPE yaml_Ivec

TYPE, PUBLIC :: yaml_Lvec ! logical structure
logical, allocatable :: vector(:) ! vector values
END TYPE yaml_Lvec

TYPE, PUBLIC :: yaml_Rvec ! real structure
real (kind_real), allocatable :: vector(:) ! vector values
END TYPE yaml_Rvec

TYPE, PUBLIC :: yaml_Svec ! string structure
character (len=:), allocatable :: value ! scalar value
TYPE (yaml_Svec), pointer :: vector(:) ! recursive vector
END TYPE yaml_Svec ! values

The derived-type structure below, extended/inherited from parent "yaml_Svec", extracts hierarchies of keys and associated values from the YAML dictionary object. The calling program specifies a key string that may be generated by aggregating nested keys with a period. Also, it can extract flow sequence string element values that are separated by commas.

TYPE, PRIVATE, EXTENDS(yaml_Svec) :: yaml_extract
logical :: has_vector ! true if loaded vector values
END TYPE yaml_extract

These public structures can be used in applications to extract block list YAML constructs. The key may represent a sequence flow [...] with a vector of values. The values can be integers, logicals, reals, or strings. For example, suppose that the YAML file has the following blocking list entries:


- standard_name: surface_downward_heat_flux_in_sea_water
long_name: surface net heat flux
short_name: shflux
data_variables: [shf, shf_time]
scale: 1.0d0
debug write: true

- standard_name: surface_wind_x_stress
long_name: surface zonal wind stress component
short_name: sustr
data_variables: [taux, atm_time]
scale: 1.0d0
debug write: false

- standard_name: surface_wind_y_stress
long_name: surface meridional wind stress component
short_name: svstr
data_variables: [tauy, atm_time]
scale: 1.0d0
debug write: false

Then, the code to extract the data from the YAML dictionary (self) is straightforward (see above for how to create and initialize a YAML dictionary object):

USE yaml_parser_mod, ONLY : yaml_get

integer :: ErrorFlag ! processing error flag

logical, allocatable :: L(:) ! logical data
real(dp), allocatable :: F(:) ! floating point data

TYPE (yaml_Svec), allocatable :: S1(:), S2(:) ! string data

ErrorFlag = yaml_get (self, 'import.short_name', S1)
ErrorFlag = yaml_get (self, 'import.data_variables', S2)
ErrorFlag = yaml_get (self, 'import.scale', F)
ErrorFlag = yaml_get (self, 'import.debug write', L)

Notice that the internal adopted syntax is to aggregate the hierarchy of keys separated by a dot in the keystring (e.g., 'import.debug write') during the extraction call. The above extraction statement yields the following values from a single invocation of the overloaded function yaml_get:

S1(1)%value = 'shflux' ! 'short_name' single value
S1(2)%value = 'sustr'
S1(3)%value = 'svstr'

S2(1)%vector(1)%value = 'shf' ! 'data_variables' vector of values
S2(1)%vector(2)%value = 'shf_time'

S2(2)%vector(1)%value = 'taux' ! 'data_variables' vector of values
S2(2)%vector(2)%value = 'atm_time'

S2(3)%vector(1)%value = 'tauy' ! 'data_variables' vector of values
S2(3)%vector(2)%value = 'atm_time'

F(1) = 1.0d0 ! 'scale' single value
F(2) = 1.0d0
F(3) = 1.0d0

L(1) = .TRUE. ! 'debug write' single value
L(2) = .FALSE.
L(3) = .FALSE.

Check ROMS/Utility/get_metdata.F for more details on how the metadata is processed from input YAML files.