NUOPC Cap UFS
Overview
This document describes the stand-alone ROMS ESMF/NUOPC cap module to be used by third-party coupling frameworks, like the Unified Forecast System (UFS). It is a lightweight software layer on top of ROMS that can be used by NUOPC-based packages (CMEPS/CDEPS, NEMS, and others) to couple to other Earth System Models (ESMs). Detailed information about ESMF/NUOPC and its implementation in ROMS can be found on the Earth System Modeling Framework WikiROMS page.
A NUOPC cap is a Fortran module that serves as the interface to a model when it's used in a NUOPC-based coupled system. The term cap is used because it is a lightweight software layer that sits on top of model code, making calls to it and exposing model data structures in a standard way.
Implementation
The cmeps_roms.F module declares several derived-type structures to facilitate the management of all internal objects and variables:
- ESM coupling time managing variables and ESMF objects. TYPE, PRIVATE :: ESM_Clock
logical :: Restarted
integer (i8b) :: AdvanceCount ! advance counter
real (dp) :: Current_Time ! seconds
real (dp) :: Time_Reference ! seconds
real (dp) :: Time_Restart ! seconds
real (dp) :: Time_Start ! seconds
real (dp) :: Time_Stop ! seconds
real (dp) :: Time_Step ! seconds
character (len=22) :: Name
character (len=22) :: CalendarString ! 360_day, gregorian
character (len=22) :: Time_ReferenceString
character (len=22) :: Time_RestartString
character (len=22) :: Time_StartString
character (len=22) :: Time_StopString
TYPE (ESMF_Calendar) :: Calendar
TYPE (ESMF_Clock) :: Clock
TYPE (ESMF_Direction_flag) :: Direction
TYPE (ESMF_Time) :: CurrentTime
TYPE (ESMF_Time) :: ReferenceTime
TYPE (ESMF_Time) :: RestartTime
TYPE (ESMF_Time) :: StartTime
TYPE (ESMF_Time) :: StopTime
TYPE (ESMF_TimeInterval) :: TimeStep
END TYPE ESM_Clock
TYPE (ESM_Clock), allocatable, target :: ClockInfo(:)
- ESM coupled state sets. If appropriate, it includes the logic for connecting nested grids. TYPE, PRIVATE :: ESM_CplSet
logical, allocatable :: LinkedGrid(:,:) ! connected grid
character (len=100), allocatable :: SetLabel(:) ! set label
character (len=100), allocatable :: ExpLabel(:) ! export label
character (len=100), allocatable :: ImpLabel(:) ! import label
END TYPE ESM_CplSet
TYPE (ESM_CplSet), allocatable, target :: COUPLED(:)
- Import and export fields metadata information. TYPE, PRIVATE :: ESM_Field
logical :: connected ! connected to coupler
logical :: debug_write ! write exchanged field
integer :: gtype ! field grid mesh type
integer :: Tindex ! rolling two-time indices
character (len=:), allocatable :: short_name ! short name
character (len=:), allocatable :: standard_name ! standard name
character (len=:), allocatable :: long_name ! long name
character (len=:), allocatable :: dst_gtype ! DST grid type
character (len=:), allocatable :: dst_units ! DST units
character (len=:), allocatable :: src_gtype ! SRC grid type
character (len=:), allocatable :: src_units ! SRC units
character (len=:), allocatable :: nc_vname ! DATA Vname
character (len=:), allocatable :: nc_tname ! DATA Tname
character (len=:), allocatable :: map_norm ! mapping norm
character (len=:), allocatable :: map_type ! regrid method
character (len=22) :: DateString(2) ! snapshots date
real (dp) :: scale_factor ! field scale factor
real (dp) :: add_offset ! field add offset value
real (dp) :: Tmin ! DATA time minimum value
real (dp) :: Tmax ! DATA time maximum value
real (dp) :: Tstr ! DATA lower time-snapshot
real (dp) :: Tend ! DATA upper time-snapshot
real (dp) :: Tintrp(2) ! interpolation time (day)
real (dp) :: Vtime(2) ! latest two-time values
TYPE (ESMF_RouteHandle) :: rhandle ! field RouteHandle
END TYPE ESM_Field
- Import and export fields mesh data. TYPE, PRIVATE :: ESM_Mesh
integer :: gid ! grid ID
integer :: gtype ! grid mesh type
integer (i4b), allocatable :: mask(:,:) ! grid land/sea mask
real (r8), allocatable :: lon(:,:) ! grid longitude
real (r8), allocatable :: lat(:,:) ! grid latitude
real (r8), allocatable :: area(:,:) ! grid area
END TYPE ESM_Mesh
- Coupled models high-level object, [Nmodels=1]. TYPE, PRIVATE :: ESM_Model
logical :: IsActive ! active for coupling
integer (i4b) :: LandValue ! land mask value
integer (i4b) :: SeaValue ! sea mask value
integer :: Ngrids ! number nested grids
integer :: ExportCalls ! export CALL counter
integer :: ImportCalls ! import CALL counter
integer :: nPETs ! number model PETs
integer, allocatable :: PETlist(:) ! component PETs list
integer, allocatable :: TimeFrac(:,:) ! driver time fraction
character (len=:), allocatable :: name ! component name
TYPE (ESMF_Grid), allocatable :: grid(:) ! grid object
TYPE (ESM_Mesh), allocatable :: mesh(:) ! mesh
TYPE (ESM_Field), allocatable :: ImportField(:) ! import fields
TYPE (ESM_Field), allocatable :: ExportField(:) ! export fields
TYPE (ESMF_State), allocatable :: ImportState(:) ! import state
TYPE (ESMF_State), allocatable :: ExportState(:) ! export state
END TYPE ESM_Model
TYPE (ESM_Model), allocatable, target :: MODELS(:)
- Internal module parameters and variables:
- Switch to trace/track run sequence during debugging. All information is written to Fortran unit trac. For now, it uses the standard output unit. logical :: ESM_track = .TRUE. ! trace/track CALL sequence switch
- Number of coupled ESM gridded components and identification index. integer, parameter :: Nmodels = 1
integer, parameter :: Idriver = 0
integer, parameter :: Iroms = 1 - Number of ROMS export and import fields per component. integer, allocatable :: Nexport(:)
integer, allocatable :: Nimport(:) - Model coupling type: [1] Explicit, [otherwise] Semi-Implicit. In explicit coupling, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In implicit coupling, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In semi-implicit coupling, ROMS -> ATM is explicit, ATM -> ROMS is implicit. integer :: CouplingType = 1
- Linked/coupled ROMS nested grid number. integer :: linked_grid
- Distributed-memory communicator handle for each component, rank of each PET, and PET layout (sequential or concurrent). integer, allocatable :: ESMcomm(:)
integer :: PETrank
character (len=10), allocatable :: PETlayoutOption(:) - Coupling debugging flag: [0] no debugging, [1] reports informative messages, or [2] '1' plus writes grid information in VTK format, [3] '2' plus writes exchage fields into NetCDF files. integer :: DebugLevel = 0
- Execution tracing level flag: [0] no tracing, [1] reports sequence of coupling subroutine calls, or [2] <1> plus writes voluminous ESMF library tracing information which slowdown performace, and creates large log file. integer :: TraceLevel = 0
- Standard output units. integer :: cplout = 77 ! coupling driver
integer :: trac = 6 ! trace/track CALL sequence unit - Coupled model staggered grid-cell type indices. integer, parameter :: Inan = 0 ! unstaggered, cell center
integer, parameter :: Icenter = 1 ! cell center
integer, parameter :: Icorner = 2 ! cell corners
integer, parameter :: Iupoint = 3 ! right and left cell faces
integer, parameter :: Ivpoint = 4 ! upper and lower cell faces
character (len=6), dimension(0:4) :: GridType = &
& (/ 'N/A ', &
& 'Center', &
& 'Corner', &
& 'U ', &
& 'V ' /) - Standard input filename for each coupled model, [Nmodels]. character (len=256), allocatable :: INPname(:)
- ROMS coupling YAML configuration filename. character (len=:), allocatable :: CPLname
- ESM strings. character (len=:), allocatable :: CoupledSet
character (len=:), allocatable :: ExportStateName
character (len=:), allocatable :: ImportStateName - ESM single and double precision constants. integer (i4b), parameter :: MAPPED_MASK = 99_i4b
integer (i4b), parameter :: UNMAPPED_MASK = 98_i4b
real (dp), parameter :: MISSING_dp = 1.0E20_dp
real (r4), parameter :: MISSING_r4 = 1.0E20_r4
real (r8), parameter :: MISSING_r8 = 1.0E20_r8
real (dp), parameter :: TOL_dp = 0.5E20_dp
real (r4), parameter :: TOL_r4 = 0.5E20_r4
real (r8), parameter :: TOL_r8 = 0.5E20_r8
- Switch to trace/track run sequence during debugging. All information is written to Fortran unit trac. For now, it uses the standard output unit.
UFS Interface
The NUOPC cap file is connected to the UFS.F90 and EARTH_GRID_COMP.F90 with C-preprocessing option FRONT_ROMS as follows:
...
#ifdef FRONT_ROMS
USE cmeps_roms_mod, ONLY : ROMS_SS => ROMS_SetServices
#endif
...
END MODULE module_EARTH_GRID_COMP
Module Subroutines
The ROMS cap module contains a set of subroutines that are required by NUOPC. These subroutines are called by the NUOPC infrastructure according to a predefined calling sequence. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.
The initialization sequence is the most complex and is governed by the NUOPC technical rules. Details about the initialization sequence can be found in the NUOPC Reference Manual. The ROMS cap requires ESMF version 8 or higher.
| ROMS_SetServices | Entry point to the ROMS cap and the only public routine. It sets the ROMS component shared-object entry points for using NUOPC generic methods for initialize, run, and finalize. |
| ROMS_Create | Allocates module internal structures and processes configuration from YAML file: roms_cmeps.yaml. |
| ROMS_SetInitializeP1 | ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state. |
| ROMS_SetInitializeP2 | ROMS component phase 2 initialization which initializes the ROMS component (ROMS_Initialize), sets component grid (ROMS_SetGridArrays), and adds fields into import and export into respective states. |
| ROMS_DataInit | Exports ROMS component fields during initialization or restart. |
| ROMS_SetClock | Sets ROMS component date calendar, start and stop times, and coupling interval. |
| ROMS_SetRunClock | Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call. |
| ROMS_CheckImport | Checks if ROMS component import field is at the correct time. |
| ROMS_SetGridArrays | Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any. |
| ROMS_SetStates | Adds ROMS component export and import fields into its respective state. |
| ROMS_ModelAdvance | Advances ROMS component for a coupling interval. It calls ROMS_Import and ROMS_Export routines. |
| ROMS_SetFinalize | Finalizes ROMS component execution. |
| ROMS_Import | Loads import fields into ROMS internal kernel arrays. |
| ROMS_Export | Exports ROMS fields to other gridded components. |
| ROMS_Rotate | Rotates vector components from computational grid to geographical EAST and NORTH directions or vice versa. |
| Report_TimeStamp | Reports coupling time-stamp. |
Import and Export Fields
The export and import fields are configured and specified in the YAML file roms_cmeps.yaml. Check the following wikiROMS page for detailed information. The user has full control of the coupling exchange fields metadata.
Currently, CMEPS does not support coupling of nested grids. The ESMF/NUOPC library version 8.0.0 or higher supports native nesting capabilities. The NUOPC layer now allows coupling sets in the import and export states of the ESM components. A nested grid is represented as another ESMF_Grid object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The NUOPC connector can recognize the different grids and exchange fields to a connected ESM component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The NUOPC_AddNestedState routine is used to advertise the nested couple sets.
For example, the NOUPC cap module routines ROMS_SetInitializeP1 can be modified easily to add the coupled sets to the ImportState and ExportState as shown below in the green blocks of code.
|
|
