From WikiROMS
Jump to navigationJump to search


This document describes the ROMS ESMF/NUOPC cap module Master/esmf_roms.F used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (Master/esmf_driver.h), which creates, register, initialize, run, and finalize the coupling of each Earth System Model (ESM) component (data, atmosphere, sea-ice, wave, and ocean). It also sets the coupler connectors and the RunSequence policy (Master/esmf_esm.F). The framework provides the NUOPC cap modules for each coupled component. Detailed information about ESMF/NUOPC and how it is implemented in ROMS can be found on the Earth System Modeling Framework WikiROMS page.

A NUOPC cap is a Fortran module that serves as the interface to a model when used in a NUOPC-based coupled system. The term cap is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its initialize, run, and finalize phases and exposing the model data structures to the field exchange (import and export states) between coupled components.

ROMS Coupling.png
Figure 1: ROMS native ESMF/NUOPC coupling infrastructure


The Master/mod_esmf_esm.F module declares several derived-type structures to facilitate the management of all internal objects and variables in each ESM coupled component:

  • ESM coupling time managing variables and ESMF objects. The ClockInfo object is allocated as (0:Nmodels), where Nmodels is the number of coupled ESM components. Here, index zero corresponds to the driver.
    TYPE :: ESM_Clock

    logical :: Restarted

    integer (i8b) :: AdvanceCount ! advance counter

    real (dp) :: Current_Time ! seconds
    real (dp) :: Time_Reference ! seconds
    real (dp) :: Time_Restart ! seconds
    real (dp) :: Time_Start ! seconds
    real (dp) :: Time_Stop ! seconds
    real (dp) :: Time_Step ! seconds

    character (len=22) :: Name
    character (len=22) :: CalendarString ! 360_day, gregorian
    character (len=22) :: Time_ReferenceString
    character (len=22) :: Time_RestartString
    character (len=22) :: Time_StartString
    character (len=22) :: Time_StopString

    TYPE (ESMF_Calendar)  :: Calendar
    TYPE (ESMF_Clock)  :: Clock
    TYPE (ESMF_Direction_flag) :: Direction
    TYPE (ESMF_Time)  :: CurrentTime
    TYPE (ESMF_Time)  :: ReferenceTime
    TYPE (ESMF_Time)  :: RestartTime
    TYPE (ESMF_Time)  :: StartTime
    TYPE (ESMF_Time)  :: StopTime
    TYPE (ESMF_TimeInterval)  :: TimeStep

    END TYPE ESM_Clock

    . . .

    TYPE (ESM_Clock), allocatable, target :: ClockInfo(:)
  • ESM coupled state sets. If appropriate, it includes the logic for connecting nested grids. The COUPLED object is of size Nmodels.
    TYPE :: ESM_CplSet

    logical, allocatable :: LinkedGrid(:,:) ! connected grid

    logical, allocatable :: DataCoupledSets(:,:) ! DATA linked sets

    character (len=100), allocatable :: SetLabel(:) ! set label
    character (len=100), allocatable :: ExpLabel(:) ! export label
    character (len=100), allocatable :: ImpLabel(:) ! import label

    TYPE (ESMF_State), allocatable :: ExportState(:,:) ! export set


    TYPE (ESM_CplSet), allocatable, target :: COUPLED(:)
  • DATA model field processing information. The DATA component is often necessary for applications with incongruent grids to provide complete field melding capabilities or methods for reading and interpolating stream data in space and time from archived NetCDF files.
    TYPE ESM_Data

    logical :: Lcycle ! cycling time coordinate
    logical :: Lcoord ! coordinates attribute
    logical :: Lmask ! land/sea mask
    logical :: Lmulti ! field across multi-files
    logical :: LastRec ! processed last record

    integer :: Icomp ! target component index
    integer :: ncid ! NetCDF file ID
    integer :: Tid ! time variable ID
    integer :: Vid ! field variable ID
    integer :: Nvdim ! number spatial dimensions
    integer :: Zlevel ! level index to process
    integer :: Nrec ! number of time records
    integer :: Trec ! latest read time record
    integer :: Tindex ! rolling two-time indices
    integer :: LandValue ! Masking land value
    integer :: SeaValue ! Masking sea value

    integer, allocatable :: Vsize(:) ! dimensions size

    real(dp) :: add_offset ! add_offset attribute
    real(dp) :: FillValue ! _FillValue attribute
    real(dp) :: scale_factor ! scale_factor attribute
    real(dp) :: Clength ! time cycling length
    real(r8) :: LonMin ! grid minimum longitude
    real(r8) :: LonMax ! grid maximum longitude
    real(r8) :: LatMin ! grid minimum latitude
    real(r8) :: LatMax ! grid maximum latitude
    real(dp) :: Tscale ! time scale to day
    real(dp) :: Tmono ! monotonic time (days)
    real(dp) :: Tmin ! time minimum value
    real(dp) :: Tmax ! time maximum value
    real(dp) :: Tstr ! lower time-snapshot
    real(dp) :: Tend ! upper time-snapshot
    real(r8) :: Vmin ! variable minimum value
    real(r8) :: Vmax ! variable maximum value
    real(dp) :: Tintrp(2) ! interpolation time (days)
    real(dp) :: Vtime(2) ! latest two-time values
    real(dp) :: Date(6,2) ! time-snapshots dates YYYY,MM,DD

    character(len=30), allocatable :: Dname(:) ! variable dimensions names
    character(len=20), allocatable :: Vcoord(:) ! variable coordinates names

    character(len=22 ) :: DateString(2) ! date-snapshots string
    character(len=30 ) :: SpecialAction ! special processing
    character(len=100) :: Tname ! time variable name
    character(len=100) :: Tunits ! time variable units
    character(len=100) :: Vname ! variable name
    character(len=100) :: Vunits ! variable units
    character(len=256) :: Vdescriptor ! Variable descriptive name
    character(len=256) :: Vlongname ! long_name attribute
    character(len=256) :: ncfile ! NetCDF filename

    real(r8), allocatable :: lon(:,:) ! field longitude
    real(r8), allocatable :: lat(:,:) ! field latitude
    real(r8), allocatable :: mask(:,:) ! field land/sea mask

    real(r8), allocatable :: A2d(:,:) ! time interpolated 2D data
    real(r8), allocatable :: A3d(:,:,:) ! time interpolated 3D data
    real(r8), allocatable :: A2dG(:,:,:) ! latest 2D data snapsnots
    real(r8), allocatable :: A3dG(:,:,:,:)! latest 3D data snapsnots

    TYPE (ESMF_Field)  :: field ! field object
    TYPE (ESMF_Grid)  :: grid ! field grid object
    TYPE (ESMF_RouteHandle) :: rhandle ! field RouteHandle

  • DATA model high-level structure.
    TYPE ESM_DataSet

    integer :: Nfields ! number of fields
    integer :: Nfiles ! number of input files

    character(len=20), allocatable :: Ctarget(:) ! component target
    character(len=20), allocatable :: Field(:) ! field short-name

    TYPE (ESM_Data), allocatable :: Export(:) ! Export field structure
    TYPE (T_IO), allocatable :: IFS(:) ! Input Files structure

    END TYPE ESM_DataSet
  • Import and export field metadata information between the source (SRC) and destination (DST) components.
    TYPE :: ESM_Field

    logical :: connected ! connected to coupler
    logical :: debug_write ! write exchanged field
    logical :: enable_integral_adj ! area integral adjusted

    integer :: fid ! internal field ID
    integer :: gtype ! field grid mesh type
    integer :: etype ! field extrapolation flag
    integer :: itype ! field interpolation flag
    integer :: Tindex ! rolling two-time indices

    character (len=20) :: Ctarget ! component destination
    character (len=22) :: DateString(2) ! date-snapshots string

    character (len=:), allocatable :: short_name ! short name
    character (len=:), allocatable :: standard_name ! standard name
    character (len=:), allocatable :: long_name ! long name
    character (len=:), allocatable :: dst_gtype ! DST grid type
    character (len=:), allocatable :: dst_units ! DST units
    character (len=:), allocatable :: src_gtype ! SRC grid type
    character (len=:), allocatable :: src_units ! SRC units
    character (len=:), allocatable :: nc_vname ! DATA Vname
    character (len=:), allocatable :: nc_tname ! DATA Tname
    character (len=:), allocatable :: RegridMethod ! regrid method
    character (len=:), allocatable :: ExtrapMethod ! extrapolate

    real (dp) :: scale_factor ! field scale factor
    real (dp) :: add_offset ! field add offset value
    real (dp) :: Tmin ! DATA time minimum value
    real (dp) :: Tmax ! DATA time maximum value
    real (dp) :: Tstr ! DATA lower time-snapshot
    real (dp) :: Tend ! DATA upper time-snapshot
    real (dp) :: Tintrp(2) ! interpolation time (day)
    real (dp) :: Vtime(2) ! latest two-time values

    TYPE (ESMF_RouteHandle) :: rhandle  ! field RouteHandle

    END TYPE ESM_Field
  • Import and export fields mesh data.
    TYPE :: ESM_Mesh

    integer :: gid ! grid ID
    integer :: gtype ! grid mesh type

    integer (i4b), allocatable :: mask(:,:) ! grid land/sea mask

    real (r8), allocatable :: lon(:,:) ! grid longitude
    real (r8), allocatable :: lat(:,:) ! grid latitude
    real (r8), allocatable :: area(:,:) ! grid area

  • Melding coefficients used to combine fields from DATA and ESM components. The weight factors are read from the input NetCDF specified in the WeightsFile(atmos) keyword. The user has full control of how the merging is done. It is recommended to provide a gradual transition between the two components.

    Recall that the DATA component supplies needed data to a particular ESM component. For example, it may export data to the atmosphere model at locations not covered by the other ESM components because of smaller grid coverage. If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data:
    SST_atm(:,:) = Cesm(:,:) * SST_esm(:,:) + Cdat(:,:) * SST_dat(:,:)
    where Cesm(:,:) + Cdat(:,:) = 1.
    TYPE ESM_Meld

    integer :: NestedGrid ! grid needing merged field

    character(len=100) :: VnameDATA ! DATA weights variable name
    character(len=100) :: VnameESM ! ESM weights variable name
    character(len=256) :: ncfile ! Weights NetCDF filename

    real (r8), allocatable :: Cdat(:,:) ! coefficients for DATA
    real (r8), allocatable :: Cesm(:,:) ! coefficients for ESM


    TYPE (ESM_Meld), allocatable, target :: WEIGHTS(:)

    real (dp) :: WeightDAT = 0.0_dp ! DATA component weight
    real (dp) :: WeightESM = 1.0_dp ! ESM component weight
  • Coupled models high-level object, [Nmodels=5].
    TYPE :: ESM_Model

    logical :: IsActive ! active for coupling

    integer (i4b) :: LandValue ! land mask value
    integer (i4b) :: SeaValue ! sea mask value

    integer :: Ngrids ! number nested grids

    integer :: ExportCalls ! export CALL counter
    integer :: ImportCalls ! import CALL counter

    integer :: nPETs ! number model PETs
    integer, allocatable :: PETlist(:) ! component PETs list

    integer, allocatable :: TimeFrac(:,:) ! driver time fraction

    character (len=:), allocatable :: name ! component name

    TYPE (ESMF_Grid), allocatable :: grid(:) ! grid object
    TYPE (ESM_Mesh), allocatable :: mesh(:) ! mesh
    TYPE (ESM_Field), allocatable :: ImportField(:) ! import fields
    TYPE (ESM_Field), allocatable :: ExportField(:) ! export fields
    TYPE (ESMF_State), allocatable :: ImportState(:) ! import state
    TYPE (ESMF_State), allocatable :: ExportState(:) ! export state

    END TYPE ESM_Model

    TYPE (ESM_Model), allocatable, target :: MODELS(:)
  • Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [Nmodels, Nmodels].
    TYPE ESM_Conn

    logical :: IsActive ! active connector

    integer :: divDT
    integer :: MaskInteraction ! connector mask interaction

    integer :: nPETs ! number of connector PETs
    integer, allocatable :: PETlist(:) ! connector PETs list

    character (len=100) :: name ! connector name


    TYPE (ESM_Conn), allocatable, target :: CONNECTORS(:,:)
  • Define DATA Model field processing information from input data files, [1:Nmodels]. Currently, the DATA Model only export fields and it only supports input NetCDF files.
    TYPE (ESM_DataSet), allocatable, target :: DataSet(:)
  • ESM clock for driver (zeroth element) and coupled components, [0:Nmodels].
    TYPE (ESM_Clock), allocatable, target :: ClockInfo(:)
  • Internal module parameters and variables:
    • Number of coupled ESM gridded components. Currently, five ESM components are supported (ROMS, DATA, ATMOSPHERE, SEA-ICE, and WAVE model types).
      All supported components are accounted here even if we are running an application with less number of models. The IsActive switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active.
      integer, parameter :: Nmodels = 5
    • Number of nested grids. All are initialized to just one grid; its values are overwritten during processing. An additional variable NgridsR is created to avoid using ROMS module mod_param in the generic interface. Both Ngrids and NgridsR will have the same value.
      integer, parameter :: NgridsA = 1 ! Atmosphere Model
      integer, parameter :: NgridsD = 1 ! DATA Model
      integer, parameter :: NgridsI = 1 ! Sea-ice Model
      integer, parameter :: NgridsR = 1 ! ROMS
      integer, parameter :: NgridsW = 1 ! Wave Model
    • Coupled ESM components identification indices.
      integer, parameter :: Idriver = 0
      integer, parameter :: Iroms = 1
      integer, parameter :: Iatmos = 2
      integer, parameter :: Idata = 3
      integer, parameter :: Iseaice = 4
      integer, parameter :: Iwave = 5
    • Generic ESM component labels used in the CASE constructs. We cannot use the identification indices because the vector Iroms(:) cannot be defined as a parameter and a non constant expression is illegal: CASE ( Iroms(1) : Iroms(NgridsR) )
      character (len= 3), allocatable :: Clabel(:)
      character (len=10), allocatable :: Cmodel(:)
    • Number of ROMS export and import fields per component.
      integer, allocatable :: Nexport(:)
      integer, allocatable :: Nimport(:)
    • Model coupling type: [1] Explicit, [otherwise] Semi-Implicit. In explicit coupling, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In implicit coupling, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In semi-implicit coupling, ROMS -> ATM is explicit, ATM -> ROMS is implicit.
      integer :: CouplingType = 1
    • Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent).
      integer, allocatable :: ESMcomm(:)
      integer :: sumPETs
      integer :: PETrank
      character (len=10) :: PETlayoutOption
    • Driver clock parameters specified in configuration script. A integer vector with six elements: (1) year including century, like 2017; (2) month of the year, 1 to 12; (3) day of the month; (4) hour of the day, 0 to 23; (5) minutes of the hour, 0 to 59; and (6) seconds of the minute, 0 to 59
      integer :: ReferenceDate(6) ! reference date
      integer :: RestartDate(6) ! restarting date
      integer :: StartDate(6) ! starting date
      integer :: StopDate(6) ! stopping date
      integer :: TimeStep(6) ! coupling interval
    • Today date string.
      character (len=44) :: TodayDateString
    • ESM coupling simulation reference date number, element: (1) seconds, and (2) fractional days
      real (dp) :: ReferenceDateNumber(2)
    • DATA component parallel distributed-memory domain partions in the I- and J-directions (lon,lat).
      integer :: ItileD
      integer :: JtileD
    • Coupling debugging flag: [0] no debugging, [1] reports informative messages, or [2] '1' plus writes grid information in VTK format, [3] '2' plus writes exchage fields into NetCDF files.
      integer :: DebugLevel = 0
    • Execution tracing level flag: [0] no tracing, [1] reports sequence of coupling subroutine calls, or [2] <1> plus writes voluminous ESMF library tracing information which slowdown performace, and creates large log file.
      integer :: TraceLevel = 0
    • Switch to trace/track run sequence during debugging. All information is written to Fortan unit trac. For now, use standard output unit.
      logical :: ESM_track = .FALSE. ! trace/track CALL sequence switch
      integer :: trac = 6 ! trace/track CALL sequence unit
    • Coupled model staggered grid-cell type indices.
      integer, parameter :: Inan = 0 ! unstaggered, cell center
      integer, parameter :: Icenter = 1 ! cell center
      integer, parameter :: Icorner = 2 ! cell corners
      integer, parameter :: Iupoint = 3 ! right and left cell faces
      integer, parameter :: Ivpoint = 4 ! upper and lower cell faces

      character (len=6), dimension(0:4) :: GridType = &
      & (/ 'N/A ', &
      & 'Center', &
      & 'Corner', &
      & 'U ', &
      & 'V ' /)
    • REGRID interpolation method between source and destination fields.
      integer, parameter :: Inone = 0 ! none
      integer, parameter :: Ibilin = 1 ! bilinear
      integer, parameter :: Ipatch = 2 ! high-order patch recovery
      integer, parameter :: Iconsv1 = 3 ! first-order conservative
      integer, parameter :: Iconsv2 = 4 ! second-order conservative
      integer, parameter :: InStoD = 5 ! nearest neighbor Src to Dst
      integer, parameter :: InDtoS = 6 ! nearest neighbor Dst to Src

      character (len=4), dimension(0:6) :: IntrpType = &
      & (/ 'NONE', &
      & 'BLIN', &
      & 'PTCH', &
      & 'CNS1', &
      & 'CNS2', &
      & 'NS2D', &
      & 'ND2S' /)
    • Extrapolation method for unmapped destination points.
      integer, parameter :: Enone = 0 ! none
      integer, parameter :: ExStoD = 1 ! nearear neighbor Src to Dst
      integer, parameter :: Eidavg = 2 ! inverse distance average
      integer, parameter :: Ecreep = 3 ! creep fill
      integer, parameter :: E2steps = 4 ! Turuncoglu two steps

      character (len=4), dimension(0:4) :: ExtrpType = &
      & (/ 'NONE', &
      & 'NS2D', &
      & 'IDAV', &
      & 'CREE', &
      & '2STP' /)
    • The number of levels to output for the extrapolation methods that fill levels, like creep fill (ESMF_EXTRAPMETHOD_CREEP). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels. For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding (ESMF Reference Manual, v 8.0.0).
      integer :: extrapNumLevels = 1
    • Interpolation connectors mask interaction flags.
      integer, parameter :: OverLand = 1
      integer, parameter :: OverOcean = 2
      integer, parameter :: OverAll = 3

      character (len=3), dimension(3) :: MaskType = &
      & (/ 'LND', &
      & 'OCN', &
      & 'ALL'/)
    • Coupling run mode: sequential or concurrent.
      integer, parameter :: Iseq = 1
      integer, parameter :: Ipar = 2

      character (len=10), dimension(2) :: RunMode = &
      & (/ 'SEQUENTIAL', &
      & 'CONCURRENT' /)
    • Compling standard input parameters filename.
      character (len=256) :: CinpName
    • ESM free-format run sequence configuration filename.
      character (len=256) :: CONFname
    • ROMS coupling YAML configuration filename.
      character (len=256), allocatable :: CPLname
    • Standard input filename for each coupled model, [Nmodels].
      character (len=256), allocatable :: INPname(:)
    • Standard output units and log messages filename for coupler and ESMF library.
      integer :: cplout = 77 ! coupling driver
      integer :: dataout = 77 ! data component

      character (len= 8), parameter :: ESMnameLog = 'log.esmf'
      character (len= 8), parameter :: CouplerLog = 'log.coupler'
    • Output NetCDF file used to store field snapshot attributes needed for time interpolation by the ESM component kernel during concurrent coupling.
      character (len=17), parameter :: AttFileName = ''
    • ESM single and double precision constants.
      integer (i4b), parameter :: MAPPED_MASK = 99_i4b
      integer (i4b), parameter :: UNMAPPED_MASK = 98_i4b

      real (dp), parameter :: MISSING_dp = 1.0E20_dp
      real (r4), parameter :: MISSING_r4 = 1.0E20_r4
      real (r8), parameter :: MISSING_r8 = 1.0E20_r8

      real (dp), parameter :: TOL_dp = 0.5E20_dp
      real (r4), parameter :: TOL_r4 = 0.5E20_r4
      real (r8), parameter :: TOL_r8 = 0.5E20_r8


The ROMS cap module contains a set of subroutines that are required by NUOPC. According to a predefined calling sequence, these subroutines are called by the NUOPC infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.

The initialization sequence is the most complex and is governed by the NUOPC technical rules. Details about the initialization sequence can be found in the NUOPC Reference Manual. The ROMS cap requires ESMF version 8 or higher.

ROMS_SetServices Entry point to the ROMS cap and the only public routine. It sets the ROMS component chared-object entry points for using NUOPC generic methods for initialize, run, and finalize.
ROMS_SetInitializeP1 ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.
ROMS_SetInitializeP2 ROMS component phase 2 initialization which initializes the ROMS component (ROMS_initialize), sets component grid (ROMS_SetGridArrays), and adds fields into import and export into respective states.
ROMS_DataInit Exports ROMS component fields during initialization or restart.
ROMS_SetClock Sets ROMS component date calendar, start and stop times, and coupling interval.
ROMS_SetRunClock Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.
ROMS_CheckImport Checks if ROMS component import field is at the correct time.
ROMS_SetGridArrays Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.
ROMS_SetStates Adds ROMS component export and import fields into its respective state.
ROMS_ModelAdvance Advances ROMS component for a coupling interval. It calls ROMS_Import and ROMS_Export routines.
ROMS_SetFinalize Finalizes ROMS component execution.
ROMS_Import Loads import fields into ROMS internal kernel arrays.
ROMS_Export Exports ROMS fields to other gridded components.
ROMS_Rotate Rotates vector components from computational grid to geographical EAST and NORTH directions or vice versa.

Other NUOPC cap modules are available within the ROMS native framework:

ESMF/NUOPC Modules Tested Description
Master/esmf_driver.h yes ESMF/NUOPC Master Driver
Master/esmf_esm.F yes Sets ESMF/NUOPC Services and RunSequence for each coupled component
Master/esmf_coupler.h yes Computes, Execute, and Release the Connectors between source and destination fields
Master/mod_esmf_esm.h yes Defines framework object structures and variables and includes support routines
Master/esmf_atm_coamps.h yes Coupled Ocean-Atmosphere Mesoscale Prediction System (COAMPS)
Master/esmf_atm_coamps.h yes Coupled Ocean-Atmosphere Mesoscale Prediction System (COAMPS)
Master/esmf_atm_regcm.h no ICTP Regional Climate Model (RegCM, Version 4.5)
Master/esmf_atm_wrf.h yes Weather Research and Forecasting model (WRF)
Master/esmf_data.F yes Generic DATA component via NetCDF files
Master/esmf_ice_cice.h no Los Alamos Sea Ice Model (CICE)
Master/esmf_roms.F yes Regional Ocean Modeling System (ROMS)
Master/esmf_wav_wam.h no ECMWF's Wave Model (WAM)
Master/esmf_wav_ww3.h no WaveWatch III (WW3), under development

Import and Export Fields

The export and import fields are configured and specified in the YAML file coupling_esmf_atm.yaml. Check the following wikiROMS page for detailed information. The user has full control of the coupling exchange fields metadata.

The ESMF/NUOPC library version 8.0.0 or higher supports native nesting capabilities. The NUOPC layer now allows coupling sets in the import and export states of the ESM components. A nested grid is represented as another ESMF_Grid object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The NUOPC connector can recognize the different grids and exchange fields to a connected ESM component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The NUOPC_AddNestedState routine is used to advertise the nested couple sets.

For example, the ROMS NOUPC cap module routine ROMS_SetInitializeP1 illustrates how easily the coupled sets is added to the ImportState and ExportState as shown below in the green blocks of code.

ImportCoupledStates.png ExportCoupledStates.png