https://www.myroms.org/wiki/api.php?action=feedcontributions&feedformat=atom&user=ArangoWikiROMS - User contributions [en]2026-04-30T21:31:06ZUser contributionsMediaWiki 1.42.3https://www.myroms.org/wiki/index.php?title=Documentation_Portal&diff=6688Documentation Portal2025-09-24T19:38:28Z<p>Arango: </p>
<hr />
<div>[[File:AGU Award ArangoMooreWilkin.png|1200px|frameless]]<br />
<br />
We are honored to receive the '''2025 AGU Edward A. Flinn III Award'''. For over 25 years, we have developed and shared the '''Regional Ocean Modeling System (ROMS)''' with the ocean modeling community, which now has thousands of users worldwide. Our work has supported the training and research of the next generation of ocean scientists and operational specialists. Building the advanced capabilities of '''ROMS''' truly requires a team. We are grateful to all the multidisciplinary scientists and colleagues who have contributed to '''ROMS''''s development, improvement, applications, dissertations, and hundreds of scientific papers in the literature.<br />
<br />
We want to express our gratitude to all the collaborators whose contributions have been invaluable. Specifically, '''Alexander F. Shchepetkin''', '''John C. Warner''', and the '''ROMS Adjoint Group''' for their assistance with various numerical algorithms. We also appreciate '''Kate Hedstrom''' and '''David Robertson''' for their help in managing the '''ROMS''' user community and developing the necessary cyberinfrastructure. We are grateful for all the support from funding agencies that made this possible. We extend our heartfelt thanks to the nominator(s), the writers who supported the nomination package, and the selection committee for bestowing this honor upon us.<br />
<br />
We dedicate this award to the '''ROMS community'''— Hernan, Andy, and John. <br />
<br />
<br />
----<br />
<br />
<br />
<div class="title">Documentation Portal</div><br />
<br />
{|class="main" width="99%"<br />
|style="vertical-align:top;width:33%" | <!-- Left Column Portal --><br />
'''Technical Documentation'''<br />
* [[Regional_Ocean_Modeling_System_(ROMS) | Introduction]]<br />
* [[Equations of Motion]]<br />
* [[Vertical S-coordinate]]<br />
* [[Terrain-Following Coordinate Transformation]]<br />
* [[Curvilinear Coordinates Transformation]]<br />
* [[Fractional Coordinate System (&xi; - &eta; space)]]<br />
* [[Time-stepping Schemes Review]]<br />
* [[Numerical Solution Technique]]<br />
* [[Nested Grids]]<br />
* [[Boundary Conditions]]<br />
* [[Horizontal Mixing]]<br />
* [[Vertical Mixing Parameterizations]]<br />
* [[Atmospheric Boundary Layer]]<br />
* [[Bottom Boundary Layer]]<br />
* [[Lagrangian Drifters]]<br />
* [[Balance Term Diagnostics]]<br />
* [[Biogeochemical Models]]<br />
* [[Sediment Model]]<br />
* [[Wave Model]]<br />
* [[Atmospheric Model]]<br />
* [[Model Coupling]]<br />
** [[NUOPC_Cap | ROMS Native ESMF/NUOPC Cap]]<br />
** [[NUOPC_Cap_UFS | ROMS Stand-Alone ESMF/NUOPC Cap]]<br />
* [[Model Diagnostics]]<br />
* [[Sea-Ice Model]]<br />
* [[Variational Data Assimilation]]<br />
* [[ROMS-JEDI_Intro| ROMS-JEDI]]<br />
** [[ROMS-JEDI_Compiling | Compiling]]<br />
** [[ROMS-JEDI_Implementation | Implementation]]<br />
** [[ROMS-JEDI_Observations | Observations]]<br />
** [[ROMS-JEDI_Unit_Test_Cases | Unit Test Cases]]<br />
* [[IO | ROMS I/O]]<br />
* [[Bibliography]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Middle Column Portal --><br />
'''User Guide'''<br />
* [[Getting Started]]<br />
* [[Tutorials]]<br />
* [[Nesting Examples]]<br />
* [[Frequently Asked Questions]]<br />
* [[Source Code]]<br />
** [[Git]]<br />
** [[Modules]]<br />
** [[Drivers]]<br />
** [[Subroutines]]<br />
** [[Variables]]<br />
** [[Functionals]]<br />
** [[Lateral Boundary Conditions]]<br />
* [[C Preprocessor]]<br />
** [[Options]]<br />
** [[cppdefs.h]]<br />
** [[globaldefs.h]]<br />
* [[Compiling and Linking]]<br />
** [[makefile]]<br />
** [[build_roms]]<br />
** [[my_build_paths]]<br />
** [[make]]<br />
** [[gmake]]<br />
** [[MakeDepend]]<br />
** [[External Libraries]]<br />
* [[Parallelization]]<br />
* [[Benchmarking]]<br />
* [[Matlab Scripts]]<br />
* [[Visualization]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Right Column Portal --><br />
'''Examples and Applications'''<br />
* [[Test Cases]]<br />
* [[Model Coupling ESMF | Coupling Examples]]<br />
* [[Lake_Jersey_Refinement_Example_Introduction | Nesting Examples]]<br />
* [[Application Set-up]]<br />
* [[Metadata Design]]<br />
** [[YAML_Parser | YAML Parser]]<br />
** [[coupling_esmf_atm.yaml]]<br />
** [[roms_cmeps.yaml]] <br />
** [[varinfo.yaml]]<br />
* [[Grid Generation]]<br />
* [[Initialization]]<br />
* [[Forcing]]<br />
** [[Atmospheric Forcing]]<br />
** [[Tidal Forcing]]<br />
** [[River Runoff]]<br />
* [[Input Parameter Files]]<br />
** [[roms.in]]<br />
** [[s4dvar.in]]<br />
** [[biology.in]]<br />
** [[sediment.in]]<br />
** [[floats.in]]<br />
** [[stations.in]]<br />
** [[ice.in]]<br />
** [[swan.in]]<br />
* [[Standard Output Files]]<br />
|}<br />
----<br />
<br />
<br />
<!--center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center--></div>Arangohttps://www.myroms.org/wiki/index.php?title=Documentation_Portal&diff=6687Documentation Portal2025-09-24T19:28:28Z<p>Arango: </p>
<hr />
<div>[[File:AGU Award ArangoMooreWilkin.png|1200px|frameless]]<br />
<br />
We are honored to receive the '''2025 AGU Edward A. Flinn III Award'''. For over 25 years, we have developed and shared the '''Regional Ocean Modeling System (ROMS)''' with the ocean modeling community, which now has thousands of users worldwide. Our work has supported the training and research of the next generation of ocean scientists and operational specialists. Building the advanced capabilities of '''ROMS''' truly requires a team. We are grateful to all the multidisciplinary scientists and colleagues who have contributed to '''ROMS''''s development, improvement, applications, dissertations, and hundreds of scientific papers in the literature.<br />
<br />
We want to express our gratitude to all the collaborators whose contributions have been invaluable. Specifically, '''Alexander F. Shchepetkin''', '''John C. Warner''', and the '''ROMS Adjoint Group''' for their assistance with various numerical algorithms. We also appreciate '''Kate Hedstrom''' and '''David Robertson''' for their help in managing the '''ROMS''' user community and developing the necessary cyberinfrastructure. We are grateful for all the support from funding agencies that made this possible. We extend our heartfelt thanks to the nominator(s), the writers who supported the nomination package, and the selection committee for bestowing this honor upon us.<br />
<br />
We dedicate this award to the '''ROMS community'''— Hernan, Andy, and John. <br />
<br />
<br />
<div class="title">Documentation Portal</div><br />
<br />
{|class="main" width="99%"<br />
|style="vertical-align:top;width:33%" | <!-- Left Column Portal --><br />
'''Technical Documentation'''<br />
* [[Regional_Ocean_Modeling_System_(ROMS) | Introduction]]<br />
* [[Equations of Motion]]<br />
* [[Vertical S-coordinate]]<br />
* [[Terrain-Following Coordinate Transformation]]<br />
* [[Curvilinear Coordinates Transformation]]<br />
* [[Fractional Coordinate System (&xi; - &eta; space)]]<br />
* [[Time-stepping Schemes Review]]<br />
* [[Numerical Solution Technique]]<br />
* [[Nested Grids]]<br />
* [[Boundary Conditions]]<br />
* [[Horizontal Mixing]]<br />
* [[Vertical Mixing Parameterizations]]<br />
* [[Atmospheric Boundary Layer]]<br />
* [[Bottom Boundary Layer]]<br />
* [[Lagrangian Drifters]]<br />
* [[Balance Term Diagnostics]]<br />
* [[Biogeochemical Models]]<br />
* [[Sediment Model]]<br />
* [[Wave Model]]<br />
* [[Atmospheric Model]]<br />
* [[Model Coupling]]<br />
** [[NUOPC_Cap | ROMS Native ESMF/NUOPC Cap]]<br />
** [[NUOPC_Cap_UFS | ROMS Stand-Alone ESMF/NUOPC Cap]]<br />
* [[Model Diagnostics]]<br />
* [[Sea-Ice Model]]<br />
* [[Variational Data Assimilation]]<br />
* [[ROMS-JEDI_Intro| ROMS-JEDI]]<br />
** [[ROMS-JEDI_Compiling | Compiling]]<br />
** [[ROMS-JEDI_Implementation | Implementation]]<br />
** [[ROMS-JEDI_Observations | Observations]]<br />
** [[ROMS-JEDI_Unit_Test_Cases | Unit Test Cases]]<br />
* [[IO | ROMS I/O]]<br />
* [[Bibliography]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Middle Column Portal --><br />
'''User Guide'''<br />
* [[Getting Started]]<br />
* [[Tutorials]]<br />
* [[Nesting Examples]]<br />
* [[Frequently Asked Questions]]<br />
* [[Source Code]]<br />
** [[Git]]<br />
** [[Modules]]<br />
** [[Drivers]]<br />
** [[Subroutines]]<br />
** [[Variables]]<br />
** [[Functionals]]<br />
** [[Lateral Boundary Conditions]]<br />
* [[C Preprocessor]]<br />
** [[Options]]<br />
** [[cppdefs.h]]<br />
** [[globaldefs.h]]<br />
* [[Compiling and Linking]]<br />
** [[makefile]]<br />
** [[build_roms]]<br />
** [[my_build_paths]]<br />
** [[make]]<br />
** [[gmake]]<br />
** [[MakeDepend]]<br />
** [[External Libraries]]<br />
* [[Parallelization]]<br />
* [[Benchmarking]]<br />
* [[Matlab Scripts]]<br />
* [[Visualization]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Right Column Portal --><br />
'''Examples and Applications'''<br />
* [[Test Cases]]<br />
* [[Model Coupling ESMF | Coupling Examples]]<br />
* [[Lake_Jersey_Refinement_Example_Introduction | Nesting Examples]]<br />
* [[Application Set-up]]<br />
* [[Metadata Design]]<br />
** [[YAML_Parser | YAML Parser]]<br />
** [[coupling_esmf_atm.yaml]]<br />
** [[roms_cmeps.yaml]] <br />
** [[varinfo.yaml]]<br />
* [[Grid Generation]]<br />
* [[Initialization]]<br />
* [[Forcing]]<br />
** [[Atmospheric Forcing]]<br />
** [[Tidal Forcing]]<br />
** [[River Runoff]]<br />
* [[Input Parameter Files]]<br />
** [[roms.in]]<br />
** [[s4dvar.in]]<br />
** [[biology.in]]<br />
** [[sediment.in]]<br />
** [[floats.in]]<br />
** [[stations.in]]<br />
** [[ice.in]]<br />
** [[swan.in]]<br />
* [[Standard Output Files]]<br />
|}<br />
----<br />
<br />
<br />
<!--center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center--></div>Arangohttps://www.myroms.org/wiki/index.php?title=Documentation_Portal&diff=6686Documentation Portal2025-09-24T19:27:02Z<p>Arango: </p>
<hr />
<div>[[File:AGU Award ArangoMooreWilkin.png|1200px|frameless]]<br />
<br />
We are honored to receive the '''2025 AGU Edward A. Flinn III Award'''. For over 25 years, we have developed and shared the '''Regional Ocean Modeling System (ROMS)''' with the ocean modeling community, which now has thousands of users worldwide. Our work has supported the training and research of the next generation of ocean scientists and operational specialists. Building the advanced capabilities of '''ROMS''' truly requires a team. We are grateful to all the multidisciplinary scientists and colleagues who have contributed to '''ROMS''''s development, improvement, applications, dissertations, and hundreds of scientific papers in the literature.<br />
<br />
We want to express our gratitude to all the collaborators whose contributions have been invaluable. Specifically, '''Alexander F. Shchepetkin''', '''John C. Warner''', and the '''ROMS Adjoint Group''' for their assistance with various numerical algorithms. We also appreciate '''Kate Hedstrom''' and '''David Robertson''' for their help in managing the '''ROMS''' user community and developing the necessary cyberinfrastructure. We are grateful for all the support from funding agencies that made this possible. We extend our heartfelt thanks to the nominator(s), the writers who supported the nomination package, and the selection committee for bestowing this honor upon us.<br />
<br />
We dedicate this award to the '''ROMS community'''— Hernan, Andy, and John. <br />
<br />
<br />
<div class="title">Documentation Portal</div><br />
<br />
{|class="main" width="99%"<br />
|style="vertical-align:top;width:33%" | <!-- Left Column Portal --><br />
'''Technical Documentation'''<br />
* [[Regional_Ocean_Modeling_System_(ROMS) | Introduction]]<br />
* [[Equations of Motion]]<br />
* [[Vertical S-coordinate]]<br />
* [[Terrain-Following Coordinate Transformation]]<br />
* [[Curvilinear Coordinates Transformation]]<br />
* [[Fractional Coordinate System (&xi; - &eta; space)]]<br />
* [[Time-stepping Schemes Review]]<br />
* [[Numerical Solution Technique]]<br />
* [[Nested Grids]]<br />
* [[Boundary Conditions]]<br />
* [[Horizontal Mixing]]<br />
* [[Vertical Mixing Parameterizations]]<br />
* [[Atmospheric Boundary Layer]]<br />
* [[Bottom Boundary Layer]]<br />
* [[Lagrangian Drifters]]<br />
* [[Balance Term Diagnostics]]<br />
* [[Biogeochemical Models]]<br />
* [[Sediment Model]]<br />
* [[Wave Model]]<br />
* [[Atmospheric Model]]<br />
* [[Model Coupling]]<br />
** [[NUOPC_Cap | ROMS Native ESMF/NUOPC Cap]]<br />
** [[NUOPC_Cap_UFS | ROMS Stand-Alone ESMF/NUOPC Cap]]<br />
* [[Model Diagnostics]]<br />
* [[Sea-Ice Model]]<br />
* [[Variational Data Assimilation]]<br />
* [[ROMS-JEDI_Intro| ROMS-JEDI]]<br />
** [[ROMS-JEDI_Compiling | Compiling]]<br />
** [[ROMS-JEDI_Implementation | Implementation]]<br />
** [[ROMS-JEDI_Observations | Observations]]<br />
** [[ROMS-JEDI_Unit_Test_Cases | Unit Test Cases]]<br />
* [[IO | ROMS I/O]]<br />
* [[Bibliography]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Middle Column Portal --><br />
'''User Guide'''<br />
* [[Getting Started]]<br />
* [[Tutorials]]<br />
* [[Nesting Examples]]<br />
* [[Frequently Asked Questions]]<br />
* [[Source Code]]<br />
** [[Git]]<br />
** [[Modules]]<br />
** [[Drivers]]<br />
** [[Subroutines]]<br />
** [[Variables]]<br />
** [[Functionals]]<br />
** [[Lateral Boundary Conditions]]<br />
* [[C Preprocessor]]<br />
** [[Options]]<br />
** [[cppdefs.h]]<br />
** [[globaldefs.h]]<br />
* [[Compiling and Linking]]<br />
** [[makefile]]<br />
** [[build_roms]]<br />
** [[my_build_paths]]<br />
** [[make]]<br />
** [[gmake]]<br />
** [[MakeDepend]]<br />
** [[External Libraries]]<br />
* [[Parallelization]]<br />
* [[Benchmarking]]<br />
* [[Matlab Scripts]]<br />
* [[Visualization]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Right Column Portal --><br />
'''Examples and Applications'''<br />
* [[Test Cases]]<br />
* [[Model Coupling ESMF | Coupling Examples]]<br />
* [[Lake_Jersey_Refinement_Example_Introduction | Nesting Examples]]<br />
* [[Application Set-up]]<br />
* [[Metadata Design]]<br />
** [[YAML_Parser | YAML Parser]]<br />
** [[coupling_esmf_atm.yaml]]<br />
** [[roms_cmeps.yaml]] <br />
** [[varinfo.yaml]]<br />
* [[Grid Generation]]<br />
* [[Initialization]]<br />
* [[Forcing]]<br />
** [[Atmospheric Forcing]]<br />
** [[Tidal Forcing]]<br />
** [[River Runoff]]<br />
* [[Input Parameter Files]]<br />
** [[roms.in]]<br />
** [[s4dvar.in]]<br />
** [[biology.in]]<br />
** [[sediment.in]]<br />
** [[floats.in]]<br />
** [[stations.in]]<br />
** [[ice.in]]<br />
** [[swan.in]]<br />
* [[Standard Output Files]]<br />
|}<br />
----<br />
<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center></div>Arangohttps://www.myroms.org/wiki/index.php?title=File:AGU_Award_ArangoMooreWilkin.png&diff=6685File:AGU Award ArangoMooreWilkin.png2025-09-24T19:18:33Z<p>Arango: 2025 Edward A. Flinn III Award</p>
<hr />
<div>== Summary ==<br />
2025 Edward A. Flinn III Award</div>Arangohttps://www.myroms.org/wiki/index.php?title=Git&diff=6670Git2024-12-30T16:48:53Z<p>Arango: </p>
<hr />
<div><div class="title">Git</div><br />
'''ROMS''' source code is now distributed using [https://git-scm.com Git]. The '''GitHub''' repository includes the full history since 2002 of changes to the '''ROMS''' source code. There are command line and GUI Git clients available for nearly every operating system and a list of popular clients can be found [https://git-scm.com/downloads/guis here]. This page will help you get started with downloading '''ROMS''' with Git.<br />
<br />
==Git Overview==<br />
Git is a free and open-source distributed version control system designed to handle everything from small to massive projects quickly and efficiently. All the ROMS files are stored in a Git repository on <span class="twilightBlue">github.com/myroms</span>.<br />
<br />
This Git repository is the official version of the code, which only developers are allowed to change. Users should download the ROMS code to their local machines using a '''git client'''. Don't attempt to use a regular web browser to browse or download files from the Git repository—there are much better tools for interacting with the code repository.<br />
<br />
We strongly recommend users check out the current <span class="red">develop</span> branch, which has the most recent updates and bug fixes. The ''tags'' are mainly kept as a historical record of stable releases after major code upgrades.<br />
<br />
If you are making changes yourself, keep them in a separate branch, leaving the <span class="red">develop</span> branch to track changes from the source. Please check the [https://github.com/myroms/roms/wiki/Useful-GIT-Commands ROMS GitHub Wiki] for useful Git commands.<br />
<br />
Below is a general description of how Git works. For more detailed information, please review the [https://git-scm.com/book/en/v2 Pro Git] book. We have not tried GUI clients but may add brief how-tos for the most popular GUIs later.<br />
<br />
==Configuring GIT==<br />
<br />
Before downloading any of '''ROMS''' repositories, ensure that your '''~/.gitconfig''' has the appropriate '''git-lfs''' configuration for correctly downloading some '''''roms_test''''' input and observation NetCDF files. Otherwise, the '''Test Cases''' requiring input NetCDF files will fail. The '''Git LFS''' is a command line extension and specification for managing large files with '''Git'''. A sample of the configuration file looks like this:<div class="box">more ~/.gitconf<br /><br /> [User]<br /> name = GivenName MiddleName FamilyName<br /> email = your@email<br /> [credential]<br /> helper = cache --timeout=7200<br /> helper = store --file ~/.my-credentials<br /> [filter "lfs"]<br /> clean = git-lfs clean -- %f<br /> smudge = git-lfs smudge -- %f<br /> process = git-lfs filter-process<br /> required = true</div><br />
<br />
Alternatively, you may execute '''git lfs pull''' at your location of this repository to download a viable version of the '''Git LFS''' files from the remote repository in '''GitHub'''. Also, to automatically add the '''LFS''' filter to your existing '''~/.gitconfig''', you could use '''git lfs install''' from anywhere in your computer directories.<br />
<br />
==Downloading ROMS==<br />
{{warning}}'''WARNING:''' It is strongly suggested that you clone the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like <span class="red">dos2unix</span> or <span class="red">recode</span>. Even with these utilities you may still have problems compiling ROMS.<br />
<br />
<br />
In order download source code from a git repository, '''git client''' software must be installed on your local machine. Most Linux distributions come with git, so shell commands may be used without installing additional software. The general form of git commands is:<br />
<br />
<span class="red">git</span> action <repository><br />
<br />
To check-out the files from the ROMS repository '''develop''' (latest version), enter (notice https instead of http):<br />
<br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms.git</nowiki> MyDir</span><br />
<br />
where '''MyDir''' is the destination directory on your local computer. It will be created if not found. If '''MyDir''' is omitted the code will be downloaded to a directory named '''roms'''. You only clone once because Git will keep track of the source, destination and a bunch of other information. For more detail on command line use and syntax, see the [https://git-scm.com/book/en/v2 Pro Git] book.<br />
<br />
The idealized and realistic ROMS Test Cases and the Matlab processing software can be downloaded from:<br />
<br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms_test.git </nowiki></span><br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms_matlab.git</nowiki></span><br />
<br />
We highly recommend that Users clone all the ROMS repositories from the same root directory in their computer and define the '''ROMS_ROOT_DIR''' variable in their computer shell login environment, specifying where the User cloned/downloaded the ROMS source code, Test Cases, and Matlab processing software. That is, use the value of pwd from the computer directory where you executed git clone https://github.com/myroms/roms.git<br />
<br />
For bash shells:<br />
export ROMS_ROOT_DIR=MyDownlodLocationDirectory<br />
<br />
For csh/tcsh shells:<br />
setenv ROMS_ROOT_DIR MyDownlodLocationDirectory<br />
<br />
The build scripts will use this environmental variable when compiling any of the ROMS Test Cases without the need to customize the location of the ROMS source code. Also, it is used for loading the path of Matlab scripts in the startup.m configuration file.<br />
<br />
===Git Hash Revision Information=== <br />
{{note}}'''Note:''' While this is not a required step, we highly recommend it to make bug reporting and tracking easier.<br />
<br />
<br />
In order to get Git hash revision information in the header for each file, we have created a Git filter that will insert the git hash of the last time a file was modified, similar to the Subversion '''''$Id$''''' keyword. In order for the filter to work, it has to be explicitly enabled by the user. After the git clone command from above completes change into <span class="limeGreen">MyDir</span> and execute the following:<br />
<br />
> <span class="red">git config filter.id.smudge ".git_filters/id.smudge %f"</span><br />
> <span class="red">git config filter.id.clean ".git_filters/id.clean %f"</span><br />
<br />
In order to get the Git Id hashes to fill in, you will need to force the smudge filter to run on the source code you have already downloaded. This takes several minutes but only needs to be done once.<br />
<br />
For '''bash''' shells execute: <br />
<br />
> <span class="red">rm .git/index</span><br />
> <span class="red">git checkout HEAD -- "$(git rev-parse --show-toplevel)"</span><br />
<br />
For '''csh/tcsh''' shells execute:<br />
<br />
> <span class="red">rm .git/index</span><br />
> <span class="red">git checkout HEAD -- "`git rev-parse --show-toplevel`"</span><br />
<br />
If you look at the top of any ROMS source file, you will see the Git commit hash for the last time that file was modified. For example, the first few lines of <span class="limeGreen">ROMS/Modules/mod_ncparam.F</span> look like this:<div class="box">#include "cppdefs.h"<br /> MODULE mod_ncparam<br />!<br />!git $Id: caf970e3 2024-08-09 17:15:26 UTC arango $<br />!================================================== Hernan G. Arango ===<br />! Copyright (c) 2002-2024 The ROMS/TOMS Group !<br />! Licensed under a MIT/X style license !<br />! See License_ROMS.md !<br />!=======================================================================</div><br />
<br />
You can then use the 8 character (hexidecimal) Git hash to show more information about that code update:<div class="box">> <span class="red">git show --name-only caf970e3</span></div><br />
<br />
===Updates===<br />
<br />
Now and again, you might feel the urge to get up to speed with the latest changes that have been made to the ROMS repository. When that happens, simply go to the directory that was "MyDir" above and type:<br />
<br />
> <span class="red">git pull</span><br />
<br />
Git will remember where you checked out from before and see if a newer revision exists. If so, it will download and apply all the relevant changes.<br />
<br />
==Managing Your Own Modifications==<br />
This assumes that you have a fresh clone of the myroms repository on the '''develop''' branch. You want to keep '''develop''' as a pure copy of the source version and keep your own changes in say the '''arctic''' branch. Start by creating a branch and switching to it:<br />
<br />
> <span class="red">git branch arctic<br /></span>> <span class="red">git checkout arctic</span><br />
<br />
Now you can make whatever modifications you like (and test them out). To see what changed, you can use <span class="red">git status</span> and <span class="red">git diff</span>. To save your changes, do a:<br />
<br />
> <span class="red">git commit -a</span><br />
<br />
Though if you add new files you will have to <span class="red">git add</span> them first.<br />
<br />
===Getting the Updates===<br />
It is easy to fetch and merge the updates. Start by making sure your directory has been cleanly checked in with <span class="red">git status</span>. Then you can update your '''develop''' branch:<br />
<br />
> <span class="red">git checkout develop</span><br />> <span class="red">git pull</span><br />
<br />
Then bring the changes into your '''arctic''' branch:<br />
<br />
> <span class="red">git checkout arctic</span><br />> <span class="red">git merge develop</span><br />
<br />
This will bring in everything that changed since your last <span class="red">git pull</span>, so you might find it easier to keep on top of things by doing this often, not putting it off for years. You can also bring in changes one at a time with <span class="red">git cherry-pick</span>. Again, check the Pro Git book for much more information about all of these operations.<br />
<br />
Note that this will save your '''arctic''' branch locally, under the .git directory. You can back this up as you would any other important files you have.<br />
<br />
==Useful Git Commands==<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|'''Command'''<br />
|'''Description'''<br />
|-<br />
| '''git add''' <_FileName_><br />
| Add a file to the repository staging area <br />
|-<br />
| '''git branch'''<br />
| List all local branches (the asterisk denotes the current branch)<br />
|-<br />
| '''git branch -a'''<br />
| List all local and remote branches<br />
|-<br />
| '''git branch -d''' <_BranchName_><br />
| Delete a local branch<br />
|-<br />
| '''git branch -D''' <_BranchName_><br />
| Delete a local branch forcefully<br />
|-<br />
| '''git branch -m''' <_OldName_> <_NewName_><br />
| Rename a local branch<br />
|-<br />
| '''git checkout -b''' <_BranchName_><br />
| Create a new local branch and switch to it<br />
|-<br />
| '''git checkout''' <_BranchName_><br />
| Switch to an existing local or remote branch<br />
|-<br />
| '''git clone''' <_RepositoryURL_><br />
| Clone a public repository<br />
|-<br />
| '''git commit -am''' "<_Message_>"<br />
| Commit changes to all files<br />
|-<br />
| '''git commit -am''' "<_Message_>" '''-m''' "<_MessageURL_>"<br />
| Commit changes to all files with two message lines<br />
|-<br />
| '''git commit --amend -m''' "<_NewMessage_>"<br />
| Amend previous commit message<br />
|-<br />
| '''git diff'''<br />
| Show all changes between HEAD and working branch<br />
|-<br />
| '''git diff''' <_FileName_><br />
| Show changes for a specific file between HEAD and working branch<br />
|-<br />
| '''git difftool -d''' <_BranchName1_> <_BranchName2_><br />
| Compare the difference between two branches with '''`KDIFF3`'''<br />
|-<br />
| '''git fetch'''<br />
| Retrieve new work done by other people<br />
|-<br />
| '''git log'''<br />
| View repository changes<br />
|-<br />
| '''git log --oneline'''<br />
| Show the list of commits in one-line format<br />
|-<br />
| '''git log --summary'''<br />
| View repository detailed changes<br />
|-<br />
| '''git merge --no-ff --no-commit''' <_BranchName_><br />
| Merge a branch into the active branch<br />
|-<br />
| '''git pull'''<br />
| Update local repository or branch to the newest origin commit<br />
|-<br />
| '''git push origin --delete''' <_BranchName_><br />
| Delete a remote branch<br />
|-<br />
| '''git push -u origin''' <_BranchName_><br />
| Push changes to the remote repository (and remember the branch)<br />
|-<br />
| '''git reset --hard HEAD~1'''<br />
| Undo last commit<br />
|-<br />
| '''git revert''' <_CommitID_><br />
| Revert commit changes<br />
|-<br />
| '''git status'''<br />
| Check/display changes to the repository or particular branch<br />
|-<br />
| '''git tag'''<br />
| List all tags<br />
|}<br />
<br />
<br />
==Complicated Commands==<br />
<br />
If a branch is renamed in a repository on GitHub, the local clone on a computer needs to updated:<div class="box">git branch -m OldName NewName<br />git fetch origin<br />git branch -u origin/NewName NewName<br />git remote set-head origin -a</div><br />
<br />
How to merge changes in the '''feature/name1''' branch into the '''feature/name2''' branch:<div class="box">git checkout feature/name1<br />git pull<br /><br />git checkout feature/name2<br />git pull<br />git merge --no-ff --no-commit feature/name1<br />git commit -am "Merging feature/name1 into feature/name2" -m "MessageURL"<br />git difftool -d feature/name1 feature/name2<br />git push -u origin feature/name2</div></div>Arangohttps://www.myroms.org/wiki/index.php?title=Git&diff=6669Git2024-12-30T16:44:54Z<p>Arango: /* Git Overview */</p>
<hr />
<div><div class="title">Git</div><br />
ROMS source code is now distributed using [https://git-scm.com Git]. The [https://subversion.apache.org Subversion](SVN) repository is being phased out and will no longer be updated starting January 1<sup>st</sup>, 2025. The GitHub repository includes the full history of changes to the ROMS source code. There are command line and GUI Git clients available for nearly every operating system and a list of popular clients can be found [https://git-scm.com/downloads/guis here]. This page will help you get started with downloading ROMS with Git.<br />
<br />
<br />
{{note}}'''Note:''' For instructions using the deprecated and soon to be discontinued myroms.org git repository, click [[Talk:Git|here]]. <br />
<br />
<br />
{{note}}'''Note:''' Details on using the deprecated and soon to be discontinued Subversion repository can be found [[Subversion|here]].<br />
<br />
<br />
==Git Overview==<br />
Git is a free and open-source distributed version control system designed to handle everything from small to huge projects quickly and efficiently. All the ROMS files are stored in a Git repository on <span class="twilightBlue">github.com/myroms</span>.<br />
<br />
This Git repository is the official version of the code, which only developers are allowed to change. Users should download the ROMS code to their local machines using a '''git client'''. Don't attempt to use a regular web browser to browse or download files from the Git repository—there are much better tools for interacting with the code repository.<br />
<br />
We strongly recommend users check out the current <span class="red">develop</span> branch, which has the most recent updates and bug fixes. The ''tags'' are mainly kept as a historical record of stable releases after major code upgrades.<br />
<br />
If you are making changes yourself, keep them in a separate branch, leaving the <span class="red">develop</span> branch to track changes from the source. Please check the [https://github.com/myroms/roms/wiki/Useful-GIT-Commands ROMS GitHub Wiki] for useful Git commands.<br />
<br />
Below is a general description of how Git works. For more detailed information, please review the [https://git-scm.com/book/en/v2 Pro Git] book. We have not tried GUI clients but may add brief how-tos for the most popular GUIs later.<br />
<br />
==Configuring GIT==<br />
<br />
Before downloading any of '''ROMS''' repositories, ensure that your '''~/.gitconfig''' has the appropriate '''git-lfs''' configuration for correctly downloading some '''''roms_test''''' input and observation NetCDF files. Otherwise, the '''Test Cases''' requiring input NetCDF files will fail. The '''Git LFS''' is a command line extension and specification for managing large files with '''Git'''. A sample of the configuration file looks like this:<div class="box">more ~/.gitconf<br /><br /> [User]<br /> name = GivenName MiddleName FamilyName<br /> email = your@email<br /> [credential]<br /> helper = cache --timeout=7200<br /> helper = store --file ~/.my-credentials<br /> [filter "lfs"]<br /> clean = git-lfs clean -- %f<br /> smudge = git-lfs smudge -- %f<br /> process = git-lfs filter-process<br /> required = true</div><br />
<br />
Alternatively, you may execute '''git lfs pull''' at your location of this repository to download a viable version of the '''Git LFS''' files from the remote repository in '''GitHub'''. Also, to automatically add the '''LFS''' filter to your existing '''~/.gitconfig''', you could use '''git lfs install''' from anywhere in your computer directories.<br />
<br />
==Downloading ROMS==<br />
{{warning}}'''WARNING:''' It is strongly suggested that you clone the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like <span class="red">dos2unix</span> or <span class="red">recode</span>. Even with these utilities you may still have problems compiling ROMS.<br />
<br />
<br />
In order download source code from a git repository, '''git client''' software must be installed on your local machine. Most Linux distributions come with git, so shell commands may be used without installing additional software. The general form of git commands is:<br />
<br />
<span class="red">git</span> action <repository><br />
<br />
To check-out the files from the ROMS repository '''develop''' (latest version), enter (notice https instead of http):<br />
<br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms.git</nowiki> MyDir</span><br />
<br />
where '''MyDir''' is the destination directory on your local computer. It will be created if not found. If '''MyDir''' is omitted the code will be downloaded to a directory named '''roms'''. You only clone once because Git will keep track of the source, destination and a bunch of other information. For more detail on command line use and syntax, see the [https://git-scm.com/book/en/v2 Pro Git] book.<br />
<br />
The idealized and realistic ROMS Test Cases and the Matlab processing software can be downloaded from:<br />
<br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms_test.git </nowiki></span><br />
> <span class="red">git clone <nowiki>https://github.com/myroms/roms_matlab.git</nowiki></span><br />
<br />
We highly recommend that Users clone all the ROMS repositories from the same root directory in their computer and define the '''ROMS_ROOT_DIR''' variable in their computer shell login environment, specifying where the User cloned/downloaded the ROMS source code, Test Cases, and Matlab processing software. That is, use the value of pwd from the computer directory where you executed git clone https://github.com/myroms/roms.git<br />
<br />
For bash shells:<br />
export ROMS_ROOT_DIR=MyDownlodLocationDirectory<br />
<br />
For csh/tcsh shells:<br />
setenv ROMS_ROOT_DIR MyDownlodLocationDirectory<br />
<br />
The build scripts will use this environmental variable when compiling any of the ROMS Test Cases without the need to customize the location of the ROMS source code. Also, it is used for loading the path of Matlab scripts in the startup.m configuration file.<br />
<br />
===Git Hash Revision Information=== <br />
{{note}}'''Note:''' While this is not a required step, we highly recommend it to make bug reporting and tracking easier.<br />
<br />
<br />
In order to get Git hash revision information in the header for each file, we have created a Git filter that will insert the git hash of the last time a file was modified, similar to the Subversion '''''$Id$''''' keyword. In order for the filter to work, it has to be explicitly enabled by the user. After the git clone command from above completes change into <span class="limeGreen">MyDir</span> and execute the following:<br />
<br />
> <span class="red">git config filter.id.smudge ".git_filters/id.smudge %f"</span><br />
> <span class="red">git config filter.id.clean ".git_filters/id.clean %f"</span><br />
<br />
In order to get the Git Id hashes to fill in, you will need to force the smudge filter to run on the source code you have already downloaded. This takes several minutes but only needs to be done once.<br />
<br />
For '''bash''' shells execute: <br />
<br />
> <span class="red">rm .git/index</span><br />
> <span class="red">git checkout HEAD -- "$(git rev-parse --show-toplevel)"</span><br />
<br />
For '''csh/tcsh''' shells execute:<br />
<br />
> <span class="red">rm .git/index</span><br />
> <span class="red">git checkout HEAD -- "`git rev-parse --show-toplevel`"</span><br />
<br />
If you look at the top of any ROMS source file, you will see the Git commit hash for the last time that file was modified. For example, the first few lines of <span class="limeGreen">ROMS/Modules/mod_ncparam.F</span> look like this:<div class="box">#include "cppdefs.h"<br /> MODULE mod_ncparam<br />!<br />!git $Id: caf970e3 2024-08-09 17:15:26 UTC arango $<br />!================================================== Hernan G. Arango ===<br />! Copyright (c) 2002-2024 The ROMS/TOMS Group !<br />! Licensed under a MIT/X style license !<br />! See License_ROMS.md !<br />!=======================================================================</div><br />
<br />
You can then use the 8 character (hexidecimal) Git hash to show more information about that code update:<div class="box">> <span class="red">git show --name-only caf970e3</span></div><br />
<br />
===Updates===<br />
<br />
Now and again, you might feel the urge to get up to speed with the latest changes that have been made to the ROMS repository. When that happens, simply go to the directory that was "MyDir" above and type:<br />
<br />
> <span class="red">git pull</span><br />
<br />
Git will remember where you checked out from before and see if a newer revision exists. If so, it will download and apply all the relevant changes.<br />
<br />
==Managing Your Own Modifications==<br />
This assumes that you have a fresh clone of the myroms repository on the '''develop''' branch. You want to keep '''develop''' as a pure copy of the source version and keep your own changes in say the '''arctic''' branch. Start by creating a branch and switching to it:<br />
<br />
> <span class="red">git branch arctic<br /></span>> <span class="red">git checkout arctic</span><br />
<br />
Now you can make whatever modifications you like (and test them out). To see what changed, you can use <span class="red">git status</span> and <span class="red">git diff</span>. To save your changes, do a:<br />
<br />
> <span class="red">git commit -a</span><br />
<br />
Though if you add new files you will have to <span class="red">git add</span> them first.<br />
<br />
===Getting the Updates===<br />
It is easy to fetch and merge the updates. Start by making sure your directory has been cleanly checked in with <span class="red">git status</span>. Then you can update your '''develop''' branch:<br />
<br />
> <span class="red">git checkout develop</span><br />> <span class="red">git pull</span><br />
<br />
Then bring the changes into your '''arctic''' branch:<br />
<br />
> <span class="red">git checkout arctic</span><br />> <span class="red">git merge develop</span><br />
<br />
This will bring in everything that changed since your last <span class="red">git pull</span>, so you might find it easier to keep on top of things by doing this often, not putting it off for years. You can also bring in changes one at a time with <span class="red">git cherry-pick</span>. Again, check the Pro Git book for much more information about all of these operations.<br />
<br />
Note that this will save your '''arctic''' branch locally, under the .git directory. You can back this up as you would any other important files you have.<br />
<br />
==Useful Git Commands==<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|'''Command'''<br />
|'''Description'''<br />
|-<br />
| '''git add''' <_FileName_><br />
| Add a file to the repository staging area <br />
|-<br />
| '''git branch'''<br />
| List all local branches (the asterisk denotes the current branch)<br />
|-<br />
| '''git branch -a'''<br />
| List all local and remote branches<br />
|-<br />
| '''git branch -d''' <_BranchName_><br />
| Delete a local branch<br />
|-<br />
| '''git branch -D''' <_BranchName_><br />
| Delete a local branch forcefully<br />
|-<br />
| '''git branch -m''' <_OldName_> <_NewName_><br />
| Rename a local branch<br />
|-<br />
| '''git checkout -b''' <_BranchName_><br />
| Create a new local branch and switch to it<br />
|-<br />
| '''git checkout''' <_BranchName_><br />
| Switch to an existing local or remote branch<br />
|-<br />
| '''git clone''' <_RepositoryURL_><br />
| Clone a public repository<br />
|-<br />
| '''git commit -am''' "<_Message_>"<br />
| Commit changes to all files<br />
|-<br />
| '''git commit -am''' "<_Message_>" '''-m''' "<_MessageURL_>"<br />
| Commit changes to all files with two message lines<br />
|-<br />
| '''git commit --amend -m''' "<_NewMessage_>"<br />
| Amend previous commit message<br />
|-<br />
| '''git diff'''<br />
| Show all changes between HEAD and working branch<br />
|-<br />
| '''git diff''' <_FileName_><br />
| Show changes for a specific file between HEAD and working branch<br />
|-<br />
| '''git difftool -d''' <_BranchName1_> <_BranchName2_><br />
| Compare the difference between two branches with '''`KDIFF3`'''<br />
|-<br />
| '''git fetch'''<br />
| Retrieve new work done by other people<br />
|-<br />
| '''git log'''<br />
| View repository changes<br />
|-<br />
| '''git log --oneline'''<br />
| Show the list of commits in one-line format<br />
|-<br />
| '''git log --summary'''<br />
| View repository detailed changes<br />
|-<br />
| '''git merge --no-ff --no-commit''' <_BranchName_><br />
| Merge a branch into the active branch<br />
|-<br />
| '''git pull'''<br />
| Update local repository or branch to the newest origin commit<br />
|-<br />
| '''git push origin --delete''' <_BranchName_><br />
| Delete a remote branch<br />
|-<br />
| '''git push -u origin''' <_BranchName_><br />
| Push changes to the remote repository (and remember the branch)<br />
|-<br />
| '''git reset --hard HEAD~1'''<br />
| Undo last commit<br />
|-<br />
| '''git revert''' <_CommitID_><br />
| Revert commit changes<br />
|-<br />
| '''git status'''<br />
| Check/display changes to the repository or particular branch<br />
|-<br />
| '''git tag'''<br />
| List all tags<br />
|}<br />
<br />
<br />
==Complicated Commands==<br />
<br />
If a branch is renamed in a repository on GitHub, the local clone on a computer needs to updated:<div class="box">git branch -m OldName NewName<br />git fetch origin<br />git branch -u origin/NewName NewName<br />git remote set-head origin -a</div><br />
<br />
How to merge changes in the '''feature/name1''' branch into the '''feature/name2''' branch:<div class="box">git checkout feature/name1<br />git pull<br /><br />git checkout feature/name2<br />git pull<br />git merge --no-ff --no-commit feature/name1<br />git commit -am "Merging feature/name1 into feature/name2" -m "MessageURL"<br />git difftool -d feature/name1 feature/name2<br />git push -u origin feature/name2</div></div>Arangohttps://www.myroms.org/wiki/index.php?title=External_Libraries&diff=6668External Libraries2024-12-28T02:03:43Z<p>Arango: /* ARPACK (ARnoldi PACKage) */</p>
<hr />
<div><div class="title">External Libraries</div><br />
<br />
Depending on the ROMS configuration, several third-party libraries are required for linking and compiling an application.<br />
<br />
==ARPACK (ARnoldi PACKage)==<br />
Serial and parallel legacy libraries applied to solve large eigenvalue problems using either the Implicitly Restarted Arnoldi Method ('''IRAM''') for sparse matrices or the Lanczos algorithm for symmetric matrices. It includes a subset of the '''BLAS''' and '''LAPACK''' libraries. Some of its functions are used in 4D-Var and the adjoint-based stability analysis propagators.<br />
<br />
To obtain '''ARPACK''', execute the following:<br />
<div class="box">> git clone https://github.com/myroms/roms_libs.git<br />> cd roms_libraries/ARPACK</div><br />
To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.<br />
<br />
<br />
{{note}} '''Note:''' Prior to SVN revision 1080 (July 23, 2021), '''ARPACK''' was distributed in the ROMS '''Lib/ARPACK''' subdirectory. For more information about why this was changed see TRAC ticket [https://www.myroms.org/projects/src/ticket/891 #891]<br />
<br />
==ESMF (Earth System Modeling Framework)==<br />
High-performance, open-source library for grid interpolation, remapping, and data exchange between coupled Earth System Model ('''ESM''') components. It includes the '''NUOPC''' (National Unified Operation Prediction Capability) layer that provides templates and protocols for sequential and concurrent coupling between ESM components. The '''NUOPC''' ''cap'' file is a Fortran module layer that sits on top of each '''ESM''' component that provides the protocols and methods to interact and share data in a coupled system.<br />
<br />
ROMS supports version 8.0 or higher since the '''NUOPC''' layer includes the native CoupleSets for '''ESM''' applications with nested grids. Various types of connections are possible, like '''nest-to-nest''', '''fine-to-coarse''', or '''coarse-to-fine'''. The user decides which nested grids to connect.<br />
<br />
*Download from github:<div class="box">> git clone https://github.com/esmf-org/esmf.git esmf<br />> cd esmf<br />> git checkout tags/ESMF_8_1_0</div><br />
<br />
*Configure with environment variables:<div class="box">> export ESMF_DIR=/projects/dmcs_1/src/esmf<br />> export ESMF_COMM=mvapich2<br />> export ESMF_COMPILER=intel<br />> export ESMF_BOPT=O<br />> export ESMF_OPTLEVEL=3<br />> export ESMF_ABI=64<br />> export ESMF_INSTALL_PREFIX=/ESMF/install/dir ESMF_NETCDF="/path/to/netcdf/bin/nc-config"<br />> export ESMF_NETCDF_LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lcurl -lhdf5_hl -lhdf5 -lm -lz -lcurl"<br />> export ESMF_NETCDF_LIBPATH="/path/to/netcdf/lib /path/to/hdf5/lib"<br />> export ESMF_SHARED_LIB_BUILD=OFF<br />> export ESMF_MPIRUN=/path/to/cluster/submission/script</div>{{note}} Note that ''ESMF_MPIRUN'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-np''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -np N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-np" ]<br />then<br /> echo "Usage: mpirun -np # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div><br />
<br />
*Build, test, and install:<div class="box">> make -j 10<br />> make all_tests<br />> make install</div><br />
<br />
==MCT (Model Coupling Toolkit)==<br />
<br />
Open-source library distributed as a set of Fortran-90 modules for constructing a coupled model system from Earth System Model (ESM) components ([http://www-unix.mcs.anl.gov/mct Larson et al., 2004]). Each component model has its own grid and runs on its own set of processors. The MCT library provides protocols for decomposition and allocation of model grids among different processors, efficient transfer of data fields between the different models, and interpolation algorithms for the data fields that are transferred.<br />
<br />
The MCT library is used to couple ROMS atmosphere models (like WRF) and wave models (like SWAN and WW3). For example, SWAN sends to ROMS arrays of wave height, wavelength, average wave periods at the surface and near the bottom, wave propagation direction, near-bottom orbital velocity, and wave-energy dissipation rate. ROMS provides SWAN arrays of water depth, sea-surface elevation, and current velocity. Data exchange between SWAN and ROMS occurs at user-defined synchronization intervals. The frequency of data exchange depends on the application. If the exchanged fields fluctuate rapidly, more frequent synchronization is required. However, data exchange increases run time, so experience is required to determine the optimum synchronization interval for each application.<br />
<br />
MCT is available via github:<br />
<br />
<div class="box">> git clone https://github.com/MCSclimate/MCT<br />> cd MCT<br />> ./configure<br />> make</div><br />
<br />
==NetCDF (Network Common Data Form)==<br />
<br />
Open-source, machine-independent libraries for storing self-describing scientific data into portable, scalable, appendable, and sharable computer files. It provides '''API''' in several computer languages to create files, to write data into a file, to read data from the file, and access information about the dataset dimensions, variables, and attributes. <br />
<br />
ROMS uses '''NetCDF''' for all its input and output data management. Its '''NetCDF3''' or '''NetCDF4''' type files can be processed using the standard library developed and distributed [http://doi.org/10.5065/D6H70CW6 UCAR/Unidata], the Parallel-IO ('''PIO''') developed at ANL/NCAR ([https://doi.org/10.1177/1094342011428143 Dennis ''et al''., 2012]; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE Hartnett and Edwards, 2021]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library available in [https://e3sm.org/scorpio-parallel-io-library E3SM]. We have migrated to the Fortran-90 '''API''' interface, which consists of a library and one or two module files. See the [[makefile]] and [[build_roms| build script]] for how to tell ROMS where these are located. You will have to use a copy of the library and module files that were compiled using the same compiler you are using to compile ROMS - this is especially true for the module files.<br />
<br />
===Building NetCDF-4/HDF5===<br />
NetCDF-4 uses HDF5 for it's underlying data format so you first need to build HDF5. We recommend 1.10.6 because 1.10.7 causes errors in the NetCDF-C test suite. Here we will describe how to build parallel versions for use on an HPC cluster. We will note what changes are necessary if you are building for a system that doesn't have parallel I/O.<br />
<br />
====Building HDF5====<br />
* First, download '''HDF5''' source code as a gzipped tarball or via git clone:<div class="box">> wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.6/src/hdf5-1.10.6.tar.gz</div>or<div class="box">> git clone https://github.com/HDFGroup/hdf5<br />> cd hdf5<br />> git checkout tags/hdf5-1_10_6<br />> autoconf -i</div><br />
<br />
* Configure '''HDF5''':<div class="box">> ./configure CC=mpicc CXX=mpicxx FC=mpifort \<br /> RUNPARALLEL="srun --nodes=1 --ntasks=6 --cpus-per-task=1 --time=00:13:00 --export=ALL " \<br /> --prefix=/path/to/parallel/hdf5/1.10.6 --enable-static-exec --enable-shared=no \<br /> --enable-parallel --disable-silent-rules --enable-fortran</div>{{note}} '''Note:''' ''RUNPARALLEL'' is only needed if your system uses a scheduler to run MPI jobs. The example here is for a SLURM system but you will need to adjust this for your cluster.<br /><br />{{note}} '''Note:''' To compile '''serial''', remove ''RUNPARALLEL=...'' and ''--enable-parallel'' and change the CC, CXX, and FC to the corresponding serial compilers (e.g. gcc, g++, and gfortran).<br /><br />{{note}} '''Note:''' If you would like '''shared libraries''' to be built, also remove ''--enable-shared=no''.<br /><br /><br />
<br />
* Build, test, and install '''HDF5''':<div class="box">> make -j 5<br />> make all_tests<br />> make install</div><br />
<br />
====Building NetCDF-C====<br />
The C and Fortran APIs for NetCDF are in separate packages and the C version must be built first.<br />
<br />
* Download '''NetCDF-C''' source code as a gzipped tarball or via git clone:<div class="box">> wget https://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-c-4.7.4.tar.gz</div>or<div class="box">> git clone https://github.com/Unidata/netcdf-c<br />> cd netcdf-c<br />> git checkout tags/v4.7.4<br />> autoconf -i</div><br />
<br />
* Configure '''NetCDF-C''':<div class="box">> ./configure CC=mpicc FC=mpifort CFLAGS=-O3 CPPFLAGS="-I/path/to/parallel/hdf5/1.10.6/include" \<br /> LDFLAGS="-L/path/to/parallel/hdf5/1.10.6/lib" \<br /> --with-mpiexec=/path/to/job/submistion/script \<br /> --prefix=/path/to/parallel/netcdf/4.7.4 \<br /> --enable-parallel-tests --disable-shared --disable-silent-rules</div>{{note}} Note that ''--with-mpiexec'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br /> echo "Usage: mpirun -n # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>{{note}} '''Note:''' To compile '''serial''', remove ''--with-mpiexec'' and ''--enable-parallel-tests'' and change the CC, and FC to the corresponding serial compilers (e.g. gcc and gfortran).<br /><br />{{note}} '''Note:''' If you would like shared libraries to be built, also remove ''--disable-shared''.<br /><br /><br />
<br />
* Build, test, and install '''NetCDF-C''':<div class="box">> make -j 5<br />> make check<br />> make install</div><br />
<br />
====Building NetCDF-Fortran====<br />
Now we can build the Fortran API.<br />
<br />
* Download '''NetCDF-Fortran''' source code as a gzipped tarball or via git clone:<div class="box">> wget https://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-fortran-4.5.3.tar.gz</div>or<div class="box">git clone https://github.com/Unidata/netcdf-fortran<br />> cd netcdf-fortan<br />> git checkout tags/v4.5.3<br />> autoconf -i</div><br />
<br />
* There is a bug/typo in '''configure''' that makes ''nf-config'' report the wrong linking information to ROMS. To fix the error, edit the '''configure''' file (and '''configure.ac''' if you plan to run ''autoconf -i'' again) and change the line:<div class="box">NC_FLIBS="-lnetcdff $NC_FLIBS"</div>to<div class="box">NC_FLIBS="-lnetcdff $NC_LIBS"</div> more details can be found [https://github.com/Unidata/netcdf-fortran/issues/270 here].<br /><br /><br />
<br />
* Configure '''NetCDF-Fortran'''. Note that we set ''prefix'' to the same as '''NetCDF-C''' so the C and Fortran libraries live together:<div class="box">> ./configure CC=mpicc FC=mpifort CFLAGS=-O3 FFLAGS=-O3 \<br /> CPPFLAGS="-I/path/to/parallel/hdf5/1.10.6/include -I/path/to/parallel/netcdf/4.7.4/include" \<br /> LDFLAGS="-L/path/to/parallel/hdf5/1.10.6/lib -L/path/to/parallel/netcdf/4.7.4/lib" \<br /> LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lcurl" \<br /> --with-mpiexec=/path/to/job/submistion/script \<br /> --prefix=/path/to/parallel/netcdf/4.7.4 \<br /> --enable-parallel-tests --disable-shared --disable-silent-rules</div>{{note}} Note that ''--with-mpiexec'' should point to the submision script you created while building '''NetCDF-C''' and is only needed for running test on clusters that use scheduling systems like SLURM.<br /><br />{{note}} '''Note:''' To compile serial remove ''--with-mpiexec'' and ''--enable-parallel-tests'' and change the CC, and FC to the corresponding serial compilers (e.g. gcc and gfortran).<br /><br />{{note}} '''Note:''' If you would like shared libraries to be built, also remove ''--disable-shared''.<br /><br /><br />
<br />
* If you are running on clusters that use scheduling systems like SLURM, you will need to edit 4 testing scripts replacing all occurrences of '''mpiexec''' with the full path to the submision script you created for '''NetCDF-C''':<div class="box">> vi examples/F90/run_f90_par_examples.sh<br />> vi nf03_test4/run_f90_par_test.sh<br />> vi nf_test4/run_f77_par_test_03.sh<br />> vi nf_test4/run_f77_par_test.sh</div><br />
<br />
* Build, test, and install '''NetCDF-C''':<div class="box">> make -j 5<br />> make check<br />> make install</div><br />
<br />
==PIO (Parallel I/O)==<br />
<br />
Open-source parallel library for reading and writing distributed arrays to several scientific data formats like '''pNetCDF''', '''NetCDF3''', and '''NetCDF4/HDF5'''. '''PIO''' provides the flexibility to regulate the number of I/O tasks through data rearrangement between computational and I/O processes to improve performance and memory usage. It allows both '''synchronous''' and '''asynchronous''' I/O. In '''synchronous mode''', a subset of processes performs both I/O and computations (MPI intra-communications). Alternatively, in '''asynchronous mode''', the I/O is carried out by a set of disjointed and dedicated processes (MPI inter-communications). <br />
<br />
The '''PIO''' or '''SCORPIO''' library is intended for ROMS Message Passing Interface (MPI) applications running on a large number of processes in an HPC computer with a '''Parallel File System''' (Lustre, GPFS, and so on). It uses the '''MPI-IO''' interface to facilitate the partitioning of data across computational or dedicated I/O processes. Both [https://github.com/NCAR/ParallelIO PIO] and [https://github.com/E3SM-Project/scorpio SCORPIO] will work with ROMS [https://www.myroms.org/projects/src/ticket/884 svn revision 1064] (May 10, 2021) or newer but we recommend '''PIO''' for its better performance. We recommend you read the manuals for all the build options, but below we show an example build process for both '''PIO''' and '''SCORPIO'''.<br />
<br />
===Building PIO (autotools)===<br />
<br />
'''PIO''' can use NetCDF (version 4.6.1+) and/or PnetCDF (version 1.9.0+) for I/O. NetCDF may be built with or without netCDF-4 features. NetCDF is required for '''PIO''', PnetCDF is optional. {{note}}However, the ROMS build system assumes that '''PIO''' is built with PnetCDF support and will need customization if '''PIO''' is built without it.<br />
<br />
The NetCDF C library must be built with MPI, which requires that it be linked with an MPI-enabled version of HDF5. Optionally, NetCDF can be built with DAP support, which introduces a dependency on CURL. HDF5, itself, introduces dependencies on LIBZ and (optionally) SZIP.<br />
<br />
'''PIO''' recently introduced a NetCDF integration option that allows standard NetCDF library calls to be implemented with the '''PIO''' library but ROMS Parallel I/O is not currently implemented this way. In order to build '''PIO''' with NetCDF integration, you will need NetCDF version 4.7.4 or higher. However, the Fortran API integration has not been implemented in ROMS.<br />
<br />
The '''PIO''' library can be built with the classic Autotools or Cmake. For '''PIO''' we show the Autotools procedure but will show Cmake for '''SCORPIO''' in the next section.<br />
<br />
*First download the '''PIO''' source code as a gzipped tarball or via ''git clone'':<div class="box">> wget https://github.com/NCAR/ParallelIO/releases/download/pio2_5_4/pio-2.5.4.tar.gz</div>or<div class="box">> git clone https://github.com/NCAR/ParallelIO<br />> cd ParallelIO<br />> git checkout tags/pio2_5_4<br />> autoconf -i</div><br />
<br />
*If you downloaded the gzipped tarball, you should run ''make clean'' because some Fortran .mod files are included and need to be removed:<div class="box">> make clean</div><br />
<br />
*Configure '''PIO''':<div class="box">> ./configure CC=mpicc CXX=mpicxx FC=mpifort CFLAGS=-O3 FFLAGS=-O3 FCFLAGS=-O3 \<br /> CPPFLAGS="-I/path/to/netcdf/include -I/path/to/hdf5/include -I/path/to/pnetcdf/include" \<br /> LDFLAGS="-L/path/to/netcdf/lib -L/path/to/hdf5/lib -L/path/to/pnetcdf/lib" \<br /> LIBS="-lhdf5_hl -lhdf5 -lm -lcurl -lz" \<br /> --with-mpiexec=/path/to/cluster/submission/script \<br /> --prefix=/pio/install/path --enable-fortran --disable-timing \<br /> --disable-silent-rules --disable-shared</div>{{note}} Note that ''--with-mpiexec'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br /> echo "Usage: mpirun -n # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div><br />
<br />
*Build, test, and install '''PIO''':<div class="box">> make -j 5<br />> make check<br />> make install</div>{{warning}} '''Warning:''' If you are using the Intel compilers, you will probably need to remove "'''test_darray_async_many'''" from line 50 of '''test/cunit/run_tests.sh''' for ''make check'' to complete without errors. Details [https://github.com/NCAR/ParallelIO/issues/1867 here].<br />
<br />
===Building SCORPIO (Cmake)===<br />
<br />
{{note}} '''Note:''' '''SORPIO''' recently released version 1.2.0 and 1.2.1 but ROMS has not been tested with these versions. It is likely that the new version will work fine with ROMS but use these new versions at your own risk.<br />
<br />
'''SCORPIO''' can use NetCDF (version 4.3.3+), PnetCDF (version 1.6.0+) or ADIOS (version 2.6.0+) for I/O. {{note}} ADIOS is not implemented in ROMS.<br />
<br />
Ideally, the NetCDF version should be built with MPI, which requires that it be linked with an MPI-enabled version of HDF5. Optionally, NetCDF can be built with DAP support, which introduces a dependency on CURL. Additionally, HDF5, itself, introduces dependencies on LIBZ and (optionally) SZIP. If your NetCDF is built with DAP support, you will need to edit '''cmake/TryNetCDF_DAP.c''', changing line 8 from <div class="box">#if NC_HAS_DAP==1</div> to <div class="box">#if NC_HAS_DAP==1 || NC_HAS_DAP2==1 || NC_HAS_DAP4==1</div><br />
<br />
*Download the '''SCORPIO''' source code:<div class="box">> git clone https://github.com/E3SM-Project/scorpio<br />> cd scorpio<br />> git checkout tags/scorpio-v1.1.6<br /></div>If your NetCDF is built with DAP support, you will need to edit '''cmake/TryNetCDF_DAP.c''', changing line 8 from <div class="box">#if NC_HAS_DAP==1</div> to <div class="box">#if NC_HAS_DAP==1 || NC_HAS_DAP2==1 || NC_HAS_DAP4==1</div><br />
<br />
*Configure '''SCORPIO''':<div class="box">> cd ..<br />> mkdir build_scorpio<br />> cd build_scorpio<br />> CC=mpicc CXX=mpicxx FC=mpifort cmake \<br /> -DNetCDF_PATH=/path/to/netcdf \<br /> -DPnetCDF_PATH=/path/to/pnetcdf \<br /> -DMPIEXEC=/path/submission/scipt \<br /> -DMPIEXEC_MAX_NUMPROCS=8 -DMPIEXEC_NUMPROC_FLAG=-n \<br /> -DPIO_FILESYSTEM_HINTS=gpfs -DPIO_ENABLE_TIMING=OFF -DPIO_ENABLE_TESTS=ON \<br /> -DCMAKE_INSTALL_PREFIX=/scorpio/install/path ../scorpio</div>{{note}} Note that ''-DMPIEXEC'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br /> echo "Usage: mpirun -n # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div><br />
<br />
*Compile, test, and install:<div class="box">> make -j 6<br />> make -j 6 tests<br />> ctests<br />> make install</div></div>Arangohttps://www.myroms.org/wiki/index.php?title=4DVar_Tutorial_Introduction&diff=66464DVar Tutorial Introduction2024-02-14T16:55:22Z<p>Arango: /* Download Tutorial Files */</p>
<hr />
<div><div class="title">4-Dimensional Variational (4D-Var) Data Assimilation Tutorial</div>__NOTOC__<br />
<br />
<br />
This tutorial shows several examples of ROMS 4D-Var data assimilation algorithms in the California Current System (CCS), 1/3 degree resolution ([[Options#WC13|WC13]]).<br />
<br />
<br />
<!-- Edit Template:4DVar_Tut_TOC to modify this Table of Contents--><br />
<div style="float: left;margin: 0 20px 0 0;">{{4DVar Tut TOC}}</div>__TOC__<br />
<br />
<div style="clear: both"></div><br />
<br />
<br />
<section begin=setup /><br />
==Model Set-up==<br />
The [[Options#WC13|WC13]] model domain is shown in Fig. 1 and has open boundaries along the northern, western, and southern edges of the model domain.<br />
<br />
[[Image:hraw_coarse_transect.png|500px|thumb|center|<center>'''Fig. 1:''' Model Bathymetry with 37&deg;N Transect and Target Area</center>]]<br />
<div style="clear: both"></div><br />
<br />
In the tutorial, you will perform a 4D-Var data assimilation cycle that spans the period 3-6 January, 2004. The 4D-Var control vector ''&delta;'''''z''' is comprised of increments to the initial conditions, ''&delta;'''''x'''(''t<sub>0</sub>''), surface forcing, &delta;'''f'''(''t''), and open boundary conditions, &delta;'''b'''(''t''). The ''prior'' initial conditions, '''x<sub>b</sub>'''(''t<sub>0</sub>''), are taken from the sequence of 4D-Var experiments described by [[Bibliography#MooreAM_2011b|Moore ''et al.'' (2011b)]] in which data were assimilated every 7 days during the period July 2002- December 2004. The ''prior'' surface forcing, '''f<sub>b</sub>'''(''t''), takes the form of surface wind stress, heat flux, and a freshwater flux computed using the ROMS bulk flux formulation, and using near surface air data from COAMPS ([[Bibliography#DoyleJD_2009|Doyle ''et al.'', 2009]]). Clamped open boundary conditions are imposed on (''u'',''v'') and tracers, and the ''prior'' boundary conditions, '''b<sub>b</sub>'''(''t''), are taken from the global ECCO product ([[Bibliography#WunschC_2007|Wunsch and Heimbach, 2007]]). The free-surface height and vertically integrated velocity components are subject to the usual Chapman and Flather radiation conditions at the open boundaries. The ''prior'' surface forcing and open boundary conditions are provided daily and linearly interpolated in time. Similarly, the increments ''&delta;'''''f'''(''t'') and ''&delta;'''''b'''(''t'') are also computed daily and linearly interpolated in time.<br />
<br />
The observations assimilated into the model are satellite SST, satellite SSH in the form of a gridded product from Aviso, and hydrographic observations of temperature and salinity collected from Argo floats and during the GLOBEC/LTOP and CalCOFI cruises off the coast of Oregon and southern California, respectively. The observation locations are illustrated in Fig. 2.<br />
<br />
{|align="center"<br />
|+style="caption-side:bottom;"|'''Figure 2:''' WC13 Observations<br />
|-<br />
|[[Image:aviso_ssh_2019.png|thumb|400px|<center>'''a)''' Aviso SSH</center>]]<br />
|[[Image:blended_sst_2019.png|thumb|400px|<center>'''b)''' Blended SST</center>]]<br />
|-<br />
|[[Image:in_situ_T_2019.png|thumb|400px|<center>'''c)''' In Situ Temperature</center>]]<br />
|[[Image:in_situ_S_2019.png|thumb|400px|<center>'''d)''' In Situ Salinity</center>]]<br />
|-<br />
|}<section end=setup /><br />
<br />
==Download Lectures and Exercises==<br />
<br />
The workshop Agenda can be found [https://www.myroms.org/Workshops/4DVAR2019/Agenda.pdf here]. Links to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop] lectures are provided below. These lectures are meant to supplement this 4D-Var tutorial.<br />
<br />
*'''Lecture 1:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_1_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">4D-Var: Some Basics</span><br />
*'''Lecture 2:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_2_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">The Mechanics of 4D-Var</span><br />
*'''Lecture 3:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_3_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Dual 4D-Var</span><br />
*'''Lecture 4:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_4_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observing System Simulation Experiments (OSSEs)</span><br />
*'''Lecture 5:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_5_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observation Impact &amp; Observation Sensitivity</span><br />
*'''Lecture 6:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_6_2019.pdf PDF]</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The MARACOOS analysis-forecast system</span><br />
*'''Lecture 7:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_7_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Array Modes</span><br />
*'''Lecture 8:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_8_2019.pdf PDF]-->PDF</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The West Coast Ocean Forecasting System (WCOFS)</span><br />
*'''Lecture 9:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_9_2019.pdf PDF]-->PDF</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The CeNCOOS and PacIOOS analysis-forecast systems</span><br />
<br />
Below are the tutorials pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:<br />
<br />
*'''Tutorial 01:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_01_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Explanation of cpp options, ocean.in, s4dvar.in (I4DVAR)</span><br />
*'''Tutorial 02:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_02_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Multiple outer loops</span><br />
*'''Tutorial 03:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_03_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercises 1 and 2</span><br />
*'''Tutorial 04:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_04_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Calculation of prior error standard deviations</span><br />
*'''Tutorial 05:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_05_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Explanation of cpp options, ocean.in, s4dvar.in (RBL4D-Var)</span><br />
*'''Tutorial 06:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_06_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Semi-variograms</span><br />
*'''Tutorial 07:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_07_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">ERDDAP data server and management tools</span><br />
*'''Tutorial 08:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_08_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercises 3 & 4</span><br />
*'''Tutorial 09:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_09_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Computing normalization coefficients for covariance models</span><br />
*'''Tutorial 10:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_10_2019.pdf PDF_Arango] | [http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_10a_2019.pdf PDF_Wilkin]</span> &nbsp; <span class="twilightBlue">Building your observation files</span><br />
*'''Tutorial 11:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_11_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observation Impact & Observation Sensitivity</span><br />
*'''Tutorial 12:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_12_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercise 5</span><br />
*'''Tutorial 13:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_13_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Using ERDDAP to view observation impact information</span><br />
*'''Tutorial 14:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_14_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercise 6 & 7</span><br />
*'''Tutorial 15:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_15_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Putting it all together</span><br />
<br />
Below are the exercises pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:<br />
<br />
*'''Exercise 1:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_1_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Incremental, Strong Constraint 4D-Var</span><br />
*'''Exercise 2:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_2_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">I4D-Var with Multiple Outer-loops</span><br />
*'''Exercise 3:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_3_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Dual Formulation 4D-Var - RBL4D-Var</span><br />
*'''Exercise 4:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_4_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Weak Constraint Dual Formulation 4D-Var</span><br />
*'''Exercise 5:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_5_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Analysis Cycle Observation Impacts</span><br />
*'''Exercise 6:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_6_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Analysis Cycle Observation Sensitivity</span><br />
*'''Exercise 7:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_7_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Reduced-Rank Array Modes</span><br />
*'''Exercise 8:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_8_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Forecast Cycle Observation Impacts</span><br />
*'''Exercise 9:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_9_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Forecast Cycle Observation Sensitivities</span><br />
<br />
Below are the homeworks pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:<br />
<br />
*'''Homework 1:''' &nbsp; &nbsp; <span class="twilightBlue">Building the standard deviation file for user model configuration</span><br />
*'''Homework 2:''' &nbsp; &nbsp; <span class="twilightBlue">Building the file of normalization coefficients for the prior error covariance matrix for user model configuration</span><br />
*'''Homework 3:''' &nbsp; &nbsp; <span class="twilightBlue">Build observation file for user model configuration</span><br />
<br />
==Download Tutorial Files==<br />
All data files and scripts necessary to run the 4D-Var tests discussed in this tutorial are available from the ROMS [https://github.com/myroms GitHub] repository. To download, execute the following command:<br />
<div class="box"><span class="red"> git clone https://github.com/myroms/roms_test.git</div><br />
<br />
==Tutorial Directory Structure==<br />
When the checkout is complete, you will have the following directory structure under your <span class="twilightBlue">my_test</span> directory.<br />
<br />
<span class="twilightBlue">/WC13 Main California Current System 4D-Var applications<br />
/ARRAY_MODES Stabilized representer matrix array modes and clipping<br />
/Data Input data directory<br />
/Functionals Analytical expressions header files<br />
/I4DVAR Primal form of incremental, strong constraint 4D-Var, I4D-Var<br />
/Normalization 4D-Var error covariance normalization coefficients<br />
/plotting 4D-Var plotting scripts (Matlab and ROMS plotting package)<br />
/RBL4DVAR Dual form of 4D-Var, Restricted B-preconditioned Lanczos Analysis System, RBL4D-Var<br />
/RBL4DVAR_analysis_impact RBL4D-Var analysis observation impact<br />
/RBL4DVAR_analysis_sensitivity RBL4D-Var analysis observation sensitivity (adjoint of RBL4D-Var)<br />
/RBL4DVAR_forecast_impact RBL4D-Var forecast observation impact<br />
/RBL4DVAR_forecast_sensitivity RBL4D-Var forecast observation sensitivity (adjoint of RBL4D-Var)<br />
</span><br />
Mostly all the directories have a <span class="twilightBlue">Readme</span> file with detailed instructions for configuring, compiling, running, and plotting the results.<br />
<br />
==References==<br />
<br />
The technical description of the algorithms and application used in this tutorial are described in Moore ''et al.'' ([[Bibliography#MooreAM_2011a|2011a]], [[Bibliography#MooreAM_2011b|b]], [[Bibliography#MooreAM_2011c|c]]).</div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6604Source Code2023-12-08T17:17:26Z<p>Arango: </p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion]('''SVN'''). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions. This '''GitHub''' version becomes the official '''Git''' repository for downloading, updating, improving, and correcting defects/bugs in the '''ROMS''' source code. Also, it is the version used in the <span class="red">ROMS-JEDI</span> interface hosted at [https://github.com/JCSDA-internal https://github.com/JCSDA-internal], which is currently private.<br />
<br />
{{note}}<span class="mediumOrchid">Note:</span> '''GitHub''' nowadays requires two-factor authentication!<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center><br />
<br />
<br />
==<span class="red">WARNING:</span> <span class="blue">The updating of ROMS svn repositories will be discontinued on</span><span class="red"> January 1, 2025</span>==</div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6603Source Code2023-12-08T17:16:43Z<p>Arango: </p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion](SVN). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions. This '''GitHub''' version becomes the official '''Git''' repository for downloading, updating, improving, and correcting defects/bugs in the '''ROMS''' source code. Also, it is the version used in the <span class="red">ROMS-JEDI</span> interface hosted at [https://github.com/JCSDA-internal https://github.com/JCSDA-internal], which is currently private.<br />
<br />
{{note}}<span class="mediumOrchid">Note:</span> '''GitHub''' nowadays requires two-factor authentication!<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center><br />
<br />
<br />
==<span class="red">WARNING:</span> <span class="blue">The updating of ROMS svn repositories will be discontinued on</span><span class="red"> January 1, 2025</span>==</div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6602Source Code2023-12-08T17:14:28Z<p>Arango: </p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion](SVN). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions. This '''GitHub''' version becomes the official '''Git''' repository for downloading, updating, improving, and correcting defects/bugs in the ROMS source code. Also, it is the version used in the <span class="red">ROMS-JEDI</span> interface hosted at [https://github.com/JCSDA-internal https://github.com/JCSDA-internal], which is currently private.<br />
<br />
{{note}}<span class="mediumOrchid">Note:</span> '''GitHub''' nowadays requires two-factor authentication!<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center><br />
<br />
<br />
==<span class="red">WARNING:</span> <span class="blue">The updating of ROMS svn repositories will be discontinued on</span><span class="red"> January 1, 2025</span>==</div>Arangohttps://www.myroms.org/wiki/index.php?title=Subversion&diff=6601Subversion2023-12-08T17:09:42Z<p>Arango: </p>
<hr />
<div><div class="title">Subversion</div><br />
ROMS source code is distributed using [http://subversion.tigris.org Subversion](SVN). There are SVN clients available for nearly every operating system and a list of popular clients can be found [http://subversion.tigris.org/links.html#clients here]. If you wish to use Subversion to maintain your own copy of ROMS you will also need an [http://subversion.tigris.org/project_packages.html svn server]. This page will help you get started with downloading ROMS with Subversion. There is also a way to download ROMS using git, described [[ROMS_git | elsewhere]].<br />
<br />
<br />
===<span class="red">WARNING:</span> <span class="blue">The updating of ROMS svn repositories will be discontinued on</span><span class="red"> January 1, 2025</span>===<br />
<br />
'''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
==SVN Overview==<br />
Subversion is a tool for managing software development that keeps track of who modified what and allows the return to a previous version if changes don't work as expected. All the ROMS/TOMS files are stored in a SVN repository on '''www.myroms.org''' with access controlled by requiring authentication with the same ROMS Username/Password combination assigned to registered users of the ROMS Forum.<br />
<br />
This SVN repository is the official version of the code which only the developers are allowed to change. Users should download the ROMS code to their local machines using an '''svn client'''. Don't attempt to use a regular web browser to browse or download files from the SVN repository - there are much better tools for interacting with the code repository. See the notes below under the heading '''View the Repository'''.<br />
<br />
We strongly recommend users always check out the current ''trunk'' version since this has the most recent updates and bug fixes. The ''tags'' version is kept largely as an historical record of stable releases at the conclusion of major code upgrades. <br />
<br />
Below is a general description of how subversion works. Please look at the [http://svnbook.red-bean.com/ svn book] for more detailed information. Brief instructions for using two GUI clients (one for PC and one for Linux) we've tried are included.<br />
<br />
==Downloading ROMS==<br />
{{warning}}'''WARNING:''' It is strongly suggested that you checkout the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like <span class="red">dos2unix</span> or <span class="red">recode</span>. Even with these utilities you may still have problems compiling ROMS.<br />
<br />
<br />
In order download source code from a Subversion repository, '''svn client''' software must be installed on your local machine. If you are compiling subversion on your own be sure to build it with '''SSL support''' or you will not be able to download the ROMS source code. Most Linux distributions come with subversion (the command name is '''svn'''), so shell commands may be used without installing additional software. If your username on your local computer is not the same as your ROMS username you will need to pass the '''--username <username>''' option to '''svn'''; an example is given below. The general form of subversion commands is:<br />
<br />
<span class="red">svn</span> action from to {optional_qualifiers} <br />
<br />
To check-out the files from the ROMS repository '''trunk''' (latest version), enter (notice https instead of http):<br />
<br />
<span class="red">svn checkout <nowiki>https://www.myroms.org/svn/src/trunk</nowiki> MyDir</span> <br />
<br />
where '''MyDir''' is the destination directory on your local computer. It will be created if not found. If your '''username''' on your local computer is not the same as your ROMS username you will need to pass the '''--username''' option to '''svn''':<br />
<br />
<span class="red">svn checkout --username joe_roms <nowiki>https://www.myroms.org/svn/src/trunk</nowiki> MyDir</span><br />
<br />
===Files for test cases and other repositories===<br />
There is an additional repository that provides idealized [[Test Cases]] and realistic applications to introduce you to ROMS many capabilities. You may use these examples as a guideline in setting up your own application. Currently, this repository requires '''~388MB''' of disk space. This is because the realistic applications require several input NetCDF files. To checkout this repository, enter:<br />
<br />
<span class="red">svn checkout <nowiki>https://www.myroms.org/svn/src/test</nowiki> MyTest</span><br />
<br />
Similarly, to check-out the files from the ROMS repository '''matlab''' (official scripts), enter:<br />
<br />
<span class="red">svn checkout <nowiki>https://www.myroms.org/svn/src/matlab</nowiki> MyDir</span><br />
<br />
To check-out the files from the ROMS repository '''plot''' (NCAR's library-based plotting package), enter:<br />
<br />
<span class="red">svn checkout <nowiki>https://www.myroms.org/svn/src/plot</nowiki> MyDir</span> <br />
<br />
To check-out the files from the ROMS repository '''branches''', '''plot''', '''matlab''', '''tags''', '''test''', and '''trunk''', (this is not recommended because it takes a long time to complete) enter:<br />
<br />
<span class="red">svn checkout <nowiki>https://www.myroms.org/svn/src</nowiki> MyDir</span><br />
<br />
You only check out once, after that, a hidden directory called '''.svn''' exists to keep track of the source, destination and a bunch of other information. Your username and password will also be saved. For more detail on command line use and syntax, see the [http://svnbook.red-bean.com/ svn book].<br />
<br />
Several GUI front-ends to subversion exist, allowing the user to have visual prompts to help them manage their files.<br />
<br />
===Updates===<br />
<br />
Now and again, you might feel the urge to get up to speed with the latest changes that have been made to the ROMS repository. When that happens, simply go to the directory that was "MyDir" above and type:<br />
<br />
<span class="red">svn update</span><br />
<br />
Subversion will remember where you checked out from before and see if a newer revision exists. If so, it will download and apply all the relevant changes.<br />
<br />
===Code changes===<br />
<br />
As you use ROMS, you may find yourself adding new files and new chunks of code to existing files. Unless you are a developer with write access to the repository at www.myroms.org, you have no easy way to save your changes within the svn framework, since any one directory can only point to one repository. To keep getting updates from the trunk, you must keep using the svn server at myroms.org. At the very least, saving a tarball before fetching major updates is a prudent step. A more organized approach to revision control for your own files can be achieved by using [http://git-scm.com/ git].<br />
<br />
===Conflicts===<br />
<br />
Sometimes when you make changes to your copy of the ROMS code, those changes will conflict with changes made to the repository (the code that resides at <span class="forestGreen"><nowiki>https://www.myroms.org/svn/src/trunk</nowiki></span>). This means that the changes from the server overlapped with your own, and now you have to manually choose between them.<br />
<br />
Whenever a conflict occurs, three things typically occur to assist you in resolving that conflict:<br />
:*Subversion halts during the update, offering you several choices, and remembers that the file is in a state of conflict if you don't clear it right then.<br />
:*If Subversion considers the file to be mergeable, it places conflict markers (special strings of text which delimit the “sides” of the conflict, usually "<" and ">" characters) into the file to visibly demonstrate the overlapping areas.<br />
:*For every conflicted file, Subversion places three extra unversioned (not under Subversion control) files in your working copy:<br />
::*'''filename.mine:''' This is your file as it existed in your working copy (local copy) before you updated your working copy. This file has only your latest changes in it. (If Subversion considers the file to be unmergeable, then the .mine file isn't created, since it would be identical to the working file.)<br />
::*'''filename.rOLDREV:''' This is the file that was the '''BASE''' revision before you updated your working copy. That is, the file that you checked out before you made your latest edits.<br />
::*'''filename.rNEWREV:''' This is the file that your Subversion client just received from the server when you updated your working copy. This file corresponds to the HEAD (latest) revision of the repository.<br />
<br />
For example, let's say you checked out revision 280 and made some changes to <span class="forestGreen">User/Functionals/ana_hmixcoef.h</span>. Now you want to update your ROMS source code to take advantage of a new algorithm but when you run <span class="red">svn update</span> your <span class="forestGreen">ana_hmixcoef.h</span> is now in conflict with the new version in the repository.<br />
<br />
:<div class="box">&gt; <span class="red">svn update</span><br>U Version<br>U ROMS/Modules/mod_mixing.F<br>U ROMS/Functionals/ana_hmixcoef.h<br>C User/Functionals/ana_hmixcoef.h<br>Updated to revision 291.<br><br>&gt;</div><br />
<br />
The above is with an older version of svn. The latest, greatest does this:<br />
<br />
:<div class="box">Conflict discovered in 'ROMS/Utility/inp_par.F'.<br>Select: (p) postpone, (df) diff-full, (e) edit,<br> (mc) mine-conflict, (tc) theirs-conflict,<br> (s) show all options:</div><br />
<br />
Selecting (p) will behave as the old version.<br />
<br />
If you get a conflict, you need to do one of three things:<br />
:*Merge the conflicted text “by hand” by examining and editing the conflict markers within the file.<br />
:*Copy one of the temporary files on top of your working file.<br />
:*Run <span class="red">svn revert <filename></span> to throw away all of your local changes.<br />
<br />
Once you've resolved the conflict, you need to let Subversion know by running <span class="red">svn resolved</span>. This removes the three temporary files and Subversion no longer considers the file to be in a state of conflict. More on this below.<br />
<br />
====Merging Conflicts by Hand====<br />
<br />
{{note}}'''Note:''' We recommend that you make a backup of the '''<filename>.mine''' file before you begin to be certain you don't accidentally lose your modifications.<br />
<br />
To merge your changes with those from the latest revision in the repository, open <span class="forestGreen">User/Functionals/ana_hmixcoef.h</span> in your favorite editor and look for a string of "<" characters. You should see something like this:<br />
<br />
:<div class="box">&lt;&lt;&lt;&lt;&lt;&lt;&lt; .mine<br>#ifndef DISTRIBUTE<br> IF (Lanafile.and.(tile.eq.0)) THEN<br>#else<br> IF (Lanafile) THEN<br>#endif<br>=======<br>#ifdef DISTRIBUTE<br> IF (Lanafile) THEN<br>#else<br> IF (Lanafile.and.(tile.eq.0)) THEN<br>#endif<br>&gt;&gt;&gt;&gt;&gt;&gt;&gt; .r291</div><br />
<br />
After comparing the two code blocks (separated by the "=" symbol), you decide that you prefer the logic from the repository so you remove the conflict markers and your code so the section now looks like this:<br />
<br />
:<div class="box">#ifdef DISTRIBUTE<br> IF (Lanafile) THEN<br>#else<br> IF (Lanafile.and.(tile.eq.0)) THEN<br>#endif</div><br />
<br />
Now you can save the file and let Subversion know that you have resolved the conflict:<br />
<br />
:<div class="box">&gt; <span class="red">svn resolved User/Functionals/ana_hmixcoef.h</span><br>Resolved conflicted state of 'User/Functionals/ana_hmixcoef.h'</div><br />
<br />
====Copying a File Onto Your Working File====<br />
<br />
{{note}}'''Note:''' We recommend that you make a backup of the '''<filename>.mine''' file before you begin to be certain you don't accidentally lose your modifications.<br />
<br />
If you get a conflict and decide that you want to throw out your changes, you can merely copy one of the temporary files created by Subversion over the file in your working copy. Let's say you want to use the new revision from the repository:<br />
<br />
:<div class="box">&gt; <span class="red">cd User/Functionals</span><br>&gt; <span class="red">ls ana_hmixcoef.h*</span><br>ana_hmixcoef.h ana_hmixcoef.h.mine ana_hmixcoef.h.r280 ana_hmixcoef.h.r291<br><br>&gt; <span class="red">cp ana_hmixcoef.h.r291 ana_hmixcoef.h</span><br>&gt; <span class="red">svn resolved ana_hmixcoef.h</span><br>Resolved conflicted state of 'ana_hmixcoef.h'</div><br />
<br />
====Punting: Using <span class="red">svn revert</span>====<br />
<br />
{{note}}'''Note:''' We recommend that you make a backup of the '''<filename>.mine''' file before you begin to be certain you don't accidentally lose your modifications.<br />
<br />
If you get a conflict, and upon examination decide that you want to throw out your changes and start your edits again, just revert your changes:<br />
<br />
:<div class="box">&gt; <span class="red">cd User/Functionals</span><br>&gt; <span class="red">svn revert ana_hmixcoef.h</span><br>Reverted 'ana_hmixcoef.h'</div><br />
<br />
Note that when you revert a conflicted file, you don't have to run <span class="red">svn resolved</span>.<br />
<br />
==Using GUI SVN Clients==<br />
{{warning}}'''WARNING:''' It is strongly suggested that you checkout the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like <span class="red">dos2unix</span> or <span class="red">recode</span>. Even with these utilities you may still have problems compiling ROMS.<br />
<br />
<br />
Most of the GUI '''svn clients''' are very similar to each other. In this example we will use eSVN, although several users have reported better luck with the free "Foundation Version" of [http://www.syntevo.com/smartsvn/index.jsp/ SmartSVN].<br />
<br />
===View the Repository===<br />
The easiest way to view a repository is to simply look at it with [http://trac.edgewall.org/ Trac] in a web browser. Enter the URL:<br />
<br />
https://www.myroms.org/projects/src/browser<br />
<br />
You will be prompted for your ROMS username and password. This is also a good way to see individual files in the repository without actually downloading them.<br />
<br />
To view the repository with eSVN, Click on the file menu and choose "Browse Repository" and enter repository URL:<br />
<br />
[[Image:esvn_browser.jpg]]<br />
<br />
Click "Browse" and you should be able to look at the '''branches''', '''tags''', and '''trunk'''. You can navigate the repository with a web browser as described above, but cannot do any of the version control stuff that follows.<br />
<br />
===Checkout a Copy of the Trunk===<br />
<br />
This process brings a copy of the main code from the repository to your computer and places it under svn control. "Local Path" tells eSVN where you would like your copy of the source code to be located.<br />
<br />
[[Image:esvn_checkout.jpg]]<br />
<br />
You should see a bunch of text stream by like:<br />
<br />
svn checkout <nowiki>https://www.myroms.org/svn/src/trunk</nowiki> /home/ocean/src --username ocean <br />
--password ****** --non-interactive <br />
A /home/ocean/src/ROMS<br />
A /home/ocean/src/ROMS/License_ROMS.txt<br />
A /home/ocean/src/ROMS/Tangent<br />
A /home/ocean/src/ROMS/Tangent/tl_rho_eos.F<br />
A /home/ocean/src/ROMS/Tangent/tl_uv3dmix.F<br />
A /home/ocean/src/ROMS/Tangent/tl_def_his.F<br />
...<br />
A /home/ocean/src/Compilers/UNICOS-mk-f90.mk<br />
A /home/ocean/src/Compilers/CYGWIN-g95.mk<br />
A /home/ocean/src/Compilers/OSF1-f90.mk<br />
A /home/ocean/src/Compilers/Darwin-ifort.mk<br />
A /home/ocean/src/Compilers/AIX-xlf.mk<br />
A /home/ocean/src/Compilers/SunOS-f95.mk<br />
A /home/ocean/src/makefile<br />
Checked out revision 11.<br />
successfully (0)</div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6600Source Code2023-12-08T17:01:59Z<p>Arango: </p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion](SVN). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions. This '''GitHub''' version becomes the official '''Git''' repository for downloading, updating, improving, and correcting defects/bugs in the ROMS source code. Also, it is the version used in the <span class="red">ROMS-JEDI</span> interface hosted at [https://github.com/JCSDA-internal https://github.com/JCSDA-internal], which is currently private.<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center><br />
<br />
<br />
==<span class="red">WARNING:</span> <span class="blue">The updating of ROMS svn repositories will be discontinued on</span><span class="red"> January 1, 2025</span>==</div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6599Source Code2023-12-08T16:57:35Z<p>Arango: </p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion](SVN). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions. This '''GitHub''' version becomes the official '''Git''' repository for downloading, updating, improving, and correcting defects/bugs in the ROMS source code. Also, it is the version used in the <span class="red">ROMS-JEDI</span> interface hosted at [https://github.com/JCSDA-internal https://github.com/JCSDA-internal], which is currently private.<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center></div>Arangohttps://www.myroms.org/wiki/index.php?title=Source_Code&diff=6598Source Code2023-12-08T16:52:52Z<p>Arango: Created page with "<div class="title">Regional Ocean Modeling System (ROMS)</div> This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of N..."</p>
<hr />
<div><div class="title">Regional Ocean Modeling System (ROMS)</div><br />
<br />
This official community version of '''ROMS''' is developed and maintained at Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA. Currently, '''ROMS''' source code is distributed using both [https://git-scm.com Git] and [https://subversion.apache.org Subversion](SVN). However, starting <span class="red">January 1, 2025</span>, '''ROMS''' will be exclusively distributed from its '''GitHub''' repository at [https://github.com/myroms https://github.com/myroms]. Updating the '''SVN''' repositories at [https://www.myroms.org https://www.myroms.org] will be discontinued but kept for historical reasons.<br />
<br />
The '''ROMS''' framework is intended for users interested in ocean modeling. It requires an extensive background in ocean dynamics, numerical modeling, and computers to configure, run, and analyze the results to ensure you get the correct solution for your application. Therefore, we highly recommend users register at [https://www.myroms.org https://www.myroms.org] and set up a <span class="red">username</span> and <span class="red">password</span> to access the '''ROMS''' forum, email notifications for bugs/updates, technical support from the community, trac code maintenance history, tutorials, workshops, and publications. The User's '''ROMS''' forum has over 24,000 posts with helpful information. Technical support is limited to registered users. We do not provide user technical support, usage, or answers in '''GitHub'''.<br />
<br />
The <span class="red">ROMS GitHub</span> repository also has its own wiki at [https://github.com/myroms/roms/wiki https://github.com/myroms/roms/wiki] with useful information and instructions.</div>Arangohttps://www.myroms.org/wiki/index.php?title=ROMS-JEDI_Intro&diff=6595ROMS-JEDI Intro2023-09-11T20:21:45Z<p>Arango: /* Introduction */</p>
<hr />
<div><div class="title">ROMS-JEDI</div><br />
<br />
<div style="clear: both"></div><br />
<br />
<!-- Edit Template:ROMS-JEDI_TOC to modify this Table of Contents--><br />
<div style="float: left;margin: 0 20px 0 0;">{{ROMS-JEDI_TOC}}</div><br />
<br />
<div style="clear: both"></div><br />
<br />
<br />
[[Image:romsjedi_logo.png|left]]<br />
[[Image:romsjedi_schematic.png|1000px|left]]<br />
<div style="clear: both"></div><br />
<br />
<br />
==Introduction==<br />
The Joint Effort for Data Assimilation Integration ('''JEDI''') framework is a unified, high-level, state-of-the-art data assimilation ('''DA''') system for scientific exploration and operational forecasting. The diagram above shows that the '''JEDI''' building blocks are model agnostic, and any modern geophysical numerical kernels can be interfaced. Its separable generic design and abstraction allow the implementation of various '''DA''' methodologies, including variational, ensemble, hybrid, and coupled approaches that are common to any geophysical system, without the need to develop such algorithms separately for each model. The '''JEDI''' framework is centered around the Object-Oriented Prediction System ('''OOPS''') layer classes and operators implemented in '''C++''' using templates. The specific implementations like '''ROMS-JEDI''' are written in Fortran 2003 because of the need for class objects, polymorphism, inheritance, and interoperability between '''C++''' and '''Fortran'''. The interoperability binding allows '''Fortran''' to invoke '''C++''' functions and vice versa '''C++''' to invoke '''Fortran''' procedures.<br />
<br />
The '''JEDI''' separation of concerns design allows the incorporation of diverse (current and future) observational data streams essential for any DA system. Observation operators can be efficiently coded and added to the Unified Forward Operator ('''UFO''') interface, which simulates, interpolates, and relates observable data to the model component state vector, '''H(x)'''. The '''UFO''' interface also implements quality control, data thinning, and bias correction. The Interface for Observation Data Access ('''IODA''') storage layer ingests observational data from providers into the '''UFO''' and '''OOPS''' building blocks. The immense amount of incoming data can be trimmed down, quality-controlled, and filtered by '''UFO''' operators before it is assimilated and during postprocessing when computing diagnostics, impacts, and sensitivities.<br />
<br />
The '''ROMS-JEDI''' framework is intended for users interested in ocean prediction and data assimilation. Currently, we use the '''JEDI''' framework to incorporate physical observational data into regional '''ROMS''' applications using variational ('''3DVar'''/'''4DVar''', '''3D-FGAT'''/'''4D-FGAT'''), ensemble ('''EDA''', '''EnKF''', '''LETKF'''), and hybrid ('''3DEnVar'''/'''4DEnVar''') data assimilation methodologies. The '''ROMS''' Tangent Linear Model ('''TLM''') and Adjoint Model ('''ADM''') kernels are used in the '''4DVar''' family of algorithms.</div>Arangohttps://www.myroms.org/wiki/index.php?title=ROMS-JEDI_Compiling&diff=6594ROMS-JEDI Compiling2023-08-31T13:00:43Z<p>Arango: /* Compiling ROMS-JEDI Default Application WC13 */</p>
<hr />
<div><div class="title">Compiling ROMS-JEDI</div><br />
<br />
<!-- Edit Template:ROMS-JEDI_TOC to modify this Table of Contents--><br />
<div style="float: left;margin: 0 20px 0 0;">{{ROMS-JEDI_TOC}}</div>__TOC__<br />
<br />
<div style="clear: both"></div><br />
<br />
<br />
==Configuring GIT==<br />
<br />
Before downloading the '''ROMS-JEDI''' repository, ensure that your '''~/.gitconfig''' has the appropriate '''git-lfs''' configuration for correctly downloading the default application (U.S. West Coast, '''WC13''' option) input and observation NetCDF files. Otherwise, the '''Unit Test Cases''' requiring input NetCDF files will fail. The '''Git LFS''' is a command line extension and specification for managing large files with '''Git'''. A sample of the configuration file looks like this:<div class="box"><span class="red">more ~/.gitconf</span><br /><br />[User]<br /> name = GivenName MiddleName FamilyName<br /> email = your@email<br />[credential]<br /> helper = cache --timeout=7200<br /> helper = store --file ~/.my-credentials<br />[filter "lfs"]<br /> clean = git-lfs clean -- %f<br /> smudge = git-lfs smudge -- %f<br /> process = git-lfs filter-process<br /> required = true</div><br />
<br />
Alternatively, you may execute '''git lfs pull''' at your location of this repository to download a viable version of the '''Git LFS''' files from the remote repository in '''GitHub'''. Also, to automatically add the '''LFS''' filter to your existing '''~/.gitconfig''', you could use '''git lfs install''' from anywhere in your computer directories.<br />
<br />
==Downloading ROMS-JEDI==<br />
<br />
The '''ROMS-JEDI''' repository is private within the Joint Center for Satellite Data Assimilation ('''JCSDA''') community. Users need to be partners, contributors, or members of '''JCSDA''' to access all the '''JEDI''' components that are part of the abstract layer infrastructure centered around the Object Oriented Prediction System ('''OOPS'''). <br />
<br />
Use the following command to download the '''ROMS-JEDI''' interface:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span></div><br />
<br />
==Requirements==<br />
<br />
* Make sure you have the '''spack-stack''' software installed for either '''gfortran''' or '''ifort''' containing several packages need to run the '''ROMS-JEDI''' interface. For more information, please check https://github.com/JCSDA/spack-stack.<br /><br />In our computers at Rutgers University, the '''spack-stack''' can be loaded by executing either:<div class="box"><span class="red">module purge; module load stack-gcc; module list</span> (gfortran compiler)<br /><span class="red">module purge; module load stack-intel; module list</span> (ifort compiler)</div>Installing the '''spack-stack''' is not trivial and requires computer software expertise.<br />
<br />
* We also like to define the '''MPIRUN''' environmental variable to specify the '''MPI''' executable in a particular computer (say, '''srun''', '''mpirun''', etc.). We need the full path for '''ctest''' to work. Notice that we specify this executable in the '''ecbuild''' command below.<br /><br />For example:<div class="box"><span class="twilightBlue">setenv MPIRUN /opt/slurm/bin/srun</span></div><br />
<br />
* Several environment variables can be activated for '''verbose debugging''' of the building blocks for the '''ROMS-JEDI''' interface: (use '''1''' or '''true''' to activate and '''0''' or '''false''' to deactivate)<div class="box"><span class="twilightBlue">setenv LdebugGeometry 1<br />setenv LdebugAnalyticInit 1<br />setenv LdebugField 1<br />setenv LdebugFields 1<br />setenv LdebugFieldsUtils 1<br />setenv LdebugGeometry 1<br />setenv LdebugLinearModel 1<br />setenv LdebugLinearModel2Geovals 1<br />setenv LdebugModel 1<br />setenv LdebugModel2Geovals 1<br />setenv LdebugTrajectory 1</span></div><br />
<br />
==Compiling ROMS-JEDI Default Application WC13==<br />
<br />
By default, '''ROMS-JEDI''' is configured with a coarse (54x53x30 grid points) US West Coast ('''WC13''') application. See [[4DVar Tutorial Introduction]] for detailed information about the '''WC13''' configuration. However, any other ROMS application can be configured by including in the '''ecbuild''' command the directives '''-D<span class="twilightBlue">ROMS_APP</span>=MyAppCPP''' and '''-D<span class="twilightBlue">ROMS_APP_DIR</span>=MyAppDirPath'''.<br />
<br />
The strategy is to create a '''Bundle''' and '''build''' subdirectory for each '''ROMS-JEDI''' application. For '''WC13''' (default), we create '''Bundle_wc13''' and '''build_wc13''' below. Please follow the following steps to configure '''ROMS-JEDI''' in your computer:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span> !> If the first time on your computer<br /><br /><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_wc13</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_wc13</span><br /><span class="red">mkdir build_wc13</span><br /><span class="red">cd build_wc13</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DCMAKE_BUILD_TYPE=Release ../Bundle_wc13</span><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Tests<br /><br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi/test/Data<br /> !> to check the results in various sub-directories</div><br />
<br />
The '''ecbuild''' command will clone the required '''JEDI''' components that are specified in '''Bundle_wc13/CMakeList.txt'''. For example:<div class="box"># Required repositories<br /><br /> ecbuild_bundle( PROJECT roms GIT "<nowiki>https://github.com/myroms/roms.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT oops GIT "<nowiki>https://github.com/jcsda-internal/oops.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT vader GIT "<nowiki>https://github.com/jcsda-internal/vader.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT saber GIT "<nowiki>https://github.com/jcsda-internal/saber.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT gsw GIT "<nowiki>https://github.com/jcsda-internal/GSW-Fortran.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ioda GIT "<nowiki>https://github.com/jcsda-internal/ioda.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ufo GIT "<nowiki>https://github.com/jcsda-internal/ufo.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT roms-jedi SOURCE ../ )</div><br />
<br />
The '''ecbuild''' tool builds by default the '''RelWithDebInfo''' ('''-O2 -g''' options) for optimized with debugging information version of '''JEDI''' and '''ROMS-JEDI''', rendering slower execution. However, for faster and optimized execution, we include the directive '''-DCMAKE_BUILD_TYPE=Release''' to the '''ecbuild''' command. For debugging with TotalView, we include instead the directive '''-DCMAKE_BUILD_TYPE=Debug''', which is much slower.<br />
<br />
==Compiling other ROMS-JEDI Applications==<br />
<br />
Any '''ROMS''' application can be run in '''ROMS-JEDI''' by specifying the appropriate CPP option and necessary input files. Only '''WC13''' is provided with the interface. However, we facilitate how to run such applications. Notice that the '''WC13''' configuration and input files are located in '''roms-jedi/test/Applications/wc13'''. The template input YAML files are located '''roms-jedi/test/templates'''. We use a '''Perl''' script to create the application input YAML files from the templates. Please check in '''roms-jedi/test/Applications/wc13/testinput'''.<br />
<br />
For example, if running the U.S. East Coast '''DOPPIO''' application (240x104x40 grid points) or any other, please follow the following steps to configure '''ROMS-JEDI''':<div class="box"><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_doppio</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_doppio</span><br /><span class="red">mkdir build_doppio</span><br /><span class="red">cd build_doppio</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DROMS_APP="DOPPIO" \<br />-DROMS_APP_DIR="DOPPIOpath" -DCMAKE_BUILD_TYPE=Release ../Bundle_doppio</span><br /><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_doppio/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Test<br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_doppio/roms-doppio/test/Data<br /> !> to check the results in various sub-directories</div></div>Arangohttps://www.myroms.org/wiki/index.php?title=ROMS-JEDI_Compiling&diff=6593ROMS-JEDI Compiling2023-08-31T05:06:18Z<p>Arango: /* Compiling ROMS-JEDI Default Application WC13 */</p>
<hr />
<div><div class="title">Compiling ROMS-JEDI</div><br />
<br />
<!-- Edit Template:ROMS-JEDI_TOC to modify this Table of Contents--><br />
<div style="float: left;margin: 0 20px 0 0;">{{ROMS-JEDI_TOC}}</div>__TOC__<br />
<br />
<div style="clear: both"></div><br />
<br />
<br />
==Configuring GIT==<br />
<br />
Before downloading the '''ROMS-JEDI''' repository, ensure that your '''~/.gitconfig''' has the appropriate '''git-lfs''' configuration for correctly downloading the default application (U.S. West Coast, '''WC13''' option) input and observation NetCDF files. Otherwise, the '''Unit Test Cases''' requiring input NetCDF files will fail. The '''Git LFS''' is a command line extension and specification for managing large files with '''Git'''. A sample of the configuration file looks like this:<div class="box"><span class="red">more ~/.gitconf</span><br /><br />[User]<br /> name = GivenName MiddleName FamilyName<br /> email = your@email<br />[credential]<br /> helper = cache --timeout=7200<br /> helper = store --file ~/.my-credentials<br />[filter "lfs"]<br /> clean = git-lfs clean -- %f<br /> smudge = git-lfs smudge -- %f<br /> process = git-lfs filter-process<br /> required = true</div><br />
<br />
Alternatively, you may execute '''git lfs pull''' at your location of this repository to download a viable version of the '''Git LFS''' files from the remote repository in '''GitHub'''. Also, to automatically add the '''LFS''' filter to your existing '''~/.gitconfig''', you could use '''git lfs install''' from anywhere in your computer directories.<br />
<br />
==Downloading ROMS-JEDI==<br />
<br />
The '''ROMS-JEDI''' repository is private within the Joint Center for Satellite Data Assimilation ('''JCSDA''') community. Users need to be partners, contributors, or members of '''JCSDA''' to access all the '''JEDI''' components that are part of the abstract layer infrastructure centered around the Object Oriented Prediction System ('''OOPS'''). <br />
<br />
Use the following command to download the '''ROMS-JEDI''' interface:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span></div><br />
<br />
==Requirements==<br />
<br />
* Make sure you have the '''spack-stack''' software installed for either '''gfortran''' or '''ifort''' containing several packages need to run the '''ROMS-JEDI''' interface. For more information, please check https://github.com/JCSDA/spack-stack.<br /><br />In our computers at Rutgers University, the '''spack-stack''' can be loaded by executing either:<div class="box"><span class="red">module purge; module load stack-gcc; module list</span> (gfortran compiler)<br /><span class="red">module purge; module load stack-intel; module list</span> (ifort compiler)</div>Installing the '''spack-stack''' is not trivial and requires computer software expertise.<br />
<br />
* We also like to define the '''MPIRUN''' environmental variable to specify the '''MPI''' executable in a particular computer (say, '''srun''', '''mpirun''', etc.). We need the full path for '''ctest''' to work. Notice that we specify this executable in the '''ecbuild''' command below.<br /><br />For example:<div class="box"><span class="twilightBlue">setenv MPIRUN /opt/slurm/bin/srun</span></div><br />
<br />
* Several environment variables can be activated for '''verbose debugging''' of the building blocks for the '''ROMS-JEDI''' interface: (use '''1''' or '''true''' to activate and '''0''' or '''false''' to deactivate)<div class="box"><span class="twilightBlue">setenv LdebugGeometry 1<br />setenv LdebugAnalyticInit 1<br />setenv LdebugField 1<br />setenv LdebugFields 1<br />setenv LdebugFieldsUtils 1<br />setenv LdebugGeometry 1<br />setenv LdebugLinearModel 1<br />setenv LdebugLinearModel2Geovals 1<br />setenv LdebugModel 1<br />setenv LdebugModel2Geovals 1<br />setenv LdebugTrajectory 1</span></div><br />
<br />
==Compiling ROMS-JEDI Default Application WC13==<br />
<br />
By default, '''ROMS-JEDI''' is configured with a coarse (54x53x30 grid points) US West Coast ('''WC13''') application. See [[4DVar Tutorial Introduction]] for detailed information about the '''WC13''' configuration. However, any other ROMS application can be configured by including in the '''ecbuild''' command the directives '''-D<span class="twilightBlue">ROMS_APP</span>=MyAppCPP''' and '''-D<span class="twilightBlue">ROMS_APP_DIR</span>=MyAppDirPath'''.<br />
<br />
The strategy is to create a '''Bundle''' and '''build''' subdirectory for each '''ROMS-JEDI''' application. For '''WC13''' (default), we create '''Bundle_wc13''' and '''build_wc13''' below. Please follow the following steps to configure '''ROMS-JEDI''' in your computer:<div class="box"><span class="red">git clone <nowiki>https://github.com/JCSDA-internal/roms-jedi.git</nowiki></span> !> If the first time on your computer<br /><br /><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_wc13</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_wc13</span><br /><span class="red">mkdir build_wc13</span><br /><span class="red">cd build_wc13</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DCMAKE_BUILD_TYPE=Release ../Bundle_wc13</span><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Tests<br /><br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_wc13/roms-jedi/test/Data<br /> !> to check the results in various sub-directories</div><br />
<br />
The '''ecbuild''' command will clone the required '''JEDI''' components that are specified in '''Bundle_wc13/CMakeList.txt'''. For example:<div class="box"># Required repositories<br /><br /> ecbuild_bundle( PROJECT roms GIT "<nowiki>https://github.com/JCSDA-internal/roms_src.git</nowiki>" BRANCH master UPDATE )<br /> ecbuild_bundle( PROJECT oops GIT "<nowiki>https://github.com/jcsda-internal/oops.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT vader GIT "<nowiki>https://github.com/jcsda-internal/vader.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT saber GIT "<nowiki>https://github.com/jcsda-internal/saber.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT gsw GIT "<nowiki>https://github.com/jcsda-internal/GSW-Fortran.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ioda GIT "<nowiki>https://github.com/jcsda-internal/ioda.git</nowiki>" BRANCH develop UPDATE )<br /> ecbuild_bundle( PROJECT ufo GIT "<nowiki>https://github.com/jcsda-internal/ufo.git</nowiki>" BRANCH develop UPDATE )<br /><br /> ecbuild_bundle( PROJECT roms-jedi SOURCE ../ )</div><br />
<br />
The '''ecbuild''' tool builds by default the '''RelWithDebInfo''' ('''-O2 -g''' options) for optimized with debugging information version of '''JEDI''' and '''ROMS-JEDI''', rendering slower execution. However, for faster and optimized execution, we include the directive '''-DCMAKE_BUILD_TYPE=Release''' to the '''ecbuild''' command. For debugging with TotalView, we include instead the directive '''-DCMAKE_BUILD_TYPE=Debug''', which is much slower.<br />
<br />
==Compiling other ROMS-JEDI Applications==<br />
<br />
Any '''ROMS''' application can be run in '''ROMS-JEDI''' by specifying the appropriate CPP option and necessary input files. Only '''WC13''' is provided with the interface. However, we facilitate how to run such applications. Notice that the '''WC13''' configuration and input files are located in '''roms-jedi/test/Applications/wc13'''. The template input YAML files are located '''roms-jedi/test/templates'''. We use a '''Perl''' script to create the application input YAML files from the templates. Please check in '''roms-jedi/test/Applications/wc13/testinput'''.<br />
<br />
For example, if running the U.S. East Coast '''DOPPIO''' application (240x104x40 grid points) or any other, please follow the following steps to configure '''ROMS-JEDI''':<div class="box"><span class="red">cd roms-jedi</span> !> ROMS-JEDI interface directory, <interface_dir><br /><span class="red">mkdir Bundle_doppio</span><br /><span class="red">cp bundle/.gitignore bundle/CMakeLists.txt Bundle_doppio</span><br /><span class="red">mkdir build_doppio</span><br /><span class="red">cd build_doppio</span><br /><br /><span class="red">ecbuild -DMPIEXEC_EXECUTABLE=$MPIRUN -DMPIEXEC_NUMPROC_FLAG="-n" -DROMS_APP="DOPPIO" \<br />-DROMS_APP_DIR="DOPPIOpath" -DCMAKE_BUILD_TYPE=Release ../Bundle_doppio</span><br /><br /><span class="red">make -j 10</span><br /><br /><span class="red">cd roms-jedi</span> !> sub-directory: <interface_dir>/build_doppio/roms-jedi<br /><span class="red">ctest -N</span> !> lists all the Unit Tests available<br /><span class="red">ctest -E -V get_</span> !> runs all the Unit Test<br /><span class="red">cd test/Data</span> !> sub-directory: <interface_dir>/build_doppio/roms-doppio/test/Data<br /> !> to check the results in various sub-directories</div></div>Arangohttps://www.myroms.org/wiki/index.php?title=Documentation_Portal&diff=6592Documentation Portal2023-08-31T04:58:22Z<p>Arango: </p>
<hr />
<div><div class="title">Documentation Portal</div><br />
<br />
{|class="main" width="99%"<br />
|style="vertical-align:top;width:33%" | <!-- Left Column Portal --><br />
'''Technical Documentation'''<br />
* [[Regional_Ocean_Modeling_System_(ROMS) | Introduction]]<br />
* [[Equations of Motion]]<br />
* [[Vertical S-coordinate]]<br />
* [[Terrain-Following Coordinate Transformation]]<br />
* [[Curvilinear Coordinates Transformation]]<br />
* [[Fractional Coordinate System (&xi; - &eta; space)]]<br />
* [[Time-stepping Schemes Review]]<br />
* [[Numerical Solution Technique]]<br />
* [[Nested Grids]]<br />
* [[Boundary Conditions]]<br />
* [[Horizontal Mixing]]<br />
* [[Vertical Mixing Parameterizations]]<br />
* [[Atmospheric Boundary Layer]]<br />
* [[Bottom Boundary Layer]]<br />
* [[Lagrangian Drifters]]<br />
* [[Balance Term Diagnostics]]<br />
* [[Biogeochemical Models]]<br />
* [[Sediment Model]]<br />
* [[Wave Model]]<br />
* [[Atmospheric Model]]<br />
* [[Model Coupling]]<br />
** [[NUOPC_Cap | ROMS Native ESMF/NUOPC Cap]]<br />
** [[NUOPC_Cap_UFS | ROMS Stand-Alone ESMF/NUOPC Cap]]<br />
* [[Model Diagnostics]]<br />
* [[Sea-Ice Model]]<br />
* [[Variational Data Assimilation]]<br />
* [[ROMS-JEDI_Intro| ROMS-JEDI]]<br />
** [[ROMS-JEDI_Compiling | Compiling]]<br />
** [[ROMS-JEDI_Implementation | Implementation]]<br />
** [[ROMS-JEDI_Observations | Observations]]<br />
** [[ROMS-JEDI_Unit_Test_Cases | Unit Test Cases]]<br />
* [[IO | ROMS I/O]]<br />
* [[Bibliography]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Middle Column Portal --><br />
'''User Guide'''<br />
* [[Getting Started]]<br />
* [[Tutorials]]<br />
* [[Nesting Examples]]<br />
* [[Frequently Asked Questions]]<br />
* [[Source Code]]<br />
** [[Git]]<br />
** [[Subversion]]<br />
** [[Modules]]<br />
** [[Drivers]]<br />
** [[Subroutines]]<br />
** [[Variables]]<br />
** [[Functionals]]<br />
** [[Lateral Boundary Conditions]]<br />
* [[C Preprocessor]]<br />
** [[Options]]<br />
** [[cppdefs.h]]<br />
** [[globaldefs.h]]<br />
* [[Compiling and Linking]]<br />
** [[makefile]]<br />
** [[build_roms]]<br />
** [[my_build_paths]]<br />
** [[make]]<br />
** [[gmake]]<br />
** [[MakeDepend]]<br />
** [[External Libraries]]<br />
* [[Parallelization]]<br />
* [[Benchmarking]]<br />
* [[Matlab Scripts]]<br />
* [[Visualization]]<br />
|style="vertical-align:top;width:33%;border-left:1px solid #aaa;padding-left:1em"| <!-- Right Column Portal --><br />
'''Examples and Applications'''<br />
* [[Test Cases]]<br />
* [[Model Coupling ESMF | Coupling Examples]]<br />
* [[Lake_Jersey_Refinement_Example_Introduction | Nesting Examples]]<br />
* [[Application Set-up]]<br />
* [[Metadata Design]]<br />
** [[YAML_Parser | YAML Parser]]<br />
** [[coupling_esmf_atm.yaml]]<br />
** [[roms_cmeps.yaml]] <br />
** [[varinfo.yaml]]<br />
* [[Grid Generation]]<br />
* [[Initialization]]<br />
* [[Forcing]]<br />
** [[Atmospheric Forcing]]<br />
** [[Tidal Forcing]]<br />
** [[River Runoff]]<br />
* [[Input Parameter Files]]<br />
** [[roms.in]]<br />
** [[s4dvar.in]]<br />
** [[biology.in]]<br />
** [[sediment.in]]<br />
** [[floats.in]]<br />
** [[stations.in]]<br />
** [[ice.in]]<br />
** [[swan.in]]<br />
* [[Standard Output Files]]<br />
|}<br />
----<br />
<br />
<br />
<center>[[Image:ROMS_circle_logo.png]] &nbsp; [[Image:ROMS-JEDI_circle_logo.png]]</center></div>Arangohttps://www.myroms.org/wiki/index.php?title=ROMS-JEDI_Intro&diff=6591ROMS-JEDI Intro2023-08-31T04:56:47Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS-JEDI</div><br />
<br />
<div style="clear: both"></div><br />
<br />
<!-- Edit Template:ROMS-JEDI_TOC to modify this Table of Contents--><br />
<div style="float: left;margin: 0 20px 0 0;">{{ROMS-JEDI_TOC}}</div><br />
<br />
<div style="clear: both"></div><br />
<br />
<br />
[[Image:romsjedi_logo.png|left]]<br />
[[Image:romsjedi_schematic.png|1000px|left]]<br />
<div style="clear: both"></div><br />
<br />
<br />
==Introduction==<br />
The Joint Effort for Data Assimilation Integration ('''JEDI''') framework is a unified, high-level, state-of-the-art data assimilation ('''DA''') system for scientific exploration and operational forecasting. The diagram above shows that the '''JEDI''' building blocks are model agnostic, and any modern geophysical numerical kernels can be interfaced. Its separable generic design and abstraction allow the implementation of various '''DA''' methodologies, including variational, ensemble, hybrid, and coupled approaches that are common to any geophysical system, without the need to develop such algorithms separately for each model technically. The '''JEDI''' framework is centered around the Object-Oriented Prediction System ('''OOPS''') layer classes and operators implemented in '''C++''' using templates. The specific implementations like '''ROMS-JEDI''' are written in Fortran 2003 because of the need for class objects, polymorphism, inheritance, and interoperability between '''C++''' and '''Fortran'''. The interoperability binding allows '''Fortran''' to invoke '''C++''' functions and vice versa '''C++''' to invoke '''Fortran''' procedures.<br />
<br />
The '''JEDI''' separation of concerns design allows the incorporation of diverse (current and future) observational data streams essential for any DA system. Observation operators can be efficiently coded and added to the Unified Forward Operator ('''UFO''') interface, which simulates, interpolates, and relates observable data to the model component state vector, '''H(x)'''. The '''UFO''' interface also implements quality control, data thinning, and bias correction. The Interface for Observation Data Access ('''IODA''') storage layer ingests observational data from providers into the '''UFO''' and '''OOPS''' building blocks. The immense amount of incoming data can be trimmed down, quality-controlled, and filtered by '''UFO''' operators before it is assimilated and during postprocessing when computing diagnostics, impacts, and sensitivities.<br />
<br />
The '''ROMS-JEDI''' framework is intended for users interested in ocean prediction and data assimilation. Currently, we use the '''JEDI''' framework to incorporate physical observational data into regional '''ROMS''' applications using variational ('''3DVar'''/'''4DVar''', '''3D-FGAT'''/'''4D-FGAT'''), ensemble ('''EDA''', '''EnKF''', '''LETKF'''), and hybrid ('''3DEnVar'''/'''4DEnVar''') data assimilation methodologies. The '''ROMS''' Tangent Linear Model ('''TLM''') and Adjoint Model ('''ADM''') kernels are used in the '''4DVar''' family of algorithms.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6564IO2023-03-11T20:51:32Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records <span class="mediumOrchid">(file with only 2 cycling time records)</span><br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 365.0d0 ! days <span class="mediumOrchid">(ROMS is initialized on 2007-01-01 00:00:00Z)</span><br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days <span class="mediumOrchid">(zero phase date is set when preparing the tidal data for 2006-01-01)</span><br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd <span class="mediumOrchid">(Very important: ROMS time is seconds since 2006-01-01 00:00:00Z)</span></div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain '''5'''-time records because of the initial conditions, and the rest of the files will have '''4'''-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain '''9'''-time records, and the rest will have '''8'''-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6563IO2023-03-11T20:49:59Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records <span class="mediumOrchid">(file with only 2 cycling time records)</span><br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 365.0d0 ! days <span class="mediumOrchid">(ROMS is initialized on 2007-01-01 00:00:00Z)</span><br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days <span class="mediumOrchid">(zero phase date is set when preparing the tidal data for 2006-01-01)</span><br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd <span class="mediumOrchid">(Very important: ROMS time is seconds since 2006-01-01 00:00:00Z)</span></div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6562IO2023-03-11T20:43:30Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records <span class="mediumOrchid">(file with only 2 cycling time records)</span><br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 365.0d0 ! days <span class="mediumOrchid">(ROMS is initialized on 2007-01-01 00:00:00Z)</span><br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd <span class="mediumOrchid">(Very important: ROMS time is seconds since 2006-01-01 00:00:00Z)</span></div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6561IO2023-03-11T20:39:56Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records <span class="mediumOrchid">(file with only 2 cycling time records)</span><br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 365.0d0 ! days <span class="mediumOrchid">(ROMS is initialized on 2007-01-01 00:00:00Z)</span><br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6560IO2023-03-11T20:32:49Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records <span class="mediumOrchid">(file with only 2 cycling time records)</span><br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6559IO2023-03-11T20:26:49Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. If [[Variables#dstart|DSTART]] is changed, it will reset the timestep internal counter <span class="red">iic(ng)</span>, and we are in deep trouble! Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6558IO2023-03-11T20:21:58Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the <span class="twilightBlue">averages</span> and <span class="twilightBlue">diagnostic</span> files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You '''cannot''' change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. Notice that the value [[Variables#nRST|NRST]] is crucial for the restart of ROMS with multi-file. In addition, we need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]] because of the accumulation sums when computing time-averaging fields. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6557IO2023-03-11T20:13:50Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular region with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds, as shown below. Then, one could split the <span class="twilightBlue">history</span> and <span class="twilightBlue">quicksave</span> output data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the averages and diagnostic files are single records files representing daily averaged fields. The above analysis also holds when converted to time in seconds since every parameter is multiplied by the [[Variables#dt|DT]] timestep.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You cannot change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. Notice that the value [[Variables#nRST|NRST]] is crucial for all the restart of ROMS with multi-file. We need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]]. If balancing terms and budgets from output data, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6556IO2023-03-11T19:56:46Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output data (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular area with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds. Then, one could split the <span class="twilightBlue">history</span> and quicksave data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the averages and diagnostic files are single records files representing daily averaged fields.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You cannot change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. Notice that the value [[Variables#nRST|NRST]] is crucial for all the restart of ROMS with multi-file. We need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]]. If doing balancing terms and budgets, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6555IO2023-03-11T19:50:25Z<p>Arango: /* Output Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular area with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds. Then, one could split the <span class="twilightBlue">history</span> and quicksave data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day; 365 days simulation)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 366.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 20060101.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the averages and diagnostic files are single records files representing daily averaged fields.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You cannot change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. Notice that the value [[Variables#nRST|NRST]] is crucial for all the restart of ROMS with multi-file. We need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]]. If doing balancing terms and budgets, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6554IO2023-03-11T19:45:28Z<p>Arango: /* Ouput Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Output Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular area with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds. Then, one could split the <span class="twilightBlue">history</span> and quicksave data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div><br />
<br />
* Timestamp assigned for model initialization, reference time origin for tidal forcing, and model reference time for output NetCDF units attribute.<div class="box"> [[Variables#dstart|DSTART]] = 0.0d0 ! days<br /> [[Variables#tide_start|TIDE_START]] = 0.0d0 ! days<br /> [[Variables#time_ref|TIME_REF]] = 0.0d0 ! yyyymmdd.dd</div><br />
<br />
Notice that all the parameters are an exact integer multiple from each other:<br />
<div class="box"> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFHIS]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefHIS|NDEFQCK]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefAVG|NDEFAVG]]) = 0 <br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#ndefDIA|NDEFDIA]]) = 0 <br /><br /> MOD([[Variables#ntimes|NTIMES]], [[Variables#nRST|NRST]]) = 0 <br /><br /> MOD([[Variables#ndefHIS|NDEFHIS]], [[Variables#nHIS|NHIS]]) = 0 <br /> MOD([[Variables#ndefQCK|NDEFQCK]], [[Variables#nQCK|NQCK]]) = 0 <br /> MOD([[Variables#ndefAVH|NDEFAVG]], [[Variables#nAVG|NAVG]]) = 0 <br /> MOD([[Variables#ndefDIA|NDEFDIA]], [[Variables#nHIS|NDIA]]) = 0 <br /></div><br />
<br />
where the Fortran intrinsic function '''MOD(<span class="red">X</span>,<span class="red">Y</span>)''' computes the remainder of the division of <span class="red">X</span> by <span class="red">Y</span>, and has to be always '''zero''' for ROMS multi-file option to work. Notice that the first files in the <span class="twilightBlue">history</span> data series (<span class="violet">roms_his_0001.nc</span>) will contain 5-time records because of the initial conditions, and the rest of the files will have 4-time records. Similarly, the first file in the <span class="twilightBlue">quicksave</span> series (<span class="violet">roms_qck_0001.nc</span>) will contain 9-time records, and the rest will have 8-time records. The time-averaged data in the averages and diagnostic files are single records files representing daily averaged fields.<br />
<br />
<span class="red">WARNING:</span> If running your application in a supercomputer with limited-time job queues, long simulations will require a restart. You cannot change the values of [[Variables#ntimes|NTIMES]] or [[Variables#dstart|DSTART]] because the multi-file will fail. Notice that the value [[Variables#nRST|NRST]] is crucial for all the restart of ROMS with multi-file. We need the following mathematical equalities:<br />
<div class="box"> MOD([[Variables#nRST|NRST]], [[Variables#nHIS|NHIS]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nQCK|NQCK]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nAVG|NAVG]]) = 0<br /> MOD([[Variables#nRST|NRST]], [[Variables#nDIA|NDIA]]) = 0<br /></div><br />
Also, it will be trivial to restart if [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] = [[Variables#nRST|NRST]]. If doing balancing terms and budgets, you will need [[Variables#nAVG|NAVG]] = [[Variables#nDIA|NDIA]] too.</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6553IO2023-03-11T18:50:12Z<p>Arango: /* Ouput Multi-Files */</p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Ouput Multi-Files==<br />
<br />
Sometimes, it is advantageous to time-split ROMS output (<span class="twilightBlue">averages</span>, <span class="twilightBlue">diagnostic</span>, <span class="twilightBlue">history</span>, and <span class="twilightBlue">quicksave</span>) into multiple NetCDF files to avoid creating huge files on disk for storage in applications with large grids. Smaller files are easy to handle and can be concatenated in OpenDAP catalogs.<br />
<br />
For example, an application for a particular area with a substantial grid size needs to be run for one year. The ROMS timestep is [[Variables#DT|DT]] = 300 seconds. Then, one could split the <span class="twilightBlue">history</span> and quicksave data into daily NetCDF files with single or multiple time records every '''3''', '''6''', or '''24''' hours. In this case, ROMS will generate a sequence of files with suffixes <span class="violet">_0001.nc</span> to <span class="violet">_0365.nc</span> filenames. Therefore, we need the following parameters in [[roms.in|roms.in]]:<br />
<br />
* Timestepping parameters.<div class="box"> [[Variables#ntimes|NTIMES]] = 105120 ! Number of timesteps <span class="mediumOrchid">(288 steps per day)</span><br /> [[Variables#dt|DT]] == 300.0d0 ! Timestep size (seconds)<br /> [[Variables#ndtfast|NDTFAST]] == 30 ! Number of barotropic steps</div><br />
<br />
* Flags controlling the frequency of output.<div class="box"> [[Variables#nrrec|NRREC]] = 0 ! Model restart flag<br /> [[Variables#LcycleRST|LcycleRST]] == T ! Switch to recycle restart time records<br /> [[Variables#nRST|NRST]] == 288 ! Number of timesteps between writing restart records <span class="mediumOrchid">(daily)</span><br /> [[Variables#nSTA|NSTA]] == 1 ! Number of timesteps between stations records<br /> [[Variables#nFLT|NFLT]] == 1 ! Number of timesteps between floats records<br /> [[Variables#ninfo|NINFO]] == 1 ! Number of timesteps between printing information diagnostics</div><br />
<br />
* Output history, average, and diagnostic file parameters.<div class="box"> [[Variables#ldefout|LDEFOUT]] == T ! File creation/append switch<br /> [[Variables#nHIS|NHIS]] == 72 ! Number of timesteps between writing history records <span class="mediumOrchid">(every 6 hours)</span><br /> [[Variables#ndefHIS|NDEFHIS]] == 288 ! Number of timesteps between the creation of new history files <span class="mediumOrchid">(daily files)</span><br /> [[Variables#nQCK|NQCK]] == 36 ! Number of timesteps between writing quicksave records <span class="mediumOrchid">(every 3 hours)</span><br /> [[Variables#ndefQCK|NDEFQCK]] == 288 ! Number of timesteps between the creation of new quicksave file <span class="mediumOrchid">(daily)</span><span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsAVG|NTSAVG]] == 1 ! Starting averages timestep<br /> [[Variables#nAVG|NAVG]] == 288 ! Number of timesteps between writing averages records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefAVG|NDEFAVG]] == 288 ! Number of timesteps between the creation of new averages file <span class="mediumOrchid">(daily, single record files)</span><br /> [[Variables#ntsDIA|NTSDIA]] == 1 ! Starting diagnostics timestep<br /> [[Variables#nDIA|NDIA]] == 288 ! Number of timesteps between writing diagnostics records <span class="mediumOrchid">(daily averages)</span><br /> [[Variables#ndefDIA|NDEFDIA]] == 288 ! Number of timesteps between the creation of new diagnostics file <span class="mediumOrchid">(daily, single record files)</span></div></div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6552IO2023-03-11T16:50:55Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}<br />
<br />
==Ouput Multi-Files==</div>Arangohttps://www.myroms.org/wiki/index.php?title=IO&diff=6551IO2023-03-11T16:48:25Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS I/O</div><br />
<br />
ROMS uses NetCDF for all its input and output data management. The NetCDF files can be processed using the standard library developed by [http://doi.org/10.5065/D6H70CW6 UNIDATA], the Parallel-IO ('''PIO''') library developed at NCAR (Hartnett and Edwards, 2021; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE unpublished paper]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library intended for the DOE's Energy Exascale Earth Model System ([https://e3sm.org/scorpio-parallel-io-library/ E<sup>3</sup>SM]). Furthermore, another parallel I/O strategy has been available in ROMS for several years with the NetCDF4/HDF5 libraries by activating the [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options.<br />
<br />
The '''SCORPIO''' library was forked from the '''PIO''' library several years ago and evolved separately. The generic interface for parallel I/O in ROMS works for both the '''PIO''' or '''SCORPIO''' libraries and available by activating the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="red">However</span>, we recommend using the '''PIO''' library because it is more efficient in processing I/O in our benchmark tests.<br />
<br />
Generally, writing is usually a more frequent and more complicated operation than reading. There are four strategies for writing (Mendez ''et al.'', 2019; [https://www.researchgate.net/publication/332190037_Best_Practice_Guide_-_Parallel_IO Preprint]):<br />
<br />
#<span class="blue">Single file, single writer:</span> '''Serial I/O''' in non-parallel or parallel applications. It is the default strategy in ROMS using the '''NetCDF3''' or '''NetCDF4''' libraries. <br />
#<span class="blue">Single file, multiple writers:</span> '''Parallel I/O''' with each partition tile writing its data to a single file. In ROMS, this capability is achieved by activating [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]]. It is only possible with the '''NetCDF4/HDF5''' libraries. <br />
#<span class="blue">Single file, collective writers:</span> '''Parallel I/O''' with either one or a subset of processes performing I/O operations. The I/O operations can be '''synchronous''' or '''asynchronous'''. In ROMS, this capability uses the '''PIO''' or '''SCORPIO''' libraries and available when [[Options#PIO_LIB|PIO_LIB]] is activated. <br />
#<span class="blue">Multiple files, multiple writers:</span> '''Parallel I/O''' in which each distributed-memory or shared-memory tile decomposition writes its data into the partition file. Still, post-processing is required to pack the data into a single file. Currently, this capability is not available in ROMS but can be easily implemented within its current I/O infrastructure. However, this strategy is cumbersome and undesired in variational data assimilation (4D-Var) algorithms that require reading forwards and backward, in time, the state trajectories. As a consequence, reading becomes the bottleneck.<br />
<br />
__TOC__<br />
<br />
==Serial I/O==<br />
Serial I/O is the standard option that has been in ROMS since the beginning. In this setup, all input and output data flows through the master parallel process. All data for output is collected from all processes by the master process and written to disk. Likewise, all data for input is read by the master process and distributed to the rest of the processes. When using serial I/O, files can be written in NetCDF classic/64-bit offset (NetCDF-3) or NetCDF-4/HDF5 ([[Options#HDF5|HDF5]] CPP option) formats. File compression is available in the NetCDF-4/HDF5 library.<br />
<br />
==Parallel I/O with NetCDF-4==<br />
Parallel I/O using parallel HDF5 and NetCDF-4 has been available in ROMS for many years. This I/O option requires parallel enabled HDF5 and NetCDF-4 and is activated by defining [[Options#PARALLEL_IO|PARALLEL_IO]] and [[Options#HDF5|HDF5]] CPP options. Each parallel tile reads and writes the decomposed data. This approach does not scale well because it requires every process to participate in reading and writing, which quickly overloads the file system with requests as the number of tiles (NtileI x NtileJ) increases.<br />
<br />
==Parallel I/O with PIO or SCORPIO==<br />
<br />
'''PIO''' and '''SCORPIO''' libraries are primarily intended for ROMS '''distributed-memory (MPI) applications''' running on a large number of processes in an HPC system with a Parallel File System (like Lustre, GPFS, and so on) for high-performance I/O. The '''PIO''' and '''SCORPIO''' library uses the '''MPI-IO''' interface to facilitate the partitioning of the data across computational or dedicated I/O processes. For example, in an HPC cluster environment with a '''Parallel File System''', the user can instruct '''PIO''' or '''SCORPIO''' to designate which processes per node to perform I/O. This is a much more reasonable approach for larger applications running on hundreds of processors. To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time by defining the [[Options#PIO_LIB|PIO_LIB]] CPP option. <span class="blue">It is only available in distributed-memory applications since it uses '''MPI-IO'''</span>.<br />
<br />
There are two modes of parallel I/O in '''PIO''' and '''SCORPIO''':<br />
<br />
<ol style="list-style-type: upper-alpha"><br />
<li>'''Synchronous:''' MPI intra-communications mode. A subset or all processes does I/O and also computations. The user specifies the total number of I/O tasks and how they are distributed across the HPC nodes as a function of the ROMS MPI-communicator object, '''OCN_COMM_WORLD'''. It is often desirable to shift the first I/O task away from the first computation task since it has higher memory requirements than other processes. If the MPI processes are spread over several computer nodes, it is highly recommended to spread all I/O tasks over all nodes. Avoid all I/O processes occupying the same node.</li><br />
<li>'''Asynchronous:''' MPI inter-communications mode. The I/O tasks are a disjointed set of dedicated I/O processes and do not perform computations. It is possible to have groups of computational units running separate models (coupling) where all the I/O data are sent to dedicated processes. In ROMS, the asynchronous mode is possible by activation either [[Options#ASYNCHRONOUS_PIO|ASYNCHRONOUS_PIO]] or [[Options#ASYNCHRONOUS_SCORPIO|ASYNCHRONOUS_SCORPIO]]. '''<span class="red">However</span>, this capability is still under development and not recommended for use at this time.'''</li><br />
</ol><br />
<br />
The Parallel I/O configuration options are set in [[roms.in]]:<br />
{{#lst:roms.in|PIO}}</div>Arangohttps://www.myroms.org/wiki/index.php?title=YAML_Parser&diff=6547YAML Parser2023-03-08T15:50:05Z<p>Arango: </p>
<hr />
<div><div class="title">YAML Parser</div><br />
Starting with '''svn''' revision '''-r 1902''' released on March 1, 2022, the ROMS metadata is managed with a [[varinfo.yaml|YAML file]], and the regular text file '''varinfo.dat''' is deprecated. The '''YAML''' files are simple, easy to follow, elegant, portable, and expandable. ROMS can now process YAML files with its parser module, '''[https://www.myroms.org/doxygen/yaml__parser_8F_source.html yaml_parser.F]'''. Therefore, there is no need to use third-party '''YAML''' parsers.<br />
<br />
The ROMS '''YAML''' parser source code can be found in <span class="forestGreen">ROMS/Utility</span>. It is written in Fortran 2003 and includes a '''CLASS''' of type '''yaml_tree''' for parsing input '''YAML''' files. You may download our YAML parser [https://www.myroms.org/links/yaml_parser.F here]. <br />
<br />
==Introduction==<br />
Although several '''YAML''' parsers for Fortran exist, a more straightforward and uncomplicated parser with substantial capabilities was coded in '''ROMS''', '''[https://www.myroms.org/links/yaml_parser.F yaml_parser.F]'''. It is a hybrid between standard and Object-Oriented Programming (OOP) principles but without the need for recurrency, polymorphism, and containers (another library).<br />
<br />
The only constraint in the '''ROMS''' parser is that the '''YAML''' file is read '''twice''' for simplicity and to avoid containers. The container is a Fortran vector! The first read determines the indentation policy and the length of the collection vector, <span class="red">list(:)</span> pairs object (derived-type structure <span class="forestGreen">yaml_pair</span>). The first reading is quick.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_pair</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_alias</span> <span class="twilightBlue">! alias <span class="violet">*</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_anchor</span> <span class="twilightBlue">! anchor <span class="violet">&</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_block</span> <span class="twilightBlue">! block <span class="violet">-</span> list</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_sequence</span> <span class="twilightBlue">! sequence <span class="violet">[]</span> tokens</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_logical</span> <span class="twilightBlue">! logical value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_integer</span> <span class="twilightBlue">! integer value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_real</span> <span class="twilightBlue">! floating-point value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_string</span> <span class="twilightBlue">! string value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">id</span> <span class="twilightBlue">! key/value ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">parent_id</span> <span class="twilightBlue">! parent ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">left_padding</span> <span class="twilightBlue">! indent level: 0,1,..</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">line</span> <span class="twilightBlue">! YAML line</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">key</span> <span class="twilightBlue">! YAML keyword</span><span class="violet">:</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">value</span> <span class="twilightBlue">! YAML value(s)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">anchor</span> <span class="twilightBlue">! anchor keyword</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_pair</span></div><br />
The '''YAML''' file dictionary <span class="darkTurquoise">CLASS</span> <span class="forestGreen">yaml_tree</span> is defined as:<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_tree</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsCreated</span> '''= .FALSE.''' <span class="twilightBlue">! YAML object creation switch</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nbranches</span> <span class="twilightBlue">! total number of branches</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Npairs</span> <span class="twilightBlue">! total number of pairs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">indent</span> <span class="twilightBlue">! blank indentation policy</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">filename</span> <span class="twilightBlue">! YAML file name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_pair</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">list(:)</span> <span class="twilightBlue">! collection pairs</span><br /><br /> <span class="darkTurquoise">CONTAINS</span> <span class="twilightBlue">! CLASS objects</span><br /><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">create</span> => <span class="blue">yaml_tree_create</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">destroy</span> => <span class="blue">yaml_tree_destroy</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">dump</span> => <span class="blue">yaml_tree_dump</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">extract</span> => <span class="blue">yaml_tree_extract</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill</span> => <span class="blue">yaml_tree_fill</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill_aliases</span> => <span class="blue">yaml_tree_fill_aliases</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">has</span> => <span class="blue">yaml_tree_has</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">read_line</span> => <span class="blue">yaml_tree_read_line</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_tree</span></div><br />
The <span class="forestGreen">yaml_tree</span> object stores all the data contained in a specific '''YAML''' file. For Example, in '''ROMS''' the input '''YAML''' metadata dictionary is created and initialized as follows:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="blue">ONLY</span> : <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lreport</span> <span class="twilightBlue">! verbose report switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_tree</span>) :: <span class="red">self</span> <span class="twilightBlue">! declare a dummy YAML object</span><br /><br /> <span class="darkTurquoise">IF</span> (.<span class="darkTurquoise">not</span>.<span class="red">self%IsCreated</span>) <span class="darkTurquoise">THEN</span> <span class="twilightBlue">! process input YAML file</span><br /> <span class="red">Lreport</span> = <span class="red">.TRUE.</span><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_initialize</span> (<span class="red">self</span>, <span class="red">'varinfo.yaml'</span>, <span class="red">Lreport</span>)<br /> <span class="darkTurquoise">END IF</span></div><br />
The error management is omitted for clarity. Then, the needed data is extracted from the <span class="red">self</span> object and loaded into the internal '''ROMS''' variables using the overloaded <span class="mediumOrchid">yaml_get</span> API:<br />
<br />
<div class="code"> <span class="darkTurquoise">INTERFACE</span> <span class="mediumOrchid">yaml_get</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_i_struc</span> <span class="twilightBlue">! Gets integer structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_l_struc</span> <span class="twilightBlue">! Gets logical structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_r_struc</span> <span class="twilightBlue">! Gets real structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_s_struc</span> <span class="twilightBlue">! Gets string structure</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_0d</span> <span class="twilightBlue">! Gets integer value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_1d</span> <span class="twilightBlue">! Gest integer values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_0d</span> <span class="twilightBlue">! Gets logical value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_1d</span> <span class="twilightBlue">! Gets logical values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_0d</span> <span class="twilightBlue">! Gets real value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_1d</span> <span class="twilightBlue">! Gets real values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_0d</span> <span class="twilightBlue">! Gets string value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_1d</span> <span class="twilightBlue">! Gets string values</span><br /><br /> <span class="darkTurquoise">END INTERFACE</span> <span class="mediumOrchid">yaml_get</span></div><br />
Additionally, the '''YAML''' parser module has the following public and private routines/functions used during processing:<br />
<br />
<div class="code"> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_AssignString</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_Error</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_get</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_CountKeys</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_LowerCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_UpperCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_ValueType</span></div><br />
<br />
The parser module is self-contained with minimal dependencies on other '''ROMS''' modules:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">mod_kinds</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">kind_real</span> => <span class="red">dp</span> <span class="twilightBlue">! double-precision</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_parallel</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_Master</span> => <span class="red">Master</span> <span class="twilightBlue">! master PET</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_scalars</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_ErrFlag</span> => <span class="red">exit_flag</span> <span class="twilightBlue">! error flag</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_iounits</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_stdout</span> => <span class="red">stdout</span> <span class="twilightBlue">! standard ouput</span></div><br />
<br />
Overall, the parser is very fast and works in parallel. All PETs are involved in their dictionary copy to avoid overhead from collective MPI calls.<br />
<br />
==Capabilities==<br />
Currently, the '''YAML''' parser supports the following features:<br />
<br />
*'''Single''' or '''multiple''' line comments start with a hash '''<span class="red">#</span>'''. Also, comment after a key/value pair is allowed. All comments are skipped during processing.<br />
<br />
*It has an '''unlimited nested structure''' (lists, mappings, hierarchies). Indentation of whitespace is used to denote structure.<br />
<br />
*It has an '''unrestricted schema indentation'''. However, some schema validators recommend or impose '''two whitespaces''' indentations.<br />
<br />
*A <span class="red">colon</span> follows a <span class="violet">key</span> to denote a mapping <span class="twilightBlue">value</span> like:<div class="box" style="margin-left: 30px;"><span class="violet">ocean_model</span>: <span class="twilightBlue">ROMS</span></div><br />
<br />
*It supports <span class="green">Anchors</span> and <span class="salmon">Aliases</span>.<div class="box" style="margin-left: 30px;"><span class="violet">ATM_component</span>: <span class="green">&ATM</span> <span class="twilightBlue">WRF</span><br /><br /><span class="violet">metadata</span>:<br /><br /> - <span class="violet">standard_name</span>: <span class="twilightBlue">surface_eastward_wind</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface eastward wind</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">Uwind</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">uwind</span>, <span class="twilightBlue">time</span>]</span><br /> <span class="violet">source_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">destination_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">source_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">destination_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">add_offset</span>: <span class="twilightBlue">0.0d0</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug_write</span>: <span class="twilightBlue">false</span><br /> <span class="violet">connected_to</span>: <span class="salmon">*ATM</span> <span class="twilightBlue"># u10</span><br /> <span class="violet">regrid_method</span>: <span class="twilightBlue">bilinear</span><br /> <span class="violet">extrapolate_method</span>: <span class="twilightBlue">none</span></div><br />
<br />
*It supports '''blocking lists''': members are denoted by a leading <span class="red">hyphen-and-space</span>, which is considered part of the indentation. <br />
<br />
*It supports a '''flow sequence''': a vector list with values enclosed in <span class="red">square brackets</span> and separated by a <span class="red">comma-and-space</span>, like a <span class="violet">keyword</span>: <span class="red">[</span><span class="twilightBlue">val1</span><span class="red">,</span> ...<span class="red">,</span> <span class="twilightBlue">valN</span><span class="red">]</span>. <br />
<br />
*The <span class="violet">keyword</span> <span class="twilightBlue">value(s)</span> is (are) processed and stored as strings but converted to a logical, integer, floating-point, or derived-type when appropriate during extraction. If particular derived-type values are needed, the caller can process such a structure outside the parser. <br />
<br />
*It removes '''unwanted control characters''' like tabs and separators ('''ASCII''' character code '''0-31'''). <br />
<br />
*It is restricted to the '''English''' uppercase and lowercase alphabet but can be expanded to other characters (see <span class="blue">yaml_ValueType</span> routine). <br />
<br />
*Multiple or continuation lines are supported. So, for example, we can have:<div class="box" style="margin-left: 30px;"><span class="violet">state variables</span>: <span class="red">[</span><span class="twilightBlue">sea_surface_height_anomaly</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_potential_temperature</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_practical_salinity</span><span class="red">]</span></div><br />
<br />
==Extraction==<br />
<br />
Several derived-type structures are declared to facilitate the extraction of similar data blocks compactly from the YAML dictionary ('''CLASS''' <span class="forestGreen">yaml_tree</span>) in a compact way.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Ivec</span> <span class="twilightBlue">! integer structure</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Ivec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Lvec</span> <span class="twilightBlue">! logical structure</span><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Lvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Rvec</span> <span class="twilightBlue">! real structure</span><br /> <span class="darkTurquoise">real</span> (<span class="red">kind_real</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Rvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! string structure</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="darkTurquoise">value</span> <span class="twilightBlue">! scalar value</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! recursive vector</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! values</span></div><br />
<br />
The derived-type structure below, extended/inherited from parent "yaml_Svec", extracts hierarchies of keys and associated values from the '''YAML''' dictionary object. The calling program specifies a key string that may be generated by aggregating nested keys with a period. Also, it can extract flow sequence string element values that are separated by commas.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span>, <span class="darkTurquoise">EXTENDS</span>(<span class="forestGreen">yaml_Svec</span>) :: <span class="forestGreen">yaml_extract</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_vector</span> <span class="twilightBlue">! true if loaded vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_extract</span></div><br />
<br />
These public structures can be used in applications to extract block list '''YAML''' constructs. The key may represent a '''sequence flow''' <span class="red">[</span>...<span class="red">]</span> with a vector of values. The values can be integers, logicals, reals, or strings. For example, suppose that the '''YAML''' file has the following '''blocking list''' entries:<br />
<br />
<div class="box"><span class="violet">import</span>:<br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_downward_heat_flux_in_sea_water</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface net heat flux</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">shflux</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">shf</span>, <span class="twilightBlue">shf_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">true</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_x_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface zonal wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">sustr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">taux</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_y_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface meridional wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">svstr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">tauy</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span></div><br />
<br />
Then, the code to extract the data from the '''YAML''' dictionary (<span class="red">self</span>) is straightforward (see above for how to create and initialize a '''YAML''' dictionary object):<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="darkTurquoise">ONLY</span> : <span class="blue">yaml_get</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">L(:)</span> <span class="twilightBlue">! logical data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">F(:)</span> <span class="twilightBlue">! floating point data</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">S1(:)</span>, <span class="red">S2(:)</span> <span class="twilightBlue">! string data</span><br /><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>short_name'</span>, <span class="red">S1</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>data_variables'</span>, <span class="red">S2</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>scale'</span>, <span class="red">F</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>, <span class="red">L</span>)</div><br />
<br />
Notice that the internal adopted syntax is to aggregate the hierarchy of <span class="violet">keys</span> separated by a <span class="red">dot</span> in the <span class="mediumOrchid">keystring</span> (''e.g.'', <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>) during the extraction call. The above extraction statement yields the following values from a single invocation of the overloaded function <span class="blue">yaml_get</span>:<br />
<br />
<div class="code"> <span class="red">S1(1)%value</span> = <span class="mediumOrchid">'shflux'</span> <span class="twilightBlue">! <span class="violet">'short_name'</span> single value</span><br /> <span class="red">S1(2)%value</span> = <span class="mediumOrchid">'sustr'</span><br /> <span class="red">S1(3)%value</span> = <span class="mediumOrchid">'svstr'</span><br /><br /> <span class="red">S2(1)%vector(1)%value</span> = <span class="mediumOrchid">'shf'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(1)%vector(2)%value</span> = <span class="mediumOrchid">'shf_time'</span><br /><br /> <span class="red">S2(2)%vector(1)%value</span> = <span class="mediumOrchid">'taux'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(2)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">S2(3)%vector(1)%value</span> = <span class="mediumOrchid">'tauy'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(3)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">F(1)</span> = <span class="mediumOrchid">1.0d0</span> <span class="twilightBlue">! <span class="violet">'scale'</span> single value</span><br /> <span class="red">F(2)</span> = <span class="mediumOrchid">1.0d0</span><br /> <span class="red">F(3)</span> = <span class="mediumOrchid">1.0d0</span><br /><br /> <span class="red">L(1)</span> = <span class="mediumOrchid">.TRUE.</span> <span class="twilightBlue">! <span class="violet">'debug write'</span> single value</span><br /> <span class="red">L(2)</span> = <span class="mediumOrchid">.FALSE.</span><br /> <span class="red">L(3)</span> = <span class="mediumOrchid">.FALSE.</span></div><br />
<br />
Check <span class="blue">ROMS/Utility/get_metdata.F</span> for more details on how the metadata is processed from input '''YAML''' files.</div>Arangohttps://www.myroms.org/wiki/index.php?title=NUOPC_Cap&diff=6538NUOPC Cap2022-08-30T19:47:03Z<p>Arango: /* Implementation */</p>
<hr />
<div><div class="title">ROMS Native ESMF/NUOPC Cap</div><br />
__TOC__<br />
<br />
==Overview==<br />
This document describes the ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span> module <span class="blue">Master/esmf_roms.F</span> used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (<span class="blue">Master/esmf_driver.h</span>), which '''creates''', '''register''', '''initialize''', '''run''', and '''finalize''' the coupling of each Earth System Model ('''ESM''') component ('''data''', '''atmosphere''', '''sea-ice''', '''wave''', and '''ocean'''). It also sets the coupler '''connectors''' and the '''RunSequence''' policy (<span class="blue">Master/esmf_esm.F</span>). The framework provides the '''NUOPC''' <span class="red">cap</span> modules for each coupled component. Detailed information about '''ESMF/NUOPC''' and how it is implemented in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.<br />
<br />
A '''NUOPC''' <span class="red">cap</span> is a Fortran module that serves as the interface to a model when used in a '''NUOPC'''-based coupled system. The term <span class="red">cap</span> is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its '''initialize''', '''run''', and '''finalize''' phases and exposing the model data structures to the field exchange ('''import''' and '''export''' states) between coupled components.<br />
<br />
<br />
[[Image:ROMS_Coupling.png|800px|center]]<br />
<br />
<center>'''''Figure 1:''''' ROMS native ESMF/NUOPC coupling infrastructure</center><br />
<br />
==Implementation==<br />
<br />
The <span class="blue">Master/mod_esmf_esm.F</span> module declares several derived-type structures to facilitate the management of all internal objects and variables in each '''ESM''' coupled component:<br />
<br />
* '''ESM''' coupling time managing variables and '''ESMF''' objects. The <span class="red">ClockInfo</span> object is allocated as <span class="red">(0:Nmodels)</span>, where <span class="red">Nmodels</span> is the number of coupled '''ESM''' components. Here, index zero corresponds to the '''driver'''.<div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Clock</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Restarted</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i8b</span>) :: <span class="red">AdvanceCount</span> <span class="twilightBlue">! advance counter</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Current_Time</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Reference</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Restart</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Start</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Stop</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Step</span> <span class="twilightBlue">! seconds</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">CalendarString</span> <span class="twilightBlue">! 360_day, gregorian</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_ReferenceString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_RestartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StopString</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Calendar</span>) :: <span class="red">Calendar</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Clock</span>) :: <span class="red">Clock</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Direction_flag</span>) :: <span class="red">Direction</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">CurrentTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">ReferenceTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">RestartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StopTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_TimeInterval</span>) :: <span class="red">TimeStep</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Clock</span><br /><br />'''. . .'''<br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''ESM''' coupled state sets. If appropriate, it includes the logic for connecting nested grids. The <span class="red">COUPLED</span> object is of size <span class="red">Nmodels</span>.<div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">LinkedGrid(:,:)</span> <span class="twilightBlue">! connected grid</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">DataCoupledSets(:,:)</span> <span class="twilightBlue">! DATA linked sets</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">SetLabel(:)</span> <span class="twilightBlue">! set label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExpLabel(:)</span> <span class="twilightBlue">! export label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImpLabel(:)</span> ! <span class="twilightBlue">import label</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:,:)</span> <span class="twilightBlue">! export set</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_CplSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">COUPLED(:)</span></div><br />
<br />
* '''DATA''' model field processing information. The '''DATA''' component is often necessary for applications with incongruent grids to provide complete field melding capabilities or methods for reading and interpolating stream data in space and time from archived NetCDF files.<div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Data</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcycle</span> <span class="twilightBlue">! cycling time coordinate</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcoord</span> <span class="twilightBlue">! coordinates attribute</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmask</span> <span class="twilightBlue">! land/sea mask</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmulti</span> <span class="twilightBlue">! field across multi-files</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">LastRec</span> <span class="twilightBlue">! processed last record</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Icomp</span> <span class="twilightBlue">! target component index</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ncid</span> <span class="twilightBlue">! NetCDF file ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tid</span> <span class="twilightBlue">! time variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Vid</span> <span class="twilightBlue">! field variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nvdim</span> <span class="twilightBlue">! number spatial dimensions</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Zlevel</span> <span class="twilightBlue">! level index to process</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nrec</span> <span class="twilightBlue">! number of time records</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Trec</span> <span class="twilightBlue">! latest read time record</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">LandValue</span> <span class="twilightBlue">! Masking land value</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">SeaValue</span> <span class="twilightBlue">! Masking sea value</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Vsize(:)</span> <span class="twilightBlue">! dimensions size</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! add_offset attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">FillValue</span> <span class="twilightBlue">! _FillValue attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! scale_factor attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Clength</span> <span class="twilightBlue">! time cycling length</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMin</span> <span class="twilightBlue">! grid minimum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMax</span> <span class="twilightBlue">! grid maximum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMin</span> <span class="twilightBlue">! grid minimum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMax</span> <span class="twilightBlue">! grid maximum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tscale</span> <span class="twilightBlue">! time scale to day</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmono</span> <span class="twilightBlue">! monotonic time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! time minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! time maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! lower time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! upper time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmin</span> <span class="twilightBlue">! variable minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmax</span> <span class="twilightBlue">! variable maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Date(6,2)</span> <span class="twilightBlue">! time-snapshots dates YYYY,MM,DD hh:mm:ss.ss</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30), <span class="darkTurquoise">allocatable</span> :: <span class="red">Dname(:)</span> <span class="twilightBlue">! variable dimensions names</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Vcoord(:)</span> <span class="twilightBlue">! variable coordinates names</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=22 ) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30 ) :: <span class="red">SpecialAction</span> <span class="twilightBlue">! special processing</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tname</span> <span class="twilightBlue">! time variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tunits</span> <span class="twilightBlue">! time variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vname</span> <span class="twilightBlue">! variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vunits</span> <span class="twilightBlue">! variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vdescriptor</span> <span class="twilightBlue">! Variable descriptive name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vlongname</span> <span class="twilightBlue">! long_name attribute</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! field longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! field latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! field land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2d(:,:)</span> <span class="twilightBlue">! time interpolated 2D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3d(:,:,:)</span> <span class="twilightBlue">! time interpolated 3D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2dG(:,:,:)</span> <span class="twilightBlue">! latest 2D data snapsnots</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3dG(:,:,:,:)</span><span class="twilightBlue">! latest 3D data snapsnots</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Field</span>) :: <span class="red">field</span> <span class="twilightBlue">! field object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>) :: <span class="red">grid</span> <span class="twilightBlue">! field grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> <span class="twilightBlue">! field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Data</span></div><br />
<br />
* '''DATA''' model high-level structure. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_DataSet</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfields</span> <span class="twilightBlue">! number of fields</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfiles</span> <span class="twilightBlue">! number of input files</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Ctarget(:)</span> <span class="twilightBlue">! component target</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Field(:)</span> <span class="twilightBlue">! field short-name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Data</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Export(:)</span> <span class="twilightBlue">! Export field structure</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">T_IO</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">IFS(:)</span> <span class="twilightBlue">! Input Files structure</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_DataSet</span></div><br />
<br />
* Import and export field metadata information between the source (<span class="twilightBlue">SRC</span>) and destination (<span class="twilightBlue">DST</span>) components. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Field</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">connected</span> <span class="twilightBlue">! connected to coupler</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">debug_write</span> <span class="twilightBlue">! write exchanged field</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">enable_integral_adj</span> <span class="twilightBlue">! area integral adjusted</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">fid</span> <span class="twilightBlue">! internal field ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! field grid mesh type</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">etype</span> <span class="twilightBlue">! field extrapolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">itype</span> <span class="twilightBlue">! field interpolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=20) :: <span class="red">Ctarget</span> <span class="twilightBlue">! component destination</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">short_name</span> <span class="twilightBlue">! short name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">standard_name</span> <span class="twilightBlue">! standard name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">long_name</span> <span class="twilightBlue">! long name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_gtype</span> <span class="twilightBlue">! DST grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_units</span> <span class="twilightBlue">! DST units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_gtype</span> <span class="twilightBlue">! SRC grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_units</span> <span class="twilightBlue">! SRC units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_vname</span> <span class="twilightBlue">! DATA Vname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_tname</span> <span class="twilightBlue">! DATA Tname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">RegridMethod</span> <span class="twilightBlue">! regrid method</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExtrapMethod</span> <span class="twilightBlue">! extrapolate</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! field scale factor</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! field add offset value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! DATA time minimum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! DATA time maximum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! DATA lower time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! DATA upper time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (day)</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> ! <span class="twilightBlue">field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Field</span></div><br />
<br />
* Import and export fields mesh data. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Mesh</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gid</span> <span class="twilightBlue">! grid ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! grid mesh type</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! grid land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! grid longitude</span><br /> <span class="red">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! grid latitude</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">area(:,:)</span> <span class="twilightBlue">! grid area</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Mesh</span></div><br />
<br />
* Melding coefficients used to combine fields from '''DATA''' and '''ESM''' components. The weight factors are read from the input NetCDF specified in the <span class="blue">WeightsFile(atmos)</span> keyword. The user has full control of how the merging is done. It is recommended to provide a gradual transition between the two components. <br /><br /> Recall that the '''DATA''' component supplies needed data to a particular '''ESM''' component. For example, it may export data to the atmosphere model at locations not covered by the other '''ESM''' components because of smaller grid coverage. If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data: <div class="code"> <span class="red">SST_atm(:,:)</span> = <span class="red">Cesm(:,:)</span> * <span class="red">SST_esm(:,:)</span> + <span class="red">Cdat(:,:)</span> * <span class="red">SST_dat(:,:)</span></div> where<span class="red"> Cesm(:,:)</span> + <span class="red">Cdat(:,:)</span> = <span class="red">1</span>.<br /><div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">NestedGrid</span> <span class="twilightBlue">! grid needing merged field</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameDATA</span> <span class="twilightBlue">! DATA weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameESM</span> <span class="twilightBlue">! ESM weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! Weights NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="forestGreen">allocatable</span> :: <span class="red">Cdat(:,:)</span> <span class="twilightBlue">! coefficients for DATA</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cesm(:,:)</span> <span class="twilightBlue">! coefficients for ESM</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Meld</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">WEIGHTS(:)</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightDAT</span> = <span class="red">0.0_dp</span> <span class="twilightBlue">! DATA component weight</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightESM</span> = <span class="red">1.0_dp</span> <span class="twilightBlue">! ESM component weight</span></div><br />
<br />
* Coupled models high-level object, [<span class="red">Nmodels=5</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active for coupling</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">LandValue</span> <span class="twilightBlue">! land mask value</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">SeaValue</span> <span class="twilightBlue">! sea mask value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Ngrids</span> <span class="twilightBlue">! number nested grids</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ExportCalls</span> <span class="twilightBlue">! export CALL counter</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ImportCalls</span> <span class="twilightBlue">! import CALL counter</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number model PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! component PETs list</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">TimeFrac(:,:)</span> <span class="twilightBlue">! driver time fraction</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">name</span> <span class="twilightBlue">! component name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">grid(:)</span> <span class="twilightBlue">! grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Mesh</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mesh(:)</span> <span class="twilightBlue">! mesh</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportField(:)</span> <span class="twilightBlue">! import fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportField(:)</span> <span class="twilightBlue">! export fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportState(:)</span> <span class="twilightBlue">! import state</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:)</span> <span class="twilightBlue">! export state</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Model</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">MODELS(:)</span></div><br />
<br />
* Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [<span class="red">Nmodels</span>, <span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active connector</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">divDT</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">MaskInteraction</span> <span class="twilightBlue">! connector mask interaction</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number of connector PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! connector PETs list</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100) :: <span class="red">name</span> <span class="twilightBlue">! connector name</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Conn</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">CONNECTORS(:,:)</span></div><br />
<br />
* Define '''DATA''' Model field processing information from input data files, [<span class="red">1</span>:<span class="red">Nmodels</span>]. Currently, the '''DATA''' Model only export fields and it only supports input NetCDF files. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_DataSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">DataSet(:)</span></div><br />
<br />
* '''ESM''' clock for driver (zeroth element) and coupled components, [<span class="red">0</span>:<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''Internal module parameters and variables:'''<br /> <br />
** Number of coupled '''ESM''' gridded components. Currently, five '''ESM''' components are supported ('''ROMS''', '''DATA''', '''ATMOSPHERE''', '''SEA-ICE''', and '''WAVE''' model types). <br /> All supported components are accounted here even if we are running an application with less number of models. The <span class="red">IsActive</span> switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Nmodels</span> = <span class="red">5</span></div><br />
** Number of nested grids. All are initialized to just one grid; its values are overwritten during processing. An additional variable '''NgridsR''' is created to avoid using ROMS module <span class="blue">mod_param</span> in the generic interface. Both '''Ngrids''' and '''NgridsR''' will have the same value.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsA</span> = <span class="red">1</span> <span class="twilightBlue">! Atmosphere Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsD</span> = <span class="red">1</span> <span class="twilightBlue">! DATA Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsI</span> = <span class="red">1</span> <span class="twilightBlue">! Sea-ice Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsR</span> = <span class="red">1</span> <span class="twilightBlue">! ROMS</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsW</span> = <span class="red">1</span> <span class="twilightBlue">! Wave Model</span></div><br />
** Coupled '''ESM''' components identification indices.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idriver</span> = <span class="red">0</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iroms</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iatmos</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idata</span> = <span class="red">3</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseaice</span> = <span class="red">4</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iwave</span> = <span class="red">5</span></div><br />
** Generic '''ESM''' component labels used in the <span class="darkTurquoise">CASE</span> constructs. We cannot use the identification indices because the vector <span class="red">Iroms(:)</span> cannot be defined as a parameter and a non constant expression is illegal: <span class="darkTurquoise">CASE</span> ( <span class="red">Iroms</span>(<span class="red">1</span>) : <span class="red">Iroms</span>(<span class="red">NgridsR</span>) )<div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 3), <span class="darkTurquoise">allocatable</span> :: <span class="red">Clabel(:)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len=</span>10), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cmodel(:)</span></div><br />
** Number of ROMS export and import fields per component. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nexport(:)</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nimport(:)</span></div><br />
** Model coupling type: [<span class="red">1</span>] Explicit, [<span class="red">otherwise</span>] Semi-Implicit. In <span class="red">explicit coupling</span>, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In <span class="red">implicit coupling</span>, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In <span class="red">semi-implicit coupling</span>, '''ROMS -> ATM''' is explicit, '''ATM -> ROMS''' is implicit. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">CouplingType</span> = <span class="red">1</span></div><br />
** Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent). <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">ESMcomm(:)</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">sumPETs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">PETrank</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=10) :: <span class="red">PETlayoutOption</span></div><br />
** Driver clock parameters specified in configuration script. A integer vector with six elements: (<span class="red">1</span>) year including century, like 2017; (<span class="red">2</span>) month of the year, 1 to 12; (<span class="red">3</span>) day of the month; (<span class="red">4</span>) hour of the day, 0 to 23; (<span class="red">5</span>) minutes of the hour, 0 to 59; and (<span class="red">6</span>) seconds of the minute, 0 to 59 <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ReferenceDate</span>(6) <span class="twilightBlue">! reference date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">RestartDate</span>(6) <span class="twilightBlue">! restarting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StartDate</span>(6) <span class="twilightBlue">! starting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StopDate</span>(6) <span class="twilightBlue">! stopping date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">TimeStep</span>(6) <span class="twilightBlue">! coupling interval</span></div><br />
** Today date string. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=44) :: <span class="red">TodayDateString</span></div><br />
** '''ESM''' coupling simulation reference date number, element: (<span class="red">1</span>) seconds, and (<span class="red">2</span>) fractional days<div class="code"> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">ReferenceDateNumber</span>(2)</div><br />
** '''DATA''' component parallel distributed-memory domain partions in the '''I'''- and '''J'''-directions (lon,lat). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ItileD</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">JtileD</span></div><br />
** Coupling debugging flag: [<span class="red">0</span>] no debugging, [<span class="red">1</span>] reports informative messages, or [<span class="red">2</span>] <span class="red">'1'</span> plus writes grid information in '''VTK''' format, [<span class="red">3</span>] <span class="red">'2'</span> plus writes exchage fields into '''NetCDF''' files. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">DebugLevel</span> = <span class="red">0</span></div><br />
** Execution tracing level flag: [<span class="red">0</span>] no tracing, [<span class="red">1</span>] reports sequence of coupling subroutine calls, or [<span class="red">2</span>] <<span class="red">1</span>> plus writes voluminous '''ESMF''' library tracing information which slowdown performace, and creates large log file. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">TraceLevel</span> = <span class="red">0</span></div><br />
** Switch to trace/track run sequence during debugging. All information is written to Fortan unit <span class="red">trac</span>. For now, use standard output unit. <div class="code"> <span class="darkTurquoise">logical</span> :: <span class="red">ESM_track</span> = <span class="red">.FALSE.</span> <span class="twilightBlue">! trace/track CALL sequence switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">trac</span> = <span class="red">6</span> <span class="twilightBlue">! trace/track CALL sequence unit</span></div><br />
** Coupled model staggered grid-cell type indices. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inan</span> = <span class="red">0</span> <span class="twilightBlue">! unstaggered, cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icenter</span> = <span class="red">1</span> <span class="twilightBlue">! cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icorner</span> = <span class="red">2</span> <span class="twilightBlue">! cell corners</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iupoint</span> = <span class="red">3</span> <span class="twilightBlue">! right and left cell faces</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ivpoint</span> = <span class="red">4</span> <span class="twilightBlue">! upper and lower cell faces</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=6), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">GridType</span> = &<br /> & (/ '<span class="red">N/A</span> ', &<br /> & '<span class="red">Center</span>', &<br /> & '<span class="red">Corner</span>', &<br /> & '<span class="red">U</span> ', &<br /> & '<span class="red">V</span> ' /)</div><br />
** '''REGRID''' interpolation method between source and destination fields. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ibilin</span> = <span class="red">1</span> <span class="twilightBlue">! bilinear</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipatch</span> = <span class="red">2</span> <span class="twilightBlue">! high-order patch recovery</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv1</span> = <span class="red">3</span> <span class="twilightBlue">! first-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv2</span> = <span class="red">4</span> <span class="twilightBlue">! second-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InStoD</span> = <span class="red">5</span> <span class="twilightBlue">! nearest neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InDtoS</span> = <span class="red">6</span> <span class="twilightBlue">! nearest neighbor Dst to Src</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:6) :: <span class="red">IntrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">BLIN</span>', &<br /> & '<span class="red">PTCH</span>', &<br /> & '<span class="red">CNS1</span>', &<br /> & '<span class="red">CNS2</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">ND2S</span>' /)</div><br />
** Extrapolation method for unmapped destination points. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Enone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">ExStoD</span> = <span class="red">1</span> <span class="twilightBlue">! nearear neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Eidavg</span> = <span class="red">2</span> <span class="twilightBlue">! inverse distance average</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ecreep</span> = <span class="red">3</span> <span class="twilightBlue">! creep fill</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">E2steps</span> = <span class="red">4</span> <span class="twilightBlue">! Turuncoglu two steps</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">ExtrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">IDAV</span>', &<br /> & '<span class="red">CREE</span>', &<br /> & '<span class="red">2STP</span>' /)</div><br />
** The number of levels to output for the extrapolation methods that fill levels, like creep fill (<span class="red">ESMF_EXTRAPMETHOD_CREEP</span>). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels. For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding ('''ESMF''' Reference Manual, v '''8.0.0'''). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">extrapNumLevels</span> = <span class="red">1</span></div><br />
** Interpolation connectors mask interaction flags. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverLand</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverOcean</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverAll</span> = <span class="red">3</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=3), <span class="darkTurquoise">dimension</span>(3) :: <span class="red">MaskType</span> = &<br /> & (/ '<span class="red">LND</span>', &<br /> & '<span class="red">OCN</span>', &<br /> & '<span class="red">ALL</span>'/)</div><br />
** Coupling run mode: sequential or concurrent. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseq</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipar</span> = <span class="red">2</span><br /><br /> <span class="darkTurquoise">character</span> (len=10), <span class="darkTurquoise">dimension</span>(2) :: <span class="red">RunMode</span> = &<br /> & (/ '<span class="red">SEQUENTIA</span>L', &<br /> & '<span class="red">CONCURRENT</span>' /)</div><br />
** Compling standard input parameters filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CinpName</span></div><br />
** '''ESM''' free-format run sequence configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CONFname</span></div><br />
** ROMS coupling '''YAML''' configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">CPLname</span></div><br />
** Standard input filename for each coupled model, [<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">INPname(:)</span></div><br />
** Standard output units and log messages filename for coupler and '''ESMF''' library. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">cplout</span> = <span class="red">77</span> <span class="twilightBlue">! coupling driver</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">dataout</span> = <span class="red">77</span> <span class="twilightBlue">! data component</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">ESMnameLog</span> = '<span class="red">log.esmf</span>'<br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">CouplerLog</span> = '<span class="red">log.coupler</span>'</div><br />
** Output '''NetCDF''' file used to store field snapshot attributes needed for time interpolation by the '''ESM''' component kernel during concurrent coupling. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=17), <span class="darkTurquoise">parameter</span> :: <span class="red">AttFileName</span> = '<span class="red">time_intrp_att.nc</span>'</div><br />
** '''ESM''' single and double precision constants. <div class="code"> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MAPPED_MASK</span> = <span class="red">99_i4b</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">UNMAPPED_MASK</span> = <span class="red">98_i4b</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_dp</span> = <span class="red">1.0E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r4</span> = <span class="red">1.0E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r8</span> = <span class="red">1.0E20_r8</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_dp</span> = <span class="red">0.5E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r4</span> = <span class="red">0.5E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r8</span> = <span class="red">0.5E20_r8</span></div><br />
<br />
==Capabilities==<br />
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by '''NUOPC'''. According to a predefined calling sequence, these subroutines are called by the '''NUOPC''' infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.<br />
<br />
The initialization sequence is the most complex and is governed by the '''NUOPC''' technical rules. Details about the initialization sequence can be found in the '''NUOPC''' Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" width="1000"<br />
|valign="top"|<span class="blue">ROMS_SetServices</span> <br />
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component chared-object entry points for using '''NUOPC''' generic methods for '''initialize''', '''run''', and '''finalize'''.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span><br />
|ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP2</span><br />
|ROMS component phase 2 initialization which initializes the ROMS component (<span class="blue">ROMS_initialize</span>), sets component grid (<span class="blue">ROMS_SetGridArrays</span>), and adds fields into import and export into respective states.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_DataInit</span><br />
|Exports ROMS component fields during initialization or restart.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetClock</span><br />
|Sets ROMS component date calendar, start and stop times, and coupling interval.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetRunClock</span><br />
|Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_CheckImport</span><br />
|Checks if ROMS component import field is at the correct time.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetGridArrays</span><br />
|Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetStates</span><br />
|Adds ROMS component <span class="red">export</span> and <span class="red">import</span> fields into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_ModelAdvance</span><br />
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_Import</span> and <span class="blue">ROMS_Export</span> routines.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetFinalize</span><br />
|Finalizes ROMS component execution.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Import</span><br />
|Loads import fields into ROMS internal kernel arrays.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Export</span><br />
|Exports ROMS fields to other gridded components.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Rotate</span><br />
|Rotates vector components from computational grid to geographical '''EAST''' and '''NORTH''' directions or vice versa.<br />
|-<br />
|}<br />
<br />
<br />
Other <span class="red">NUOPC</span> cap modules are available within the ROMS native framework:<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | '''ESMF/NUOPC Modules'''<br />
| align="center" | '''Tested'''<br />
|style="width: 700px;" align="center" | '''Description'''<br />
|-<br />
| <span class="red">Master/esmf_driver.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''ESMF/NUOPC''' Master Driver<br />
|-<br />
| <span class="red">Master/esmf_esm.F</span><br />
| <span class="forestGreen">yes</span><br />
| Sets '''ESMF/NUOPC''' '''Services''' and '''RunSequence''' for each coupled component<br />
|-<br />
| <span class="red">Master/esmf_coupler.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''Computes''', '''Execute''', and '''Release''' the '''Connectors''' between <span class="violet">source</span> and <span class="violet">destination</span> fields<br />
|-<br />
| <span class="red">Master/mod_esmf_esm.h</span><br />
| <span class="forestGreen">yes</span><br />
| Defines framework object '''structures''' and '''variables''' and includes support routines<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_regcm.h</span><br />
| <span class="red">no</span><br />
| ICTP Regional Climate Model ('''RegCM''', Version '''4.5''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_wrf.h</span><br />
| <span class="forestGreen">yes</span><br />
| Weather Research and Forecasting model ('''WRF''')<br />
|-<br />
| <span class="blue">Master/esmf_data.F</span><br />
| <span class="forestGreen">yes</span><br />
| Generic '''DATA''' component via '''NetCDF''' files<br />
|-<br />
| <span class="blue">Master/esmf_ice_cice.h</span><br />
| <span class="red">no</span><br />
| Los Alamos Sea Ice Model ('''CICE''')<br />
|-<br />
| <span class="blue">Master/esmf_roms.F</span><br />
| <span class="forestGreen">yes</span><br />
| Regional Ocean Modeling System ('''ROMS''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_wam.h</span><br />
| <span class="red">no</span><br />
| ECMWF's Wave Model ('''WAM''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_ww3.h</span><br />
| <span class="red">no</span><br />
| WaveWatch III ('''WW3'''), <span class="twilightBlue">under development</span><br />
|}<br />
<div style="clear: both"></div><br />
<br />
==Import and Export Fields==<br />
<br />
The <span class="red">export</span> and <span class="red">import</span> fields are configured and specified in the '''YAML''' file [[coupling_esmf_atm.yaml]]. Check the following [[coupling_esmf_atm.yaml|wikiROMS]] page for detailed information. The user has full control of the coupling exchange fields metadata.<br />
<br />
The '''ESMF/NUOPC''' library version '''8.0.0''' or higher supports native nesting capabilities. The '''NUOPC''' layer now allows coupling sets in the <span class="red">import</span> and <span class="red">export</span> states of the '''ESM''' components. A nested grid is represented as another '''ESMF_Grid''' object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The '''NUOPC''' connector can recognize the different grids and exchange fields to a connected '''ESM''' component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The <span class="blue">NUOPC_AddNestedState</span> routine is used to advertise the nested couple sets.<br />
<br />
For example, the ROMS '''NOUPC''' <span class="red">cap</span> module routine <span class="blue">ROMS_SetInitializeP1</span> illustrates how easily the coupled sets is added to the <span class="red">ImportState</span> and <span class="red">ExportState</span> as shown below in the green blocks of code. <br />
<br />
<br />
{|cellpadding="3"<br />
|-<br />
|[[Image:ImportCoupledStates.png]]<br />
|[[Image:ExportCoupledStates.png]]<br />
|}</div>Arangohttps://www.myroms.org/wiki/index.php?title=YAML_Parser&diff=6537YAML Parser2022-08-30T19:00:19Z<p>Arango: /* Introduction */</p>
<hr />
<div><div class="title">YAML Parser</div><br />
Starting with '''svn''' revision '''-r 1902''' released on March 1, 2022, the ROMS metadata is managed with a [[varinfo.yaml|YAML file]], and the regular text file '''varinfo.dat''' is deprecated. The '''YAML''' files are simple, easy to follow, elegant, portable, and expandable. ROMS can now process YAML files with its parser module, '''[https://www.myroms.org/links/yaml_parser.F yaml_parser.F]'''. Therefore, there is no need to use third-party '''YAML''' parsers.<br />
<br />
The ROMS '''YAML''' parser source code can be found in <span class="forestGreen">ROMS/Utility</span>. It is written in Fortran 2003 and includes a '''CLASS''' of type '''yaml_tree''' for parsing input '''YAML''' files.<br />
<br />
==Introduction==<br />
Although several '''YAML''' parsers for Fortran exist, a more straightforward and uncomplicated parser with substantial capabilities was coded in '''ROMS''', '''[https://www.myroms.org/links/yaml_parser.F yaml_parser.F]'''. It is a hybrid between standard and Object-Oriented Programming (OOP) principles but without the need for recurrency, polymorphism, and containers (another library).<br />
<br />
The only constraint in the '''ROMS''' parser is that the '''YAML''' file is read '''twice''' for simplicity and to avoid containers. The container is a Fortran vector! The first read determines the indentation policy and the length of the collection vector, <span class="red">list(:)</span> pairs object (derived-type structure <span class="forestGreen">yaml_pair</span>). The first reading is quick.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_pair</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_alias</span> <span class="twilightBlue">! alias <span class="violet">*</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_anchor</span> <span class="twilightBlue">! anchor <span class="violet">&</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_block</span> <span class="twilightBlue">! block <span class="violet">-</span> list</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_sequence</span> <span class="twilightBlue">! sequence <span class="violet">[]</span> tokens</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_logical</span> <span class="twilightBlue">! logical value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_integer</span> <span class="twilightBlue">! integer value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_real</span> <span class="twilightBlue">! floating-point value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_string</span> <span class="twilightBlue">! string value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">id</span> <span class="twilightBlue">! key/value ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">parent_id</span> <span class="twilightBlue">! parent ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">left_padding</span> <span class="twilightBlue">! indent level: 0,1,..</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">line</span> <span class="twilightBlue">! YAML line</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">key</span> <span class="twilightBlue">! YAML keyword</span><span class="violet">:</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">value</span> <span class="twilightBlue">! YAML value(s)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">anchor</span> <span class="twilightBlue">! anchor keyword</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_pair</span></div><br />
The '''YAML''' file dictionary <span class="darkTurquoise">CLASS</span> <span class="forestGreen">yaml_tree</span> is defined as:<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_tree</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsCreated</span> '''= .FALSE.''' <span class="twilightBlue">! YAML object creation switch</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nbranches</span> <span class="twilightBlue">! total number of branches</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Npairs</span> <span class="twilightBlue">! total number of pairs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">indent</span> <span class="twilightBlue">! blank indentation policy</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">filename</span> <span class="twilightBlue">! YAML file name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_pair</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">list(:)</span> <span class="twilightBlue">! collection pairs</span><br /><br /> <span class="darkTurquoise">CONTAINS</span> <span class="twilightBlue">! CLASS objects</span><br /><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">create</span> => <span class="blue">yaml_tree_create</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">destroy</span> => <span class="blue">yaml_tree_destroy</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">dump</span> => <span class="blue">yaml_tree_dump</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">extract</span> => <span class="blue">yaml_tree_extract</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill</span> => <span class="blue">yaml_tree_fill</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill_aliases</span> => <span class="blue">yaml_tree_fill_aliases</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">has</span> => <span class="blue">yaml_tree_has</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">read_line</span> => <span class="blue">yaml_tree_read_line</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_tree</span></div><br />
The <span class="forestGreen">yaml_tree</span> object stores all the data contained in a specific '''YAML''' file. For Example, in '''ROMS''' the input '''YAML''' metadata dictionary is created and initialized as follows:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="blue">ONLY</span> : <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lreport</span> <span class="twilightBlue">! verbose report switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_tree</span>) :: <span class="red">self</span> <span class="twilightBlue">! declare a dummy YAML object</span><br /><br /> <span class="darkTurquoise">IF</span> (.<span class="darkTurquoise">not</span>.<span class="red">self%IsCreated</span>) <span class="darkTurquoise">THEN</span> <span class="twilightBlue">! process input YAML file</span><br /> <span class="red">Lreport</span> = <span class="red">.TRUE.</span><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_initialize</span> (<span class="red">self</span>, <span class="red">'varinfo.yaml'</span>, <span class="red">Lreport</span>)<br /> <span class="darkTurquoise">END IF</span></div><br />
The error management is omitted for clarity. Then, the needed data is extracted from the <span class="red">self</span> object and loaded into the internal '''ROMS''' variables using the overloaded <span class="mediumOrchid">yaml_get</span> API:<br />
<br />
<div class="code"> <span class="darkTurquoise">INTERFACE</span> <span class="mediumOrchid">yaml_get</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_i_struc</span> <span class="twilightBlue">! Gets integer structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_l_struc</span> <span class="twilightBlue">! Gets logical structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_r_struc</span> <span class="twilightBlue">! Gets real structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_s_struc</span> <span class="twilightBlue">! Gets string structure</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_0d</span> <span class="twilightBlue">! Gets integer value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_1d</span> <span class="twilightBlue">! Gest integer values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_0d</span> <span class="twilightBlue">! Gets logical value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_1d</span> <span class="twilightBlue">! Gets logical values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_0d</span> <span class="twilightBlue">! Gets real value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_1d</span> <span class="twilightBlue">! Gets real values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_0d</span> <span class="twilightBlue">! Gets string value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_1d</span> <span class="twilightBlue">! Gets string values</span><br /><br /> <span class="darkTurquoise">END INTERFACE</span> <span class="mediumOrchid">yaml_get</span></div><br />
Additionally, the '''YAML''' parser module has the following public and private routines/functions used during processing:<br />
<br />
<div class="code"> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_AssignString</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_Error</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_get</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_CountKeys</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_LowerCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_UpperCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_ValueType</span></div><br />
<br />
The parser module is self-contained with minimal dependencies on other '''ROMS''' modules:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">mod_kinds</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">kind_real</span> => <span class="red">dp</span> <span class="twilightBlue">! double-precision</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_parallel</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_Master</span> => <span class="red">Master</span> <span class="twilightBlue">! master PET</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_scalars</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_ErrFlag</span> => <span class="red">exit_flag</span> <span class="twilightBlue">! error flag</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_iounits</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_stdout</span> => <span class="red">stdout</span> <span class="twilightBlue">! standard ouput</span></div><br />
<br />
Overall, the parser is very fast and works in parallel. All PETs are involved in their dictionary copy to avoid overhead from collective MPI calls.<br />
<br />
==Capabilities==<br />
Currently, the '''YAML''' parser supports the following features:<br />
<br />
*'''Single''' or '''multiple''' line comments start with a hash '''<span class="red">#</span>'''. Also, comment after a key/value pair is allowed. All comments are skipped during processing.<br />
<br />
*It has an '''unlimited nested structure''' (lists, mappings, hierarchies). Indentation of whitespace is used to denote structure.<br />
<br />
*It has an '''unrestricted schema indentation'''. However, some schema validators recommend or impose '''two whitespaces''' indentations.<br />
<br />
*A <span class="red">colon</span> follows a <span class="violet">key</span> to denote a mapping <span class="twilightBlue">value</span> like:<div class="box" style="margin-left: 30px;"><span class="violet">ocean_model</span>: <span class="twilightBlue">ROMS</span></div><br />
<br />
*It supports <span class="green">Anchors</span> and <span class="salmon">Aliases</span>.<div class="box" style="margin-left: 30px;"><span class="violet">ATM_component</span>: <span class="green">&ATM</span> <span class="twilightBlue">WRF</span><br /><br /><span class="violet">metadata</span>:<br /><br /> - <span class="violet">standard_name</span>: <span class="twilightBlue">surface_eastward_wind</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface eastward wind</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">Uwind</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">uwind</span>, <span class="twilightBlue">time</span>]</span><br /> <span class="violet">source_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">destination_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">source_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">destination_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">add_offset</span>: <span class="twilightBlue">0.0d0</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug_write</span>: <span class="twilightBlue">false</span><br /> <span class="violet">connected_to</span>: <span class="salmon">*ATM</span> <span class="twilightBlue"># u10</span><br /> <span class="violet">regrid_method</span>: <span class="twilightBlue">bilinear</span><br /> <span class="violet">extrapolate_method</span>: <span class="twilightBlue">none</span></div><br />
<br />
*It supports '''blocking lists''': members are denoted by a leading <span class="red">hyphen-and-space</span>, which is considered part of the indentation. <br />
<br />
*It supports a '''flow sequence''': a vector list with values enclosed in <span class="red">square brackets</span> and separated by a <span class="red">comma-and-space</span>, like a <span class="violet">keyword</span>: <span class="red">[</span><span class="twilightBlue">val1</span><span class="red">,</span> ...<span class="red">,</span> <span class="twilightBlue">valN</span><span class="red">]</span>. <br />
<br />
*The <span class="violet">keyword</span> <span class="twilightBlue">value(s)</span> is (are) processed and stored as strings but converted to a logical, integer, floating-point, or derived-type when appropriate during extraction. If particular derived-type values are needed, the caller can process such a structure outside the parser. <br />
<br />
*It removes '''unwanted control characters''' like tabs and separators ('''ASCII''' character code '''0-31'''). <br />
<br />
*It is restricted to the '''English''' uppercase and lowercase alphabet but can be expanded to other characters (see <span class="blue">yaml_ValueType</span> routine). <br />
<br />
*Multiple or continuation lines are supported. So, for example, we can have:<div class="box" style="margin-left: 30px;"><span class="violet">state variables</span>: <span class="red">[</span><span class="twilightBlue">sea_surface_height_anomaly</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_potential_temperature</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_practical_salinity</span><span class="red">]</span></div><br />
<br />
==Extraction==<br />
<br />
Several derived-type structures are declared to facilitate the extraction of similar data blocks compactly from the YAML dictionary ('''CLASS''' <span class="forestGreen">yaml_tree</span>) in a compact way.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Ivec</span> <span class="twilightBlue">! integer structure</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Ivec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Lvec</span> <span class="twilightBlue">! logical structure</span><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Lvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Rvec</span> <span class="twilightBlue">! real structure</span><br /> <span class="darkTurquoise">real</span> (<span class="red">kind_real</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Rvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! string structure</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="darkTurquoise">value</span> <span class="twilightBlue">! scalar value</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! recursive vector</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! values</span></div><br />
<br />
The derived-type structure below, extended/inherited from parent "yaml_Svec", extracts hierarchies of keys and associated values from the '''YAML''' dictionary object. The calling program specifies a key string that may be generated by aggregating nested keys with a period. Also, it can extract flow sequence string element values that are separated by commas.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span>, <span class="darkTurquoise">EXTENDS</span>(<span class="forestGreen">yaml_Svec</span>) :: <span class="forestGreen">yaml_extract</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_vector</span> <span class="twilightBlue">! true if loaded vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_extract</span></div><br />
<br />
These public structures can be used in applications to extract block list '''YAML''' constructs. The key may represent a '''sequence flow''' <span class="red">[</span>...<span class="red">]</span> with a vector of values. The values can be integers, logicals, reals, or strings. For example, suppose that the '''YAML''' file has the following '''blocking list''' entries:<br />
<br />
<div class="box"><span class="violet">import</span>:<br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_downward_heat_flux_in_sea_water</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface net heat flux</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">shflux</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">shf</span>, <span class="twilightBlue">shf_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">true</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_x_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface zonal wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">sustr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">taux</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_y_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface meridional wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">svstr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">tauy</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span></div><br />
<br />
Then, the code to extract the data from the '''YAML''' dictionary (<span class="red">self</span>) is straightforward (see above for how to create and initialize a '''YAML''' dictionary object):<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="darkTurquoise">ONLY</span> : <span class="blue">yaml_get</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">L(:)</span> <span class="twilightBlue">! logical data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">F(:)</span> <span class="twilightBlue">! floating point data</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">S1(:)</span>, <span class="red">S2(:)</span> <span class="twilightBlue">! string data</span><br /><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>short_name'</span>, <span class="red">S1</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>data_variables'</span>, <span class="red">S2</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>scale'</span>, <span class="red">F</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>, <span class="red">L</span>)</div><br />
<br />
Notice that the internal adopted syntax is to aggregate the hierarchy of <span class="violet">keys</span> separated by a <span class="red">dot</span> in the <span class="mediumOrchid">keystring</span> (''e.g.'', <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>) during the extraction call. The above extraction statement yields the following values from a single invocation of the overloaded function <span class="blue">yaml_get</span>:<br />
<br />
<div class="code"> <span class="red">S1(1)%value</span> = <span class="mediumOrchid">'shflux'</span> <span class="twilightBlue">! <span class="violet">'short_name'</span> single value</span><br /> <span class="red">S1(2)%value</span> = <span class="mediumOrchid">'sustr'</span><br /> <span class="red">S1(3)%value</span> = <span class="mediumOrchid">'svstr'</span><br /><br /> <span class="red">S2(1)%vector(1)%value</span> = <span class="mediumOrchid">'shf'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(1)%vector(2)%value</span> = <span class="mediumOrchid">'shf_time'</span><br /><br /> <span class="red">S2(2)%vector(1)%value</span> = <span class="mediumOrchid">'taux'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(2)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">S2(3)%vector(1)%value</span> = <span class="mediumOrchid">'tauy'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(3)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">F(1)</span> = <span class="mediumOrchid">1.0d0</span> <span class="twilightBlue">! <span class="violet">'scale'</span> single value</span><br /> <span class="red">F(2)</span> = <span class="mediumOrchid">1.0d0</span><br /> <span class="red">F(3)</span> = <span class="mediumOrchid">1.0d0</span><br /><br /> <span class="red">L(1)</span> = <span class="mediumOrchid">.TRUE.</span> <span class="twilightBlue">! <span class="violet">'debug write'</span> single value</span><br /> <span class="red">L(2)</span> = <span class="mediumOrchid">.FALSE.</span><br /> <span class="red">L(3)</span> = <span class="mediumOrchid">.FALSE.</span></div><br />
<br />
Check <span class="blue">ROMS/Utility/get_metdata.F</span> for more details on how the metadata is processed from input '''YAML''' files.</div>Arangohttps://www.myroms.org/wiki/index.php?title=YAML_Parser&diff=6536YAML Parser2022-08-30T18:50:45Z<p>Arango: /* Introduction */</p>
<hr />
<div><div class="title">YAML Parser</div><br />
Starting with '''svn''' revision '''-r 1902''' released on March 1, 2022, the ROMS metadata is managed with a [[varinfo.yaml|YAML file]], and the regular text file '''varinfo.dat''' is deprecated. The '''YAML''' files are simple, easy to follow, elegant, portable, and expandable. ROMS can now process YAML files with its parser module, '''[https://www.myroms.org/links/yaml_parser.F yaml_parser.F]'''. Therefore, there is no need to use third-party '''YAML''' parsers.<br />
<br />
The ROMS '''YAML''' parser source code can be found in <span class="forestGreen">ROMS/Utility</span>. It is written in Fortran 2003 and includes a '''CLASS''' of type '''yaml_tree''' for parsing input '''YAML''' files.<br />
<br />
==Introduction==<br />
Although several '''YAML''' parsers for Fortran exist, a more straightforward and uncomplicated parser with substantial capabilities was coded in '''ROMS''', '''[https://www.myroms.org/links/yaml_parser.F yaml_parser.F]'''. It is a hybrid between standard and Object-Oriented Programming (OOP) principles but without the need for recurrency, polymorphism, and containers (another library).<br />
<br />
The only constraint in the '''ROMS''' parser is that the '''YAML''' file is read '''twice''' for simplicity and to avoid containers. The container is a Fortran vector! The first read determines the indentation policy and the length of the collection vector, <span class="red">list(:)</span> pairs object (derived-type structure <span class="forestGreen">yaml_pair</span>). The first reading is quick.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_pair</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_alias</span> <span class="twilightBlue">! alias <span class="violet">*</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_anchor</span> <span class="twilightBlue">! anchor <span class="violet">&</span> token</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_block</span> <span class="twilightBlue">! block <span class="violet">-</span> list</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_sequence</span> <span class="twilightBlue">! sequence <span class="violet">[]</span> tokens</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_logical</span> <span class="twilightBlue">! logical value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_integer</span> <span class="twilightBlue">! integer value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_real</span> <span class="twilightBlue">! floating-point value</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">is_string</span> <span class="twilightBlue">! string value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">id</span> <span class="twilightBlue">! key/value ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">parent_id</span> <span class="twilightBlue">! parent ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">left_padding</span> <span class="twilightBlue">! indent level: 0,1,..</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">line</span> <span class="twilightBlue">! YAML line</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">key</span> <span class="twilightBlue">! YAML keyword</span><span class="violet">:</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">value</span> <span class="twilightBlue">! YAML value(s)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">anchor</span> <span class="twilightBlue">! anchor keyword</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_pair</span></div><br />
The '''YAML''' file dictionary <span class="darkTurquoise">CLASS</span> <span class="forestGreen">yaml_tree</span> is defined as:<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_tree</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsCreated</span> '''= .FALSE.''' <span class="twilightBlue">! YAML object creation switch</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nbranches</span> <span class="twilightBlue">! total number of branches</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Npairs</span> <span class="twilightBlue">! total number of pairs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">indent</span> <span class="twilightBlue">! blank indentation policy</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">filename</span> <span class="twilightBlue">! YAML file name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_pair</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">list(:)</span> <span class="twilightBlue">! collection pairs</span><br /><br /> <span class="darkTurquoise">CONTAINS</span> <span class="twilightBlue">! CLASS objects</span><br /><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">create</span> => <span class="blue">yaml_tree_create</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">destroy</span> => <span class="blue">yaml_tree_destroy</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">dump</span> => <span class="blue">yaml_tree_dump</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">extract</span> => <span class="blue">yaml_tree_extract</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill</span> => <span class="blue">yaml_tree_fill</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">fill_aliases</span> => <span class="blue">yaml_tree_fill_aliases</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">has</span> => <span class="blue">yaml_tree_has</span><br /> <span class="darkTurquoise">PROCEDURE</span> :: <span class="red">read_line</span> => <span class="blue">yaml_tree_read_line</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_tree</span></div><br />
The <span class="forestGreen">yaml_tree</span> object stores all the data contained in a specific '''YAML''' file. For Example, in '''ROMS''' the input '''YAML''' metadata dictionary is created and initialized as follows:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="blue">ONLY</span> : <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lreport</span> <span class="twilightBlue">! verbose report switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_tree</span>) :: <span class="red">self</span> <span class="twilightBlue">! declare a dummy YAML object</span><br /><br /> <span class="darkTurquoise">IF</span> (.<span class="darkTurquoise">not</span>.<span class="darkTurquoise">ASSOCIATED</span>(<span class="red">self%list</span>)) <span class="darkTurquoise">THEN</span> <span class="twilightBlue">! process input YAML file</span><br /> <span class="red">Lreport</span> = <span class="red">.TRUE.</span><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_initialize</span> (<span class="red">self</span>, <span class="red">'varinfo.yaml'</span>, <span class="red">Lreport</span>)<br /> <span class="darkTurquoise">END IF</span></div><br />
The error management is omitted for clarity. Then, the needed data is extracted from the <span class="red">self</span> object and loaded into the internal '''ROMS''' variables using the overloaded <span class="mediumOrchid">yaml_get</span> API:<br />
<br />
<div class="code"> <span class="darkTurquoise">INTERFACE</span> <span class="mediumOrchid">yaml_get</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_i_struc</span> <span class="twilightBlue">! Gets integer structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_l_struc</span> <span class="twilightBlue">! Gets logical structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_r_struc</span> <span class="twilightBlue">! Gets real structure</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_s_struc</span> <span class="twilightBlue">! Gets string structure</span><br /><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_0d</span> <span class="twilightBlue">! Gets integer value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_ivar_1d</span> <span class="twilightBlue">! Gest integer values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_0d</span> <span class="twilightBlue">! Gets logical value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_lvar_1d</span> <span class="twilightBlue">! Gets logical values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_0d</span> <span class="twilightBlue">! Gets real value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_rvar_1d</span> <span class="twilightBlue">! Gets real values</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_0d</span> <span class="twilightBlue">! Gets string value</span><br /> <span class="darkTurquoise">MODULE PROCEDURE</span> <span class="blue">yaml_Get_svar_1d</span> <span class="twilightBlue">! Gets string values</span><br /><br /> <span class="darkTurquoise">END INTERFACE</span> <span class="mediumOrchid">yaml_get</span></div><br />
Additionally, the '''YAML''' parser module has the following public and private routines/functions used during processing:<br />
<br />
<div class="code"> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_AssignString</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_Error</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_get</span><br /> <span class="darkTurquoise">PUBLIC</span> :: <span class="blue">yaml_initialize</span><br /><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_CountKeys</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_LowerCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_UpperCase</span><br /> <span class="darkTurquoise">PRIVATE</span> :: <span class="blue">yaml_ValueType</span></div><br />
<br />
The parser module is self-contained with minimal dependencies on other '''ROMS''' modules:<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">mod_kinds</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">kind_real</span> => <span class="red">dp</span> <span class="twilightBlue">! double-precision</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_parallel</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_Master</span> => <span class="red">Master</span> <span class="twilightBlue">! master PET</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_scalars</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_ErrFlag</span> => <span class="red">exit_flag</span> <span class="twilightBlue">! error flag</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">mod_iounits</span>, <span class="darkTurquoise">ONLY</span> : <span class="red">yaml_stdout</span> => <span class="red">stdout</span> <span class="twilightBlue">! standard ouput</span></div><br />
<br />
Overall, the parser is very fast and works in parallel. All PETs are involved in their dictionary copy to avoid overhead from collective MPI calls.<br />
<br />
==Capabilities==<br />
Currently, the '''YAML''' parser supports the following features:<br />
<br />
*'''Single''' or '''multiple''' line comments start with a hash '''<span class="red">#</span>'''. Also, comment after a key/value pair is allowed. All comments are skipped during processing.<br />
<br />
*It has an '''unlimited nested structure''' (lists, mappings, hierarchies). Indentation of whitespace is used to denote structure.<br />
<br />
*It has an '''unrestricted schema indentation'''. However, some schema validators recommend or impose '''two whitespaces''' indentations.<br />
<br />
*A <span class="red">colon</span> follows a <span class="violet">key</span> to denote a mapping <span class="twilightBlue">value</span> like:<div class="box" style="margin-left: 30px;"><span class="violet">ocean_model</span>: <span class="twilightBlue">ROMS</span></div><br />
<br />
*It supports <span class="green">Anchors</span> and <span class="salmon">Aliases</span>.<div class="box" style="margin-left: 30px;"><span class="violet">ATM_component</span>: <span class="green">&ATM</span> <span class="twilightBlue">WRF</span><br /><br /><span class="violet">metadata</span>:<br /><br /> - <span class="violet">standard_name</span>: <span class="twilightBlue">surface_eastward_wind</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface eastward wind</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">Uwind</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">uwind</span>, <span class="twilightBlue">time</span>]</span><br /> <span class="violet">source_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">destination_units</span>: <span class="twilightBlue">m s-1</span><br /> <span class="violet">source_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">destination_grid</span>: <span class="twilightBlue">cell_center</span><br /> <span class="violet">add_offset</span>: <span class="twilightBlue">0.0d0</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug_write</span>: <span class="twilightBlue">false</span><br /> <span class="violet">connected_to</span>: <span class="salmon">*ATM</span> <span class="twilightBlue"># u10</span><br /> <span class="violet">regrid_method</span>: <span class="twilightBlue">bilinear</span><br /> <span class="violet">extrapolate_method</span>: <span class="twilightBlue">none</span></div><br />
<br />
*It supports '''blocking lists''': members are denoted by a leading <span class="red">hyphen-and-space</span>, which is considered part of the indentation. <br />
<br />
*It supports a '''flow sequence''': a vector list with values enclosed in <span class="red">square brackets</span> and separated by a <span class="red">comma-and-space</span>, like a <span class="violet">keyword</span>: <span class="red">[</span><span class="twilightBlue">val1</span><span class="red">,</span> ...<span class="red">,</span> <span class="twilightBlue">valN</span><span class="red">]</span>. <br />
<br />
*The <span class="violet">keyword</span> <span class="twilightBlue">value(s)</span> is (are) processed and stored as strings but converted to a logical, integer, floating-point, or derived-type when appropriate during extraction. If particular derived-type values are needed, the caller can process such a structure outside the parser. <br />
<br />
*It removes '''unwanted control characters''' like tabs and separators ('''ASCII''' character code '''0-31'''). <br />
<br />
*It is restricted to the '''English''' uppercase and lowercase alphabet but can be expanded to other characters (see <span class="blue">yaml_ValueType</span> routine). <br />
<br />
*Multiple or continuation lines are supported. So, for example, we can have:<div class="box" style="margin-left: 30px;"><span class="violet">state variables</span>: <span class="red">[</span><span class="twilightBlue">sea_surface_height_anomaly</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">barotropic_sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_x_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_y_velocity</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_potential_temperature</span><span class="red">,</span><br /> <span class="twilightBlue">sea_water_practical_salinity</span><span class="red">]</span></div><br />
<br />
==Extraction==<br />
<br />
Several derived-type structures are declared to facilitate the extraction of similar data blocks compactly from the YAML dictionary ('''CLASS''' <span class="forestGreen">yaml_tree</span>) in a compact way.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Ivec</span> <span class="twilightBlue">! integer structure</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Ivec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Lvec</span> <span class="twilightBlue">! logical structure</span><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Lvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Rvec</span> <span class="twilightBlue">! real structure</span><br /> <span class="darkTurquoise">real</span> (<span class="red">kind_real</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Rvec</span><br /><br /> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PUBLIC</span> :: <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! string structure</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="darkTurquoise">value</span> <span class="twilightBlue">! scalar value</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">pointer</span> :: <span class="red">vector(:)</span> <span class="twilightBlue">! recursive vector</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_Svec</span> <span class="twilightBlue">! values</span></div><br />
<br />
The derived-type structure below, extended/inherited from parent "yaml_Svec", extracts hierarchies of keys and associated values from the '''YAML''' dictionary object. The calling program specifies a key string that may be generated by aggregating nested keys with a period. Also, it can extract flow sequence string element values that are separated by commas.<br />
<br />
<div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span>, <span class="darkTurquoise">EXTENDS</span>(<span class="forestGreen">yaml_Svec</span>) :: <span class="forestGreen">yaml_extract</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">has_vector</span> <span class="twilightBlue">! true if loaded vector values</span><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">yaml_extract</span></div><br />
<br />
These public structures can be used in applications to extract block list '''YAML''' constructs. The key may represent a '''sequence flow''' <span class="red">[</span>...<span class="red">]</span> with a vector of values. The values can be integers, logicals, reals, or strings. For example, suppose that the '''YAML''' file has the following '''blocking list''' entries:<br />
<br />
<div class="box"><span class="violet">import</span>:<br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_downward_heat_flux_in_sea_water</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface net heat flux</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">shflux</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">shf</span>, <span class="twilightBlue">shf_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">true</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_x_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface zonal wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">sustr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">taux</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span><br /><br /> <span class="red">-</span> <span class="violet">standard_name</span>: <span class="twilightBlue">surface_wind_y_stress</span><br /> <span class="violet">long_name</span>: <span class="twilightBlue">surface meridional wind stress component</span><br /> <span class="violet">short_name</span>: <span class="twilightBlue">svstr</span><br /> <span class="violet">data_variables</span>: <span class="red">[<span class="twilightBlue">tauy</span>, <span class="twilightBlue">atm_time</span>]</span><br /> <span class="violet">scale</span>: <span class="twilightBlue">1.0d0</span><br /> <span class="violet">debug write</span>: <span class="twilightBlue">false</span></div><br />
<br />
Then, the code to extract the data from the '''YAML''' dictionary (<span class="red">self</span>) is straightforward (see above for how to create and initialize a '''YAML''' dictionary object):<br />
<br />
<div class="code"> <span class="darkTurquoise">USE</span> <span class="blue">yaml_parser_mod</span>, <span class="darkTurquoise">ONLY</span> : <span class="blue">yaml_get</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ErrorFlag</span> <span class="twilightBlue">! processing error flag</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">L(:)</span> <span class="twilightBlue">! logical data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">F(:)</span> <span class="twilightBlue">! floating point data</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">yaml_Svec</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">S1(:)</span>, <span class="red">S2(:)</span> <span class="twilightBlue">! string data</span><br /><br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>short_name'</span>, <span class="red">S1</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>data_variables'</span>, <span class="red">S2</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>scale'</span>, <span class="red">F</span>)<br /> <span class="red">ErrorFlag</span> = <span class="blue">yaml_get</span> (<span class="red">self</span>, <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>, <span class="red">L</span>)</div><br />
<br />
Notice that the internal adopted syntax is to aggregate the hierarchy of <span class="violet">keys</span> separated by a <span class="red">dot</span> in the <span class="mediumOrchid">keystring</span> (''e.g.'', <span class="mediumOrchid">'import<span class="red">.</span>debug write'</span>) during the extraction call. The above extraction statement yields the following values from a single invocation of the overloaded function <span class="blue">yaml_get</span>:<br />
<br />
<div class="code"> <span class="red">S1(1)%value</span> = <span class="mediumOrchid">'shflux'</span> <span class="twilightBlue">! <span class="violet">'short_name'</span> single value</span><br /> <span class="red">S1(2)%value</span> = <span class="mediumOrchid">'sustr'</span><br /> <span class="red">S1(3)%value</span> = <span class="mediumOrchid">'svstr'</span><br /><br /> <span class="red">S2(1)%vector(1)%value</span> = <span class="mediumOrchid">'shf'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(1)%vector(2)%value</span> = <span class="mediumOrchid">'shf_time'</span><br /><br /> <span class="red">S2(2)%vector(1)%value</span> = <span class="mediumOrchid">'taux'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(2)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">S2(3)%vector(1)%value</span> = <span class="mediumOrchid">'tauy'</span> <span class="twilightBlue">! <span class="violet">'data_variables'</span> vector of values</span><br /> <span class="red">S2(3)%vector(2)%value</span> = <span class="mediumOrchid">'atm_time'</span><br /><br /> <span class="red">F(1)</span> = <span class="mediumOrchid">1.0d0</span> <span class="twilightBlue">! <span class="violet">'scale'</span> single value</span><br /> <span class="red">F(2)</span> = <span class="mediumOrchid">1.0d0</span><br /> <span class="red">F(3)</span> = <span class="mediumOrchid">1.0d0</span><br /><br /> <span class="red">L(1)</span> = <span class="mediumOrchid">.TRUE.</span> <span class="twilightBlue">! <span class="violet">'debug write'</span> single value</span><br /> <span class="red">L(2)</span> = <span class="mediumOrchid">.FALSE.</span><br /> <span class="red">L(3)</span> = <span class="mediumOrchid">.FALSE.</span></div><br />
<br />
Check <span class="blue">ROMS/Utility/get_metdata.F</span> for more details on how the metadata is processed from input '''YAML''' files.</div>Arangohttps://www.myroms.org/wiki/index.php?title=NUOPC_Cap&diff=6535NUOPC Cap2022-06-26T21:18:56Z<p>Arango: /* Capabilities */</p>
<hr />
<div><div class="title">ROMS Native ESMF/NUOPC Cap</div><br />
__TOC__<br />
<br />
==Overview==<br />
This document describes the ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span> module <span class="blue">Master/esmf_roms.F</span> used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (<span class="blue">Master/esmf_driver.h</span>), which '''creates''', '''register''', '''initialize''', '''run''', and '''finalize''' the coupling of each Earth System Model ('''ESM''') component ('''data''', '''atmosphere''', '''sea-ice''', '''wave''', and '''ocean'''). It also sets the coupler '''connectors''' and the '''RunSequence''' policy (<span class="blue">Master/esmf_esm.F</span>). The framework provides the '''NUOPC''' <span class="red">cap</span> modules for each coupled component. Detailed information about '''ESMF/NUOPC''' and how it is implemented in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.<br />
<br />
A '''NUOPC''' <span class="red">cap</span> is a Fortran module that serves as the interface to a model when used in a '''NUOPC'''-based coupled system. The term <span class="red">cap</span> is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its '''initialize''', '''run''', and '''finalize''' phases and exposing the model data structures to the field exchange ('''import''' and '''export''' states) between coupled components.<br />
<br />
<br />
[[Image:ROMS_Coupling.png|800px|center]]<br />
<br />
<center>'''''Figure 1:''''' ROMS native ESMF/NUOPC coupling infrastructure</center><br />
<br />
==Implementation==<br />
<br />
The <span class="blue">Master/mod_esmf_esm.F</span> module declares several derived-type structures to facilitate the management of all internal objects and variables in each '''ESM''' coupled component:<br />
<br />
* '''ESM''' coupling time managing variables and '''ESMF''' objects. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Clock</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Restarted</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i8b</span>) :: <span class="red">AdvanceCount</span> <span class="twilightBlue">! advance counter</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Current_Time</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Reference</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Restart</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Start</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Stop</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Step</span> <span class="twilightBlue">! seconds</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">CalendarString</span> <span class="twilightBlue">! 360_day, gregorian</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_ReferenceString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_RestartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StopString</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Calendar</span>) :: <span class="red">Calendar</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Clock</span>) :: <span class="red">Clock</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Direction_flag</span>) :: <span class="red">Direction</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">CurrentTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">ReferenceTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">RestartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StopTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_TimeInterval</span>) :: <span class="red">TimeStep</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Clock</span><br /><br />'''. . .'''<br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''ESM''' coupled state sets. If appropriate, it includes the logic for connecting nested grids. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">LinkedGrid(:,:)</span> <span class="twilightBlue">! connected grid</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">DataCoupledSets(:,:)</span> <span class="twilightBlue">! DATA linked sets</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">SetLabel(:)</span> <span class="twilightBlue">! set label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExpLabel(:)</span> <span class="twilightBlue">! export label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImpLabel(:)</span> ! <span class="twilightBlue">import label</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:,:)</span> <span class="twilightBlue">! export set</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_CplSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">COUPLED(:)</span></div><br />
<br />
* '''DATA''' model field processing information. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Data</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcycle</span> <span class="twilightBlue">! cycling time coordinate</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcoord</span> <span class="twilightBlue">! coordinates attribute</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmask</span> <span class="twilightBlue">! land/sea mask</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmulti</span> <span class="twilightBlue">! field across multi-files</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">LastRec</span> <span class="twilightBlue">! processed last record</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Icomp</span> <span class="twilightBlue">! target component index</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ncid</span> <span class="twilightBlue">! NetCDF file ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tid</span> <span class="twilightBlue">! time variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Vid</span> <span class="twilightBlue">! field variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nvdim</span> <span class="twilightBlue">! number spatial dimensions</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Zlevel</span> <span class="twilightBlue">! level index to process</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nrec</span> <span class="twilightBlue">! number of time records</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Trec</span> <span class="twilightBlue">! latest read time record</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">LandValue</span> <span class="twilightBlue">! Masking land value</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">SeaValue</span> <span class="twilightBlue">! Masking sea value</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Vsize(:)</span> <span class="twilightBlue">! dimensions size</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! add_offset attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">FillValue</span> <span class="twilightBlue">! _FillValue attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! scale_factor attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Clength</span> <span class="twilightBlue">! time cycling length</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMin</span> <span class="twilightBlue">! grid minimum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMax</span> <span class="twilightBlue">! grid maximum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMin</span> <span class="twilightBlue">! grid minimum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMax</span> <span class="twilightBlue">! grid maximum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tscale</span> <span class="twilightBlue">! time scale to day</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmono</span> <span class="twilightBlue">! monotonic time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! time minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! time maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! lower time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! upper time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmin</span> <span class="twilightBlue">! variable minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmax</span> <span class="twilightBlue">! variable maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Date(6,2)</span> <span class="twilightBlue">! time-snapshots dates YYYY,MM,DD hh:mm:ss.ss</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30), <span class="darkTurquoise">allocatable</span> :: <span class="red">Dname(:)</span> <span class="twilightBlue">! variable dimensions names</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Vcoord(:)</span> <span class="twilightBlue">! variable coordinates names</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=22 ) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30 ) :: <span class="red">SpecialAction</span> <span class="twilightBlue">! special processing</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tname</span> <span class="twilightBlue">! time variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tunits</span> <span class="twilightBlue">! time variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vname</span> <span class="twilightBlue">! variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vunits</span> <span class="twilightBlue">! variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vdescriptor</span> <span class="twilightBlue">! Variable descriptive name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vlongname</span> <span class="twilightBlue">! long_name attribute</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! field longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! field latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! field land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2d(:,:)</span> <span class="twilightBlue">! time interpolated 2D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3d(:,:,:)</span> <span class="twilightBlue">! time interpolated 3D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2dG(:,:,:)</span> <span class="twilightBlue">! latest 2D data snapsnots</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3dG(:,:,:,:)</span><span class="twilightBlue">! latest 3D data snapsnots</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Field</span>) :: <span class="red">field</span> <span class="twilightBlue">! field object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>) :: <span class="red">grid</span> <span class="twilightBlue">! field grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> <span class="twilightBlue">! field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Data</span></div><br />
<br />
* '''DATA''' model high-level structure. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_DataSet</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfields</span> <span class="twilightBlue">! number of fields</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfiles</span> <span class="twilightBlue">! number of input files</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Ctarget(:)</span> <span class="twilightBlue">! component target</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Field(:)</span> <span class="twilightBlue">! field short-name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Data</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Export(:)</span> <span class="twilightBlue">! Export field structure</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">T_IO</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">IFS(:)</span> <span class="twilightBlue">! Input Files structure</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_DataSet</span></div><br />
<br />
* Import and export fields metadata information. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Field</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">connected</span> <span class="twilightBlue">! connected to coupler</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">debug_write</span> <span class="twilightBlue">! write exchanged field</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">enable_integral_adj</span> <span class="twilightBlue">! area integral adjusted</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">fid</span> <span class="twilightBlue">! internal field ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! field grid mesh type</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">etype</span> <span class="twilightBlue">! field extrapolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">itype</span> <span class="twilightBlue">! field interpolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=20) :: <span class="red">Ctarget</span> <span class="twilightBlue">! component destination</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">short_name</span> <span class="twilightBlue">! short name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">standard_name</span> <span class="twilightBlue">! standard name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">long_name</span> <span class="twilightBlue">! long name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_gtype</span> <span class="twilightBlue">! DST grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_units</span> <span class="twilightBlue">! DST units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_gtype</span> <span class="twilightBlue">! SRC grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_units</span> <span class="twilightBlue">! SRC units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_vname</span> <span class="twilightBlue">! DATA Vname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_tname</span> <span class="twilightBlue">! DATA Tname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">RegridMethod</span> <span class="twilightBlue">! regrid method</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExtrapMethod</span> <span class="twilightBlue">! extrapolate</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! field scale factor</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! field add offset value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! DATA time minimum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! DATA time maximum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! DATA lower time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! DATA upper time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (day)</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> ! <span class="twilightBlue">field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Field</span></div><br />
<br />
* Import and export fields mesh data. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Mesh</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gid</span> <span class="twilightBlue">! grid ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! grid mesh type</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! grid land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! grid longitude</span><br /> <span class="red">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! grid latitude</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">area(:,:)</span> <span class="twilightBlue">! grid area</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Mesh</span></div><br />
<br />
* Melding coefficients used to combine fields from '''DATA''' and '''ESM''' components. The weight factors are read from the input NetCDF specified in the <span class="blue">WeightsFile(atmos)</span> keyword. The user has full control of how the merging is done. It is recommended to provide a gradual transition between the two components. <br /><br /> Recall that the '''DATA''' component supplies needed data to a particular '''ESM''' component. For example, it may export data to the atmosphere model at locations not covered by the other '''ESM''' components because of smaller grid coverage. If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data: <div class="code"> <span class="red">SST_atm(:,:)</span> = <span class="red">Cesm(:,:)</span> * <span class="red">SST_esm(:,:)</span> + <span class="red">Cdat(:,:)</span> * <span class="red">SST_dat(:,:)</span></div> where<span class="red"> Cesm(:,:)</span> + <span class="red">Cdat(:,:)</span> = <span class="red">1</span>.<br /><div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">NestedGrid</span> <span class="twilightBlue">! grid needing merged field</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameDATA</span> <span class="twilightBlue">! DATA weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameESM</span> <span class="twilightBlue">! ESM weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! Weights NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="forestGreen">allocatable</span> :: <span class="red">Cdat(:,:)</span> <span class="twilightBlue">! coefficients for DATA</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cesm(:,:)</span> <span class="twilightBlue">! coefficients for ESM</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Meld</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">WEIGHTS(:)</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightDAT</span> = <span class="red">0.0_dp</span> <span class="twilightBlue">! DATA component weight</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightESM</span> = <span class="red">1.0_dp</span> <span class="twilightBlue">! ESM component weight</span></div><br />
<br />
* Coupled models high-level object, [<span class="red">Nmodels=5</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active for coupling</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">LandValue</span> <span class="twilightBlue">! land mask value</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">SeaValue</span> <span class="twilightBlue">! sea mask value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Ngrids</span> <span class="twilightBlue">! number nested grids</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ExportCalls</span> <span class="twilightBlue">! export CALL counter</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ImportCalls</span> <span class="twilightBlue">! import CALL counter</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number model PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! component PETs list</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">TimeFrac(:,:)</span> <span class="twilightBlue">! driver time fraction</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">name</span> <span class="twilightBlue">! component name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">grid(:)</span> <span class="twilightBlue">! grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Mesh</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mesh(:)</span> <span class="twilightBlue">! mesh</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportField(:)</span> <span class="twilightBlue">! import fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportField(:)</span> <span class="twilightBlue">! export fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportState(:)</span> <span class="twilightBlue">! import state</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:)</span> <span class="twilightBlue">! export state</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Model</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">MODELS(:)</span></div><br />
<br />
* Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [<span class="red">Nmodels</span>, <span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active connector</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">divDT</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">MaskInteraction</span> <span class="twilightBlue">! connector mask interaction</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number of connector PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! connector PETs list</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100) :: <span class="red">name</span> <span class="twilightBlue">! connector name</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Conn</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">CONNECTORS(:,:)</span></div><br />
<br />
* Define '''DATA''' Model field processing information from input data files, [<span class="red">1</span>:<span class="red">Nmodels</span>]. Currently, the '''DATA''' Model only export fields and it only supports input NetCDF files. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_DataSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">DataSet(:)</span></div><br />
<br />
* '''ESM''' clock for driver (zeroth element) and coupled components, [<span class="red">0</span>:<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''Internal module parameters and variables:'''<br /> <br />
** Number of coupled '''ESM''' gridded components. Currently, five '''ESM''' components are supported ('''ROMS''', '''DATA''', '''ATMOSPHERE''', '''SEA-ICE''', and '''WAVE''' model types). <br /> All supported components are accounted here even if we are running an application with less number of models. The <span class="red">IsActive</span> switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Nmodels</span> = <span class="red">5</span></div><br />
** Number of nested grids. All are initialized to just one grid; its values are overwritten during processing. An additional variable '''NgridsR''' is created to avoid using ROMS module <span class="blue">mod_param</span> in the generic interface. Both '''Ngrids''' and '''NgridsR''' will have the same value.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsA</span> = <span class="red">1</span> <span class="twilightBlue">! Atmosphere Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsD</span> = <span class="red">1</span> <span class="twilightBlue">! DATA Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsI</span> = <span class="red">1</span> <span class="twilightBlue">! Sea-ice Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsR</span> = <span class="red">1</span> <span class="twilightBlue">! ROMS</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsW</span> = <span class="red">1</span> <span class="twilightBlue">! Wave Model</span></div><br />
** Coupled '''ESM''' components identification indices.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idriver</span> = <span class="red">0</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iroms</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iatmos</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idata</span> = <span class="red">3</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseaice</span> = <span class="red">4</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iwave</span> = <span class="red">5</span></div><br />
** Generic '''ESM''' component labels used in the <span class="darkTurquoise">CASE</span> constructs. We cannot use the identification indices because the vector <span class="red">Iroms(:)</span> cannot be defined as a parameter and a non constant expression is illegal: <span class="darkTurquoise">CASE</span> ( <span class="red">Iroms</span>(<span class="red">1</span>) : <span class="red">Iroms</span>(<span class="red">NgridsR</span>) )<div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 3), <span class="darkTurquoise">allocatable</span> :: <span class="red">Clabel(:)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len=</span>10), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cmodel(:)</span></div><br />
** Number of ROMS export and import fields per component. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nexport(:)</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nimport(:)</span></div><br />
** Model coupling type: [<span class="red">1</span>] Explicit, [<span class="red">otherwise</span>] Semi-Implicit. In <span class="red">explicit coupling</span>, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In <span class="red">implicit coupling</span>, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In <span class="red">semi-implicit coupling</span>, '''ROMS -> ATM''' is explicit, '''ATM -> ROMS''' is implicit. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">CouplingType</span> = <span class="red">1</span></div><br />
** Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent). <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">ESMcomm(:)</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">sumPETs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">PETrank</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=10) :: <span class="red">PETlayoutOption</span></div><br />
** Driver clock parameters specified in configuration script. A integer vector with six elements: (<span class="red">1</span>) year including century, like 2017; (<span class="red">2</span>) month of the year, 1 to 12; (<span class="red">3</span>) day of the month; (<span class="red">4</span>) hour of the day, 0 to 23; (<span class="red">5</span>) minutes of the hour, 0 to 59; and (<span class="red">6</span>) seconds of the minute, 0 to 59 <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ReferenceDate</span>(6) <span class="twilightBlue">! reference date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">RestartDate</span>(6) <span class="twilightBlue">! restarting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StartDate</span>(6) <span class="twilightBlue">! starting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StopDate</span>(6) <span class="twilightBlue">! stopping date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">TimeStep</span>(6) <span class="twilightBlue">! coupling interval</span></div><br />
** Today date string. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=44) :: <span class="red">TodayDateString</span></div><br />
** '''ESM''' coupling simulation reference date number, element: (<span class="red">1</span>) seconds, and (<span class="red">2</span>) fractional days<div class="code"> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">ReferenceDateNumber</span>(2)</div><br />
** '''DATA''' component parallel distributed-memory domain partions in the '''I'''- and '''J'''-directions (lon,lat). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ItileD</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">JtileD</span></div><br />
** Coupling debugging flag: [<span class="red">0</span>] no debugging, [<span class="red">1</span>] reports informative messages, or [<span class="red">2</span>] <span class="red">'1'</span> plus writes grid information in '''VTK''' format, [<span class="red">3</span>] <span class="red">'2'</span> plus writes exchage fields into '''NetCDF''' files. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">DebugLevel</span> = <span class="red">0</span></div><br />
** Execution tracing level flag: [<span class="red">0</span>] no tracing, [<span class="red">1</span>] reports sequence of coupling subroutine calls, or [<span class="red">2</span>] <<span class="red">1</span>> plus writes voluminous '''ESMF''' library tracing information which slowdown performace, and creates large log file. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">TraceLevel</span> = <span class="red">0</span></div><br />
** Switch to trace/track run sequence during debugging. All information is written to Fortan unit <span class="red">trac</span>. For now, use standard output unit. <div class="code"> <span class="darkTurquoise">logical</span> :: <span class="red">ESM_track</span> = <span class="red">.FALSE.</span> <span class="twilightBlue">! trace/track CALL sequence switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">trac</span> = <span class="red">6</span> <span class="twilightBlue">! trace/track CALL sequence unit</span></div><br />
** Coupled model staggered grid-cell type indices. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inan</span> = <span class="red">0</span> <span class="twilightBlue">! unstaggered, cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icenter</span> = <span class="red">1</span> <span class="twilightBlue">! cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icorner</span> = <span class="red">2</span> <span class="twilightBlue">! cell corners</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iupoint</span> = <span class="red">3</span> <span class="twilightBlue">! right and left cell faces</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ivpoint</span> = <span class="red">4</span> <span class="twilightBlue">! upper and lower cell faces</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=6), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">GridType</span> = &<br /> & (/ '<span class="red">N/A</span> ', &<br /> & '<span class="red">Center</span>', &<br /> & '<span class="red">Corner</span>', &<br /> & '<span class="red">U</span> ', &<br /> & '<span class="red">V</span> ' /)</div><br />
** '''REGRID''' interpolation method between source and destination fields. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ibilin</span> = <span class="red">1</span> <span class="twilightBlue">! bilinear</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipatch</span> = <span class="red">2</span> <span class="twilightBlue">! high-order patch recovery</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv1</span> = <span class="red">3</span> <span class="twilightBlue">! first-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv2</span> = <span class="red">4</span> <span class="twilightBlue">! second-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InStoD</span> = <span class="red">5</span> <span class="twilightBlue">! nearest neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InDtoS</span> = <span class="red">6</span> <span class="twilightBlue">! nearest neighbor Dst to Src</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:6) :: <span class="red">IntrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">BLIN</span>', &<br /> & '<span class="red">PTCH</span>', &<br /> & '<span class="red">CNS1</span>', &<br /> & '<span class="red">CNS2</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">ND2S</span>' /)</div><br />
** Extrapolation method for unmapped destination points. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Enone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">ExStoD</span> = <span class="red">1</span> <span class="twilightBlue">! nearear neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Eidavg</span> = <span class="red">2</span> <span class="twilightBlue">! inverse distance average</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ecreep</span> = <span class="red">3</span> <span class="twilightBlue">! creep fill</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">E2steps</span> = <span class="red">4</span> <span class="twilightBlue">! Turuncoglu two steps</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">ExtrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">IDAV</span>', &<br /> & '<span class="red">CREE</span>', &<br /> & '<span class="red">2STP</span>' /)</div><br />
** The number of levels to output for the extrapolation methods that fill levels, like creep fill (<span class="red">ESMF_EXTRAPMETHOD_CREEP</span>). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels. For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding ('''ESMF''' Reference Manual, v '''8.0.0'''). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">extrapNumLevels</span> = <span class="red">1</span></div><br />
** Interpolation connectors mask interaction flags. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverLand</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverOcean</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverAll</span> = <span class="red">3</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=3), <span class="darkTurquoise">dimension</span>(3) :: <span class="red">MaskType</span> = &<br /> & (/ '<span class="red">LND</span>', &<br /> & '<span class="red">OCN</span>', &<br /> & '<span class="red">ALL</span>'/)</div><br />
** Coupling run mode: sequential or concurrent. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseq</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipar</span> = <span class="red">2</span><br /><br /> <span class="darkTurquoise">character</span> (len=10), <span class="darkTurquoise">dimension</span>(2) :: <span class="red">RunMode</span> = &<br /> & (/ '<span class="red">SEQUENTIA</span>L', &<br /> & '<span class="red">CONCURRENT</span>' /)</div><br />
** Compling standard input parameters filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CinpName</span></div><br />
** '''ESM''' free-format run sequence configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CONFname</span></div><br />
** ROMS coupling '''YAML''' configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">CPLname</span></div><br />
** Standard input filename for each coupled model, [<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">INPname(:)</span></div><br />
** Standard output units and log messages filename for coupler and '''ESMF''' library. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">cplout</span> = <span class="red">77</span> <span class="twilightBlue">! coupling driver</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">dataout</span> = <span class="red">77</span> <span class="twilightBlue">! data component</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">ESMnameLog</span> = '<span class="red">log.esmf</span>'<br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">CouplerLog</span> = '<span class="red">log.coupler</span>'</div><br />
** Output '''NetCDF''' file used to store field snapshot attributes needed for time interpolation by the '''ESM''' component kernel during concurrent coupling. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=17), <span class="darkTurquoise">parameter</span> :: <span class="red">AttFileName</span> = '<span class="red">time_intrp_att.nc</span>'</div><br />
** '''ESM''' single and double precision constants. <div class="code"> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MAPPED_MASK</span> = <span class="red">99_i4b</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">UNMAPPED_MASK</span> = <span class="red">98_i4b</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_dp</span> = <span class="red">1.0E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r4</span> = <span class="red">1.0E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r8</span> = <span class="red">1.0E20_r8</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_dp</span> = <span class="red">0.5E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r4</span> = <span class="red">0.5E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r8</span> = <span class="red">0.5E20_r8</span></div><br />
<br />
==Capabilities==<br />
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by '''NUOPC'''. According to a predefined calling sequence, these subroutines are called by the '''NUOPC''' infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.<br />
<br />
The initialization sequence is the most complex and is governed by the '''NUOPC''' technical rules. Details about the initialization sequence can be found in the '''NUOPC''' Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" width="1000"<br />
|valign="top"|<span class="blue">ROMS_SetServices</span> <br />
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component chared-object entry points for using '''NUOPC''' generic methods for '''initialize''', '''run''', and '''finalize'''.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span><br />
|ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP2</span><br />
|ROMS component phase 2 initialization which initializes the ROMS component (<span class="blue">ROMS_initialize</span>), sets component grid (<span class="blue">ROMS_SetGridArrays</span>), and adds fields into import and export into respective states.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_DataInit</span><br />
|Exports ROMS component fields during initialization or restart.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetClock</span><br />
|Sets ROMS component date calendar, start and stop times, and coupling interval.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetRunClock</span><br />
|Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_CheckImport</span><br />
|Checks if ROMS component import field is at the correct time.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetGridArrays</span><br />
|Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetStates</span><br />
|Adds ROMS component <span class="red">export</span> and <span class="red">import</span> fields into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_ModelAdvance</span><br />
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_Import</span> and <span class="blue">ROMS_Export</span> routines.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetFinalize</span><br />
|Finalizes ROMS component execution.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Import</span><br />
|Loads import fields into ROMS internal kernel arrays.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Export</span><br />
|Exports ROMS fields to other gridded components.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Rotate</span><br />
|Rotates vector components from computational grid to geographical '''EAST''' and '''NORTH''' directions or vice versa.<br />
|-<br />
|}<br />
<br />
<br />
Other <span class="red">NUOPC</span> cap modules are available within the ROMS native framework:<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | '''ESMF/NUOPC Modules'''<br />
| align="center" | '''Tested'''<br />
|style="width: 700px;" align="center" | '''Description'''<br />
|-<br />
| <span class="red">Master/esmf_driver.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''ESMF/NUOPC''' Master Driver<br />
|-<br />
| <span class="red">Master/esmf_esm.F</span><br />
| <span class="forestGreen">yes</span><br />
| Sets '''ESMF/NUOPC''' '''Services''' and '''RunSequence''' for each coupled component<br />
|-<br />
| <span class="red">Master/esmf_coupler.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''Computes''', '''Execute''', and '''Release''' the '''Connectors''' between <span class="violet">source</span> and <span class="violet">destination</span> fields<br />
|-<br />
| <span class="red">Master/mod_esmf_esm.h</span><br />
| <span class="forestGreen">yes</span><br />
| Defines framework object '''structures''' and '''variables''' and includes support routines<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_regcm.h</span><br />
| <span class="red">no</span><br />
| ICTP Regional Climate Model ('''RegCM''', Version '''4.5''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_wrf.h</span><br />
| <span class="forestGreen">yes</span><br />
| Weather Research and Forecasting model ('''WRF''')<br />
|-<br />
| <span class="blue">Master/esmf_data.F</span><br />
| <span class="forestGreen">yes</span><br />
| Generic '''DATA''' component via '''NetCDF''' files<br />
|-<br />
| <span class="blue">Master/esmf_ice_cice.h</span><br />
| <span class="red">no</span><br />
| Los Alamos Sea Ice Model ('''CICE''')<br />
|-<br />
| <span class="blue">Master/esmf_roms.F</span><br />
| <span class="forestGreen">yes</span><br />
| Regional Ocean Modeling System ('''ROMS''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_wam.h</span><br />
| <span class="red">no</span><br />
| ECMWF's Wave Model ('''WAM''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_ww3.h</span><br />
| <span class="red">no</span><br />
| WaveWatch III ('''WW3'''), <span class="twilightBlue">under development</span><br />
|}<br />
<div style="clear: both"></div><br />
<br />
==Import and Export Fields==<br />
<br />
The <span class="red">export</span> and <span class="red">import</span> fields are configured and specified in the '''YAML''' file [[coupling_esmf_atm.yaml]]. Check the following [[coupling_esmf_atm.yaml|wikiROMS]] page for detailed information. The user has full control of the coupling exchange fields metadata.<br />
<br />
The '''ESMF/NUOPC''' library version '''8.0.0''' or higher supports native nesting capabilities. The '''NUOPC''' layer now allows coupling sets in the <span class="red">import</span> and <span class="red">export</span> states of the '''ESM''' components. A nested grid is represented as another '''ESMF_Grid''' object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The '''NUOPC''' connector can recognize the different grids and exchange fields to a connected '''ESM''' component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The <span class="blue">NUOPC_AddNestedState</span> routine is used to advertise the nested couple sets.<br />
<br />
For example, the ROMS '''NOUPC''' <span class="red">cap</span> module routine <span class="blue">ROMS_SetInitializeP1</span> illustrates how easily the coupled sets is added to the <span class="red">ImportState</span> and <span class="red">ExportState</span> as shown below in the green blocks of code. <br />
<br />
<br />
{|cellpadding="3"<br />
|-<br />
|[[Image:ImportCoupledStates.png]]<br />
|[[Image:ExportCoupledStates.png]]<br />
|}</div>Arangohttps://www.myroms.org/wiki/index.php?title=NUOPC_Cap_UFS&diff=6534NUOPC Cap UFS2022-06-26T21:18:16Z<p>Arango: /* Module Subroutines */</p>
<hr />
<div><div class="title">ROMS Stand-Alone ESMF/NUOPC Cap for the UFS</div><br />
__TOC__<br />
<br />
==Overview==<br />
This document describes the stand-alone ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span> module to be used by third-party coupling frameworks, like the Unified Forecast System ('''UFS'''). It is a lightweight software layer on top of ROMS that can be used by '''NUOPC'''-based packages ('''CMEPS/CDEPS''', '''NEMS''', and others) to couple to other Earth System Models ('''ESM'''s). Detailed information about '''ESMF/NUOPC''' and its implementation in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.<br />
<br />
A '''NUOPC''' <span class="red">cap</span> is a Fortran module that serves as the interface to a model when it's used in a '''NUOPC'''-based coupled system. The term <span class="red">cap</span> is used because it is a lightweight software layer that sits on top of model code, making calls to it and exposing model data structures in a standard way.<br />
<br />
==Implementation==<br />
<br />
The <span class="blue">cmeps_roms.F</span> module declares several derived-type structures to facilitate the management of all internal objects and variables:<br />
<br />
* '''ESM''' coupling time managing variables and '''ESMF''' objects. <div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span> :: <span class="forestGreen">ESM_Clock</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Restarted</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i8b</span>) :: <span class="red">AdvanceCount</span> <span class="twilightBlue">! advance counter</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Current_Time</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Reference</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Restart</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Start</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Stop</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Step</span> <span class="twilightBlue">! seconds</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">CalendarString</span> <span class="twilightBlue">! 360_day, gregorian</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_ReferenceString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_RestartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StopString</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Calendar</span>) :: <span class="red">Calendar</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Clock</span>) :: <span class="red">Clock</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Direction_flag</span>) :: <span class="red">Direction</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">CurrentTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">ReferenceTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">RestartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StopTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_TimeInterval</span>) :: <span class="red">TimeStep</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Clock</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''ESM''' coupled state sets. If appropriate, it includes the logic for connecting nested grids. <div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span> :: <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">LinkedGrid(:,:)</span> <span class="twilightBlue">! connected grid</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">SetLabel(:)</span> <span class="twilightBlue">! set label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExpLabel(:)</span> <span class="twilightBlue">! export label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImpLabel(:)</span> ! <span class="twilightBlue">import label</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_CplSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">COUPLED(:)</span></div><br />
<br />
* Import and export fields metadata information. <div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span> :: <span class="forestGreen">ESM_Field</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">connected</span> <span class="twilightBlue">! connected to coupler</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">debug_write</span> <span class="twilightBlue">! write exchanged field</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> ! <span class="twilightBlue">field grid mesh type</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">short_name</span> <span class="twilightBlue">! short name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">standard_name</span> <span class="twilightBlue">! standard name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">long_name</span> <span class="twilightBlue">! long name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_gtype</span> <span class="twilightBlue">! DST grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_units</span> <span class="twilightBlue">! DST units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_gtype</span> <span class="twilightBlue">! SRC grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_units</span> <span class="twilightBlue">! SRC units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_vname</span> <span class="twilightBlue">! DATA Vname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_tname</span> <span class="twilightBlue">! DATA Tname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">map_norm</span> <span class="twilightBlue">! mapping norm</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">map_type</span> <span class="twilightBlue">! regrid method</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! snapshots date</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! field scale factor</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! field add offset value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! DATA time minimum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! DATA time maximum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! DATA lower time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! DATA upper time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (day)</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> ! <span class="twilightBlue">field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Field</span></div><br />
<br />
* Import and export fields mesh data. <div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span> :: <span class="forestGreen">ESM_Mesh</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gid</span> <span class="twilightBlue">! grid ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! grid mesh type</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! grid land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! grid longitude</span><br /> <span class="red">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! grid latitude</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">area(:,:)</span> <span class="twilightBlue">! grid area</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Mesh</span></div><br />
<br />
* Coupled models high-level object, [<span class="red">Nmodels=1</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span>, <span class="darkTurquoise">PRIVATE</span> :: <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active for coupling</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">LandValue</span> <span class="twilightBlue">! land mask value</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">SeaValue</span> <span class="twilightBlue">! sea mask value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Ngrids</span> <span class="twilightBlue">! number nested grids</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ExportCalls</span> <span class="twilightBlue">! export CALL counter</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ImportCalls</span> <span class="twilightBlue">! import CALL counter</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number model PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! component PETs list</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">TimeFrac(:,:)</span> <span class="twilightBlue">! driver time fraction</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">name</span> <span class="twilightBlue">! component name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">grid(:)</span> <span class="twilightBlue">! grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Mesh</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mesh(:)</span> <span class="twilightBlue">! mesh</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportField(:)</span> <span class="twilightBlue">! import fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportField(:)</span> <span class="twilightBlue">! export fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportState(:)</span> <span class="twilightBlue">! import state</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:)</span> <span class="twilightBlue">! export state</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Model</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">MODELS(:)</span></div><br />
<br />
* '''Internal module parameters and variables:'''<br />
** Switch to trace/track run sequence during debugging. All information is written to Fortran unit trac. For now, it uses the standard output unit. <div class="code"> <span class="darkTurquoise">logical</span> :: <span class="red">ESM_track</span> = <span class="red">.TRUE.</span> <span class="twilightBlue">! trace/track CALL sequence switch</span></div><br />
** Number of coupled '''ESM''' gridded components and identification index. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Nmodels</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idriver</span> = <span class="red">0</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iroms</span> = <span class="red">1</span></div><br />
** Number of ROMS export and import fields per component. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nexport(:)</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nimport(:)</span></div><br />
** Model coupling type: [<span class="red">1</span>] Explicit, [<span class="red">otherwise</span>] Semi-Implicit. In <span class="red">explicit coupling</span>, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In <span class="red">implicit coupling</span>, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In <span class="red">semi-implicit coupling</span>, '''ROMS -> ATM''' is explicit, '''ATM -> ROMS''' is implicit. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">CouplingType</span> = <span class="red">1</span></div><br />
** Linked/coupled ROMS nested grid number. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">linked_grid</span></div><br />
** Distributed-memory communicator handle for each component, rank of each PET, and PET layout (sequential or concurrent). <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">ESMcomm(:)</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">PETrank</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=10), <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlayoutOption(:)</span></div><br />
** Coupling debugging flag: [<span class="red">0</span>] no debugging, [<span class="red">1</span>] reports informative messages, or [<span class="red">2</span>] <span class="red">'1'</span> plus writes grid information in '''VTK''' format, [<span class="red">3</span>] <span class="red">'2'</span> plus writes exchage fields into '''NetCDF''' files. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">DebugLevel</span> = <span class="red">0</span></div><br />
** Execution tracing level flag: [<span class="red">0</span>] no tracing, [<span class="red">1</span>] reports sequence of coupling subroutine calls, or [<span class="red">2</span>] <<span class="red">1</span>> plus writes voluminous '''ESMF''' library tracing information which slowdown performace, and creates large log file. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">TraceLevel</span> = <span class="red">0</span></div><br />
** Standard output units. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">cplout</span> = <span class="red">77</span> <span class="twilightBlue">! coupling driver</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">trac</span> = <span class="red">6</span> <span class="twilightBlue">! trace/track CALL sequence unit</span></div><br />
** Coupled model staggered grid-cell type indices. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inan</span> = <span class="red">0</span> <span class="twilightBlue">! unstaggered, cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icenter</span> = <span class="red">1</span> <span class="twilightBlue">! cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icorner</span> = <span class="red">2</span> <span class="twilightBlue">! cell corners</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iupoint</span> = <span class="red">3</span> <span class="twilightBlue">! right and left cell faces</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ivpoint</span> = <span class="red">4</span> <span class="twilightBlue">! upper and lower cell faces</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=6), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">GridType</span> = &<br /> & (/ '<span class="red">N/A</span> ', &<br /> & '<span class="red">Center</span>', &<br /> & '<span class="red">Corner</span>', &<br /> & '<span class="red">U</span> ', &<br /> & '<span class="red">V</span> ' /)</div><br />
** Standard input filename for each coupled model, [<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">INPname(:)</span></div><br />
** ROMS coupling '''YAML''' configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">CPLname</span></div><br />
** '''ESM''' strings. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">CoupledSet</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportStateName</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportStateName</span></div><br />
** '''ESM''' single and double precision constants. <div class="code"> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MAPPED_MASK</span> = <span class="red">99_i4b</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">UNMAPPED_MASK</span> = <span class="red">98_i4b</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_dp</span> = <span class="red">1.0E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r4</span> = <span class="red">1.0E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r8</span> = <span class="red">1.0E20_r8</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_dp</span> = <span class="red">0.5E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r4</span> = <span class="red">0.5E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r8</span> = <span class="red">0.5E20_r8</span></div><br />
<br />
==UFS Interface==<br />
The '''NUOPC''' <span class="red">cap</span> file is connected to the <span class="blue">UFS.F90</span> and <span class="blue">EARTH_GRID_COMP.F90</span> with '''C'''-preprocessing option <span class="mediumOrchid">FRONT_ROMS</span> as follows:<br />
<br />
<div class="code"> <span class="darkTurquoise">MODULE</span> <span class="blue">module_EARTH_GRID_COMP</span><br /><br /> ...<br /><br /><span class="mediumOrchid">#ifdef FRONT_ROMS</span><br /> <span class="darkTurquoise">USE</span> <span class="blue">cmeps_roms_mod</span>, <span class="darkTurquoise">ONLY</span> : <span class="blue">ROMS_SS</span> => <span class="blue">ROMS_SetServices</span><br /><span class="mediumOrchid">#endif</span><br /><br /> ...<br /><br /> <span class="darkTurquoise">END MODULE</span> <span class="blue">module_EARTH_GRID_COMP</span></div> <br />
<br />
==Module Subroutines==<br />
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by '''NUOPC'''. These subroutines are called by the '''NUOPC''' infrastructure according to a predefined calling sequence. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.<br />
<br />
The initialization sequence is the most complex and is governed by the NUOPC technical rules. Details about the initialization sequence can be found in the '''NUOPC''' Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" width="1000"<br />
|valign="top"|<span class="blue">ROMS_SetServices</span> <br />
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component shared-object entry points for using '''NUOPC''' generic methods for '''initialize''', '''run''', and '''finalize'''.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Create</span> <br />
|Allocates module internal structures and processes configuration from '''YAML''' file: [[roms_cmeps.yaml]].<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span><br />
|ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP2</span><br />
|ROMS component phase 2 initialization which initializes the ROMS component (<span class="blue">ROMS_Initialize</span>), sets component grid (<span class="blue">ROMS_SetGridArrays</span>), and adds fields into import and export into respective states.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_DataInit</span><br />
|Exports ROMS component fields during initialization or restart.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetClock</span><br />
|Sets ROMS component date calendar, start and stop times, and coupling interval.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetRunClock</span><br />
|Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_CheckImport</span><br />
|Checks if ROMS component import field is at the correct time.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetGridArrays</span><br />
|Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetStates</span><br />
|Adds ROMS component <span class="red">export</span> and <span class="red">import</span> fields into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_ModelAdvance</span><br />
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_Import</span> and <span class="blue">ROMS_Export</span> routines.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetFinalize</span><br />
|Finalizes ROMS component execution.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Import</span><br />
|Loads import fields into ROMS internal kernel arrays.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Export</span><br />
|Exports ROMS fields to other gridded components.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Rotate</span><br />
|Rotates vector components from computational grid to geographical '''EAST''' and '''NORTH''' directions or vice versa.<br />
|-<br />
|valign="top"|<span class="blue">Report_TimeStamp</span><br />
|Reports coupling time-stamp.<br />
|-<br />
|}<br />
<br />
==Import and Export Fields==<br />
<br />
The <span class="red">export</span> and <span class="red">import</span> fields are configured and specified in the '''YAML''' file [[roms_cmeps.yaml]]. Check the following [[roms_cmeps.yaml|wikiROMS]] page for detailed information. The user has full control of the coupling exchange fields metadata.<br />
<br />
Currently, '''CMEPS''' does not support coupling of nested grids. The '''ESMF/NUOPC''' library version '''8.0.0''' or higher supports native nesting capabilities. The '''NUOPC''' layer now allows coupling sets in the <span class="red">import</span> and <span class="red">export</span> states of the '''ESM''' components. A nested grid is represented as another '''ESMF_Grid''' object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The '''NUOPC''' connector can recognize the different grids and exchange fields to a connected '''ESM''' component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The <span class="blue">NUOPC_AddNestedState</span> routine is used to advertise the nested couple sets.<br />
<br />
For example, the '''NOUPC''' cap module routines <span class="blue">ROMS_SetInitializeP1</span> can be modified easily to add the coupled sets to the <span class="red">ImportState</span> and <span class="red">ExportState</span> as shown below in the green blocks of code.<br />
<br />
<br />
{|cellpadding="3"<br />
|-<br />
|[[Image:ImportCoupledStates.png]]<br />
|[[Image:ExportCoupledStates.png]]<br />
|}</div>Arangohttps://www.myroms.org/wiki/index.php?title=NUOPC_Cap&diff=6533NUOPC Cap2022-06-26T21:10:21Z<p>Arango: /* Capabilities */</p>
<hr />
<div><div class="title">ROMS Native ESMF/NUOPC Cap</div><br />
__TOC__<br />
<br />
==Overview==<br />
This document describes the ROMS [[Model_Coupling_ESMF|ESMF/NUOPC]] <span class="red">cap</span> module <span class="blue">Master/esmf_roms.F</span> used in the ROMS native coupling infrastructure shown below. In this case, ROMS is driving the coupled system (<span class="blue">Master/esmf_driver.h</span>), which '''creates''', '''register''', '''initialize''', '''run''', and '''finalize''' the coupling of each Earth System Model ('''ESM''') component ('''data''', '''atmosphere''', '''sea-ice''', '''wave''', and '''ocean'''). It also sets the coupler '''connectors''' and the '''RunSequence''' policy (<span class="blue">Master/esmf_esm.F</span>). The framework provides the '''NUOPC''' <span class="red">cap</span> modules for each coupled component. Detailed information about '''ESMF/NUOPC''' and how it is implemented in ROMS can be found on the [[Model_Coupling_ESMF|Earth System Modeling Framework]] WikiROMS page.<br />
<br />
A '''NUOPC''' <span class="red">cap</span> is a Fortran module that serves as the interface to a model when used in a '''NUOPC'''-based coupled system. The term <span class="red">cap</span> is used because it is a lightweight software layer that sits on top of the model kernel, making calls to its '''initialize''', '''run''', and '''finalize''' phases and exposing the model data structures to the field exchange ('''import''' and '''export''' states) between coupled components.<br />
<br />
<br />
[[Image:ROMS_Coupling.png|800px|center]]<br />
<br />
<center>'''''Figure 1:''''' ROMS native ESMF/NUOPC coupling infrastructure</center><br />
<br />
==Implementation==<br />
<br />
The <span class="blue">Master/mod_esmf_esm.F</span> module declares several derived-type structures to facilitate the management of all internal objects and variables in each '''ESM''' coupled component:<br />
<br />
* '''ESM''' coupling time managing variables and '''ESMF''' objects. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Clock</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Restarted</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i8b</span>) :: <span class="red">AdvanceCount</span> <span class="twilightBlue">! advance counter</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Current_Time</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Reference</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Restart</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Start</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Stop</span> <span class="twilightBlue">! seconds</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Time_Step</span> <span class="twilightBlue">! seconds</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">CalendarString</span> <span class="twilightBlue">! 360_day, gregorian</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_ReferenceString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_RestartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StartString</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">Time_StopString</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Calendar</span>) :: <span class="red">Calendar</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Clock</span>) :: <span class="red">Clock</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Direction_flag</span>) :: <span class="red">Direction</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">CurrentTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">ReferenceTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">RestartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StartTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Time</span>) :: <span class="red">StopTime</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_TimeInterval</span>) :: <span class="red">TimeStep</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Clock</span><br /><br />'''. . .'''<br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''ESM''' coupled state sets. If appropriate, it includes the logic for connecting nested grids. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">LinkedGrid(:,:)</span> <span class="twilightBlue">! connected grid</span><br /><br /> <span class="darkTurquoise">logical</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">DataCoupledSets(:,:)</span> <span class="twilightBlue">! DATA linked sets</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">SetLabel(:)</span> <span class="twilightBlue">! set label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExpLabel(:)</span> <span class="twilightBlue">! export label</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImpLabel(:)</span> ! <span class="twilightBlue">import label</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:,:)</span> <span class="twilightBlue">! export set</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_CplSet</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_CplSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">COUPLED(:)</span></div><br />
<br />
* '''DATA''' model field processing information. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Data</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcycle</span> <span class="twilightBlue">! cycling time coordinate</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lcoord</span> <span class="twilightBlue">! coordinates attribute</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmask</span> <span class="twilightBlue">! land/sea mask</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">Lmulti</span> <span class="twilightBlue">! field across multi-files</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">LastRec</span> <span class="twilightBlue">! processed last record</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Icomp</span> <span class="twilightBlue">! target component index</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ncid</span> <span class="twilightBlue">! NetCDF file ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tid</span> <span class="twilightBlue">! time variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Vid</span> <span class="twilightBlue">! field variable ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nvdim</span> <span class="twilightBlue">! number spatial dimensions</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Zlevel</span> <span class="twilightBlue">! level index to process</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nrec</span> <span class="twilightBlue">! number of time records</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Trec</span> <span class="twilightBlue">! latest read time record</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">LandValue</span> <span class="twilightBlue">! Masking land value</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">SeaValue</span> <span class="twilightBlue">! Masking sea value</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Vsize(:)</span> <span class="twilightBlue">! dimensions size</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! add_offset attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">FillValue</span> <span class="twilightBlue">! _FillValue attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! scale_factor attribute</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Clength</span> <span class="twilightBlue">! time cycling length</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMin</span> <span class="twilightBlue">! grid minimum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LonMax</span> <span class="twilightBlue">! grid maximum longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMin</span> <span class="twilightBlue">! grid minimum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">LatMax</span> <span class="twilightBlue">! grid maximum latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tscale</span> <span class="twilightBlue">! time scale to day</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmono</span> <span class="twilightBlue">! monotonic time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! time minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! time maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! lower time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! upper time-snapshot</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmin</span> <span class="twilightBlue">! variable minimum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>) :: <span class="red">Vmax</span> <span class="twilightBlue">! variable maximum value</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (days)</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /> <span class="darkTurquoise">real</span>(<span class="red">dp</span>) :: <span class="red">Date(6,2)</span> <span class="twilightBlue">! time-snapshots dates YYYY,MM,DD hh:mm:ss.ss</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30), <span class="darkTurquoise">allocatable</span> :: <span class="red">Dname(:)</span> <span class="twilightBlue">! variable dimensions names</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Vcoord(:)</span> <span class="twilightBlue">! variable coordinates names</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=22 ) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=30 ) :: <span class="red">SpecialAction</span> <span class="twilightBlue">! special processing</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tname</span> <span class="twilightBlue">! time variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Tunits</span> <span class="twilightBlue">! time variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vname</span> <span class="twilightBlue">! variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">Vunits</span> <span class="twilightBlue">! variable units</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vdescriptor</span> <span class="twilightBlue">! Variable descriptive name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">Vlongname</span> <span class="twilightBlue">! long_name attribute</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! field longitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! field latitude</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! field land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2d(:,:)</span> <span class="twilightBlue">! time interpolated 2D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3d(:,:,:)</span> <span class="twilightBlue">! time interpolated 3D data</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A2dG(:,:,:)</span> <span class="twilightBlue">! latest 2D data snapsnots</span><br /> <span class="darkTurquoise">real</span>(<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">A3dG(:,:,:,:)</span><span class="twilightBlue">! latest 3D data snapsnots</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Field</span>) :: <span class="red">field</span> <span class="twilightBlue">! field object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>) :: <span class="red">grid</span> <span class="twilightBlue">! field grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> <span class="twilightBlue">! field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Data</span></div><br />
<br />
* '''DATA''' model high-level structure. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_DataSet</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfields</span> <span class="twilightBlue">! number of fields</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Nfiles</span> <span class="twilightBlue">! number of input files</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Ctarget(:)</span> <span class="twilightBlue">! component target</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=20), <span class="darkTurquoise">allocatable</span> :: <span class="red">Field(:)</span> <span class="twilightBlue">! field short-name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Data</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Export(:)</span> <span class="twilightBlue">! Export field structure</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">T_IO</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">IFS(:)</span> <span class="twilightBlue">! Input Files structure</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_DataSet</span></div><br />
<br />
* Import and export fields metadata information. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Field</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">connected</span> <span class="twilightBlue">! connected to coupler</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">debug_write</span> <span class="twilightBlue">! write exchanged field</span><br /> <span class="darkTurquoise">logical</span> :: <span class="red">enable_integral_adj</span> <span class="twilightBlue">! area integral adjusted</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">fid</span> <span class="twilightBlue">! internal field ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! field grid mesh type</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">etype</span> <span class="twilightBlue">! field extrapolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">itype</span> <span class="twilightBlue">! field interpolation flag</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Tindex</span> <span class="twilightBlue">! rolling two-time indices</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=20) :: <span class="red">Ctarget</span> <span class="twilightBlue">! component destination</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=22) :: <span class="red">DateString(2)</span> <span class="twilightBlue">! date-snapshots string</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">short_name</span> <span class="twilightBlue">! short name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">standard_name</span> <span class="twilightBlue">! standard name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">long_name</span> <span class="twilightBlue">! long name</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_gtype</span> <span class="twilightBlue">! DST grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">dst_units</span> <span class="twilightBlue">! DST units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_gtype</span> <span class="twilightBlue">! SRC grid type</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">src_units</span> <span class="twilightBlue">! SRC units</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_vname</span> <span class="twilightBlue">! DATA Vname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">nc_tname</span> <span class="twilightBlue">! DATA Tname</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">RegridMethod</span> <span class="twilightBlue">! regrid method</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExtrapMethod</span> <span class="twilightBlue">! extrapolate</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">scale_factor</span> <span class="twilightBlue">! field scale factor</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">add_offset</span> <span class="twilightBlue">! field add offset value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmin</span> <span class="twilightBlue">! DATA time minimum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tmax</span> <span class="twilightBlue">! DATA time maximum value</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tstr</span> <span class="twilightBlue">! DATA lower time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tend</span> <span class="twilightBlue">! DATA upper time-snapshot</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Tintrp(2)</span> <span class="twilightBlue">! interpolation time (day)</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">Vtime(2)</span> <span class="twilightBlue">! latest two-time values</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_RouteHandle</span>) :: <span class="red">rhandle</span> ! <span class="twilightBlue">field RouteHandle</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Field</span></div><br />
<br />
* Import and export fields mesh data. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Mesh</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gid</span> <span class="twilightBlue">! grid ID</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">gtype</span> <span class="twilightBlue">! grid mesh type</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mask(:,:)</span> <span class="twilightBlue">! grid land/sea mask</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lon(:,:)</span> <span class="twilightBlue">! grid longitude</span><br /> <span class="red">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">lat(:,:)</span> <span class="twilightBlue">! grid latitude</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">area(:,:)</span> <span class="twilightBlue">! grid area</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Mesh</span></div><br />
<br />
* Melding coefficients used to combine fields from '''DATA''' and '''ESM''' components. The weight factors are read from the input NetCDF specified in the <span class="blue">WeightsFile(atmos)</span> keyword. The user has full control of how the merging is done. It is recommended to provide a gradual transition between the two components. <br /><br /> Recall that the '''DATA''' component supplies needed data to a particular '''ESM''' component. For example, it may export data to the atmosphere model at locations not covered by the other '''ESM''' components because of smaller grid coverage. If the atmosphere and ocean model grids are incongruent, the atmosphere component needs to import sea surface temperature (SST) on those grid points not covered by the ocean component. Thus, the weighting coefficients are used to merge the SST data: <div class="code"> <span class="red">SST_atm(:,:)</span> = <span class="red">Cesm(:,:)</span> * <span class="red">SST_esm(:,:)</span> + <span class="red">Cdat(:,:)</span> * <span class="red">SST_dat(:,:)</span></div> where<span class="red"> Cesm(:,:)</span> + <span class="red">Cdat(:,:)</span> = <span class="red">1</span>.<br /><div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">NestedGrid</span> <span class="twilightBlue">! grid needing merged field</span><br /><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameDATA</span> <span class="twilightBlue">! DATA weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=100) :: <span class="red">VnameESM</span> <span class="twilightBlue">! ESM weights variable name</span><br /> <span class="darkTurquoise">character</span>(<span class="darkTurquoise">len</span>=256) :: <span class="red">ncfile</span> <span class="twilightBlue">! Weights NetCDF filename</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="forestGreen">allocatable</span> :: <span class="red">Cdat(:,:)</span> <span class="twilightBlue">! coefficients for DATA</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cesm(:,:)</span> <span class="twilightBlue">! coefficients for ESM</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Meld</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Meld</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">WEIGHTS(:)</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightDAT</span> = <span class="red">0.0_dp</span> <span class="twilightBlue">! DATA component weight</span><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">WeightESM</span> = <span class="red">1.0_dp</span> <span class="twilightBlue">! ESM component weight</span></div><br />
<br />
* Coupled models high-level object, [<span class="red">Nmodels=5</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> :: <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active for coupling</span><br /><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">LandValue</span> <span class="twilightBlue">! land mask value</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>) :: <span class="red">SeaValue</span> <span class="twilightBlue">! sea mask value</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">Ngrids</span> <span class="twilightBlue">! number nested grids</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ExportCalls</span> <span class="twilightBlue">! export CALL counter</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">ImportCalls</span> <span class="twilightBlue">! import CALL counter</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number model PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! component PETs list</span><br /><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">TimeFrac(:,:)</span> <span class="twilightBlue">! driver time fraction</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=:), <span class="darkTurquoise">allocatable</span> :: <span class="red">name</span> <span class="twilightBlue">! component name</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_Grid</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">grid(:)</span> <span class="twilightBlue">! grid object</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Mesh</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">mesh(:)</span> <span class="twilightBlue">! mesh</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportField(:)</span> <span class="twilightBlue">! import fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Field</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportField(:)</span> <span class="twilightBlue">! export fields</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ImportState(:)</span> <span class="twilightBlue">! import state</span><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESMF_State</span>), <span class="darkTurquoise">allocatable</span> :: <span class="red">ExportState(:)</span> <span class="twilightBlue">! export state</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Model</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Model</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">MODELS(:)</span></div><br />
<br />
* Coupling models connector used for the interpolation/extrapolaton between source and destination fields, [<span class="red">Nmodels</span>, <span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">logical</span> :: <span class="red">IsActive</span> <span class="twilightBlue">! active connector</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">divDT</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">MaskInteraction</span> <span class="twilightBlue">! connector mask interaction</span><br /><br /> <span class="darkTurquoise">integer</span> :: <span class="red">nPETs</span> <span class="twilightBlue">! number of connector PETs</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">PETlist(:)</span> <span class="twilightBlue">! connector PETs list</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=100) :: <span class="red">name</span> <span class="twilightBlue">! connector name</span><br /><br /> <span class="darkTurquoise">END TYPE</span> <span class="forestGreen">ESM_Conn</span><br /><br /> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Conn</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">CONNECTORS(:,:)</span></div><br />
<br />
* Define '''DATA''' Model field processing information from input data files, [<span class="red">1</span>:<span class="red">Nmodels</span>]. Currently, the '''DATA''' Model only export fields and it only supports input NetCDF files. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_DataSet</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">DataSet(:)</span></div><br />
<br />
* '''ESM''' clock for driver (zeroth element) and coupled components, [<span class="red">0</span>:<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">TYPE</span> (<span class="forestGreen">ESM_Clock</span>), <span class="darkTurquoise">allocatable</span>, <span class="darkTurquoise">target</span> :: <span class="red">ClockInfo(:)</span></div><br />
<br />
* '''Internal module parameters and variables:'''<br /> <br />
** Number of coupled '''ESM''' gridded components. Currently, five '''ESM''' components are supported ('''ROMS''', '''DATA''', '''ATMOSPHERE''', '''SEA-ICE''', and '''WAVE''' model types). <br /> All supported components are accounted here even if we are running an application with less number of models. The <span class="red">IsActive</span> switches are use to operate only on the desired coupled components. This is done to have complete infornation in the above structures. The gridded arrays are never allocated if a particular component is not active. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Nmodels</span> = <span class="red">5</span></div><br />
** Number of nested grids. All are initialized to just one grid; its values are overwritten during processing. An additional variable '''NgridsR''' is created to avoid using ROMS module <span class="blue">mod_param</span> in the generic interface. Both '''Ngrids''' and '''NgridsR''' will have the same value.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsA</span> = <span class="red">1</span> <span class="twilightBlue">! Atmosphere Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsD</span> = <span class="red">1</span> <span class="twilightBlue">! DATA Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsI</span> = <span class="red">1</span> <span class="twilightBlue">! Sea-ice Model</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsR</span> = <span class="red">1</span> <span class="twilightBlue">! ROMS</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">NgridsW</span> = <span class="red">1</span> <span class="twilightBlue">! Wave Model</span></div><br />
** Coupled '''ESM''' components identification indices.<div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idriver</span> = <span class="red">0</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iroms</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iatmos</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Idata</span> = <span class="red">3</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseaice</span> = <span class="red">4</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iwave</span> = <span class="red">5</span></div><br />
** Generic '''ESM''' component labels used in the <span class="darkTurquoise">CASE</span> constructs. We cannot use the identification indices because the vector <span class="red">Iroms(:)</span> cannot be defined as a parameter and a non constant expression is illegal: <span class="darkTurquoise">CASE</span> ( <span class="red">Iroms</span>(<span class="red">1</span>) : <span class="red">Iroms</span>(<span class="red">NgridsR</span>) )<div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 3), <span class="darkTurquoise">allocatable</span> :: <span class="red">Clabel(:)</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len=</span>10), <span class="darkTurquoise">allocatable</span> :: <span class="red">Cmodel(:)</span></div><br />
** Number of ROMS export and import fields per component. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nexport(:)</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">Nimport(:)</span></div><br />
** Model coupling type: [<span class="red">1</span>] Explicit, [<span class="red">otherwise</span>] Semi-Implicit. In <span class="red">explicit coupling</span>, exchange fields at the next time-step are defined using known values from the time-step before it. Explicit methods require less computational effort and are accurate for small coupling time-steps. In <span class="red">implicit coupling</span>, exchange fields at the next time-step are defined by including values at the next time-step. Implicit methods are stable and allow longer coupling time-steps but are more computationally expensive. In <span class="red">semi-implicit coupling</span>, '''ROMS -> ATM''' is explicit, '''ATM -> ROMS''' is implicit. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">CouplingType</span> = <span class="red">1</span></div><br />
** Distributed-memory communicator handle for each component, Total number of PETs needed in concurrent PET layout, rank of each PET, and PET layout (sequential or concurrent). <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">allocatable</span> :: <span class="red">ESMcomm(:)</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">sumPETs</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">PETrank</span><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=10) :: <span class="red">PETlayoutOption</span></div><br />
** Driver clock parameters specified in configuration script. A integer vector with six elements: (<span class="red">1</span>) year including century, like 2017; (<span class="red">2</span>) month of the year, 1 to 12; (<span class="red">3</span>) day of the month; (<span class="red">4</span>) hour of the day, 0 to 23; (<span class="red">5</span>) minutes of the hour, 0 to 59; and (<span class="red">6</span>) seconds of the minute, 0 to 59 <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ReferenceDate</span>(6) <span class="twilightBlue">! reference date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">RestartDate</span>(6) <span class="twilightBlue">! restarting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StartDate</span>(6) <span class="twilightBlue">! starting date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">StopDate</span>(6) <span class="twilightBlue">! stopping date</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">TimeStep</span>(6) <span class="twilightBlue">! coupling interval</span></div><br />
** Today date string. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=44) :: <span class="red">TodayDateString</span></div><br />
** '''ESM''' coupling simulation reference date number, element: (<span class="red">1</span>) seconds, and (<span class="red">2</span>) fractional days<div class="code"> <span class="darkTurquoise">real</span> (<span class="red">dp</span>) :: <span class="red">ReferenceDateNumber</span>(2)</div><br />
** '''DATA''' component parallel distributed-memory domain partions in the '''I'''- and '''J'''-directions (lon,lat). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">ItileD</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">JtileD</span></div><br />
** Coupling debugging flag: [<span class="red">0</span>] no debugging, [<span class="red">1</span>] reports informative messages, or [<span class="red">2</span>] <span class="red">'1'</span> plus writes grid information in '''VTK''' format, [<span class="red">3</span>] <span class="red">'2'</span> plus writes exchage fields into '''NetCDF''' files. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">DebugLevel</span> = <span class="red">0</span></div><br />
** Execution tracing level flag: [<span class="red">0</span>] no tracing, [<span class="red">1</span>] reports sequence of coupling subroutine calls, or [<span class="red">2</span>] <<span class="red">1</span>> plus writes voluminous '''ESMF''' library tracing information which slowdown performace, and creates large log file. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">TraceLevel</span> = <span class="red">0</span></div><br />
** Switch to trace/track run sequence during debugging. All information is written to Fortan unit <span class="red">trac</span>. For now, use standard output unit. <div class="code"> <span class="darkTurquoise">logical</span> :: <span class="red">ESM_track</span> = <span class="red">.FALSE.</span> <span class="twilightBlue">! trace/track CALL sequence switch</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">trac</span> = <span class="red">6</span> <span class="twilightBlue">! trace/track CALL sequence unit</span></div><br />
** Coupled model staggered grid-cell type indices. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inan</span> = <span class="red">0</span> <span class="twilightBlue">! unstaggered, cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icenter</span> = <span class="red">1</span> <span class="twilightBlue">! cell center</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Icorner</span> = <span class="red">2</span> <span class="twilightBlue">! cell corners</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iupoint</span> = <span class="red">3</span> <span class="twilightBlue">! right and left cell faces</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ivpoint</span> = <span class="red">4</span> <span class="twilightBlue">! upper and lower cell faces</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=6), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">GridType</span> = &<br /> & (/ '<span class="red">N/A</span> ', &<br /> & '<span class="red">Center</span>', &<br /> & '<span class="red">Corner</span>', &<br /> & '<span class="red">U</span> ', &<br /> & '<span class="red">V</span> ' /)</div><br />
** '''REGRID''' interpolation method between source and destination fields. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Inone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ibilin</span> = <span class="red">1</span> <span class="twilightBlue">! bilinear</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipatch</span> = <span class="red">2</span> <span class="twilightBlue">! high-order patch recovery</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv1</span> = <span class="red">3</span> <span class="twilightBlue">! first-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iconsv2</span> = <span class="red">4</span> <span class="twilightBlue">! second-order conservative</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InStoD</span> = <span class="red">5</span> <span class="twilightBlue">! nearest neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">InDtoS</span> = <span class="red">6</span> <span class="twilightBlue">! nearest neighbor Dst to Src</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:6) :: <span class="red">IntrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">BLIN</span>', &<br /> & '<span class="red">PTCH</span>', &<br /> & '<span class="red">CNS1</span>', &<br /> & '<span class="red">CNS2</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">ND2S</span>' /)</div><br />
** Extrapolation method for unmapped destination points. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Enone</span> = <span class="red">0</span> <span class="twilightBlue">! none</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">ExStoD</span> = <span class="red">1</span> <span class="twilightBlue">! nearear neighbor Src to Dst</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Eidavg</span> = <span class="red">2</span> <span class="twilightBlue">! inverse distance average</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ecreep</span> = <span class="red">3</span> <span class="twilightBlue">! creep fill</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">E2steps</span> = <span class="red">4</span> <span class="twilightBlue">! Turuncoglu two steps</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=4), <span class="darkTurquoise">dimension</span>(0:4) :: <span class="red">ExtrpType</span> = &<br /> & (/ '<span class="red">NONE</span>', &<br /> & '<span class="red">NS2D</span>', &<br /> & '<span class="red">IDAV</span>', &<br /> & '<span class="red">CREE</span>', &<br /> & '<span class="red">2STP</span>' /)</div><br />
** The number of levels to output for the extrapolation methods that fill levels, like creep fill (<span class="red">ESMF_EXTRAPMETHOD_CREEP</span>). Unmapped destination points are supplied by repeatedly moving data from mapped locations to neighboring unmapped locations for a user-specified number of levels. For each creeped point, its value is the average of the values of the immediate neighbors from the mapped points from regridding ('''ESMF''' Reference Manual, v '''8.0.0'''). <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">extrapNumLevels</span> = <span class="red">1</span></div><br />
** Interpolation connectors mask interaction flags. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverLand</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverOcean</span> = <span class="red">2</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">OverAll</span> = <span class="red">3</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=3), <span class="darkTurquoise">dimension</span>(3) :: <span class="red">MaskType</span> = &<br /> & (/ '<span class="red">LND</span>', &<br /> & '<span class="red">OCN</span>', &<br /> & '<span class="red">ALL</span>'/)</div><br />
** Coupling run mode: sequential or concurrent. <div class="code"> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Iseq</span> = <span class="red">1</span><br /> <span class="darkTurquoise">integer</span>, <span class="darkTurquoise">parameter</span> :: <span class="red">Ipar</span> = <span class="red">2</span><br /><br /> <span class="darkTurquoise">character</span> (len=10), <span class="darkTurquoise">dimension</span>(2) :: <span class="red">RunMode</span> = &<br /> & (/ '<span class="red">SEQUENTIA</span>L', &<br /> & '<span class="red">CONCURRENT</span>' /)</div><br />
** Compling standard input parameters filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CinpName</span></div><br />
** '''ESM''' free-format run sequence configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256) :: <span class="red">CONFname</span></div><br />
** ROMS coupling '''YAML''' configuration filename. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">CPLname</span></div><br />
** Standard input filename for each coupled model, [<span class="red">Nmodels</span>]. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=256), <span class="darkTurquoise">allocatable</span> :: <span class="red">INPname(:)</span></div><br />
** Standard output units and log messages filename for coupler and '''ESMF''' library. <div class="code"> <span class="darkTurquoise">integer</span> :: <span class="red">cplout</span> = <span class="red">77</span> <span class="twilightBlue">! coupling driver</span><br /> <span class="darkTurquoise">integer</span> :: <span class="red">dataout</span> = <span class="red">77</span> <span class="twilightBlue">! data component</span><br /><br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">ESMnameLog</span> = '<span class="red">log.esmf</span>'<br /> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>= 8), <span class="darkTurquoise">parameter</span> :: <span class="red">CouplerLog</span> = '<span class="red">log.coupler</span>'</div><br />
** Output '''NetCDF''' file used to store field snapshot attributes needed for time interpolation by the '''ESM''' component kernel during concurrent coupling. <div class="code"> <span class="darkTurquoise">character</span> (<span class="darkTurquoise">len</span>=17), <span class="darkTurquoise">parameter</span> :: <span class="red">AttFileName</span> = '<span class="red">time_intrp_att.nc</span>'</div><br />
** '''ESM''' single and double precision constants. <div class="code"> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MAPPED_MASK</span> = <span class="red">99_i4b</span><br /> <span class="darkTurquoise">integer</span> (<span class="red">i4b</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">UNMAPPED_MASK</span> = <span class="red">98_i4b</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_dp</span> = <span class="red">1.0E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r4</span> = <span class="red">1.0E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">MISSING_r8</span> = <span class="red">1.0E20_r8</span><br /><br /> <span class="darkTurquoise">real</span> (<span class="red">dp</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_dp</span> = <span class="red">0.5E20_dp</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r4</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r4</span> = <span class="red">0.5E20_r4</span><br /> <span class="darkTurquoise">real</span> (<span class="red">r8</span>), <span class="darkTurquoise">parameter</span> :: <span class="red">TOL_r8</span> = <span class="red">0.5E20_r8</span></div><br />
<br />
==Capabilities==<br />
The ROMS <span class="red">cap</span> module contains a set of subroutines that are required by '''NUOPC'''. According to a predefined calling sequence, these subroutines are called by the '''NUOPC''' infrastructure. Some subroutines are called during the initialization of the coupled system, some during the run of the coupled system, and some during the finalization of the coupled system.<br />
<br />
The initialization sequence is the most complex and is governed by the '''NUOPC''' technical rules. Details about the initialization sequence can be found in the '''NUOPC''' Reference Manual. The ROMS <span class="red">cap</span> requires '''ESMF version 8 or higher'''.<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" width="1000"<br />
|valign="top"|<span class="blue">ROMS_SetServices</span> <br />
|Entry point to the ROMS <span class="red">cap</span> and the only <span class="red">public</span> routine. It sets the ROMS component chared-object entry points for using '''NUOPC''' generic methods for '''initialize''', '''run''', and '''finalize'''.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP1</span><br />
|ROMS component phase 1 initialization which sets import and export fields long and short names into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetInitializeP2</span><br />
|ROMS component phase 2 initialization which initializes the ROMS component (<span class="blue">ROMS_initialize</span>), sets component grid (<span class="blue">ROMS_SetGridArrays</span>), and adds fields into import and export into respective states.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_DataInit</span><br />
|Exports ROMS component fields during initialization or restart.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetClock</span><br />
|Sets ROMS component date calendar, start and stop times, and coupling interval.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetRunClock</span><br />
|Sets ROMS run clock manually to avoid getting zero time stamps at the first regridding call.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_CheckImport</span><br />
|Checks if ROMS component import field is at the correct time.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetGridArrays</span><br />
|Sets ROMS component staggered, horizontal grid arrays, grid area, and land/sea mask if any.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetStates</span><br />
|Adds ROMS component <span class="red">export</span> and <span class="red">import</span> fields into its respective state.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_ModelAdvance</span><br />
|Advances ROMS component for a coupling interval. It calls <span class="blue">ROMS_Import</span> and <span class="blue">ROMS_Export</span> routines.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_SetFinalize</span><br />
|Finalizes ROMS component execution.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Import</span><br />
|Loads import fields into ROMS internal kernel arrays.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Export</span><br />
|Exports ROMS fields to other gridded components.<br />
|-<br />
|valign="top"|<span class="blue">ROMS_Rotate</span><br />
|Rotates vector components from computational grid to geographical EAST and NORTH directions or vice versa.<br />
|-<br />
|}<br />
<br />
<br />
Other <span class="red">NUOPC</span> cap modules are available within the ROMS native framework:<br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | '''ESMF/NUOPC Modules'''<br />
| align="center" | '''Tested'''<br />
|style="width: 700px;" align="center" | '''Description'''<br />
|-<br />
| <span class="red">Master/esmf_driver.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''ESMF/NUOPC''' Master Driver<br />
|-<br />
| <span class="red">Master/esmf_esm.F</span><br />
| <span class="forestGreen">yes</span><br />
| Sets '''ESMF/NUOPC''' '''Services''' and '''RunSequence''' for each coupled component<br />
|-<br />
| <span class="red">Master/esmf_coupler.h</span><br />
| <span class="forestGreen">yes</span><br />
| '''Computes''', '''Execute''', and '''Release''' the '''Connectors''' between <span class="violet">source</span> and <span class="violet">destination</span> fields<br />
|-<br />
| <span class="red">Master/mod_esmf_esm.h</span><br />
| <span class="forestGreen">yes</span><br />
| Defines framework object '''structures''' and '''variables''' and includes support routines<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_coamps.h</span><br />
| <span class="forestGreen">yes</span><br />
| Coupled Ocean-Atmosphere Mesoscale Prediction System ('''COAMPS''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_regcm.h</span><br />
| <span class="red">no</span><br />
| ICTP Regional Climate Model ('''RegCM''', Version '''4.5''')<br />
|-<br />
| <span class="blue">Master/esmf_atm_wrf.h</span><br />
| <span class="forestGreen">yes</span><br />
| Weather Research and Forecasting model ('''WRF''')<br />
|-<br />
| <span class="blue">Master/esmf_data.F</span><br />
| <span class="forestGreen">yes</span><br />
| Generic '''DATA''' component via '''NetCDF''' files<br />
|-<br />
| <span class="blue">Master/esmf_ice_cice.h</span><br />
| <span class="red">no</span><br />
| Los Alamos Sea Ice Model ('''CICE''')<br />
|-<br />
| <span class="blue">Master/esmf_roms.F</span><br />
| <span class="forestGreen">yes</span><br />
| Regional Ocean Modeling System ('''ROMS''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_wam.h</span><br />
| <span class="red">no</span><br />
| ECMWF's Wave Model ('''WAM''')<br />
|-<br />
| <span class="blue">Master/esmf_wav_ww3.h</span><br />
| <span class="red">no</span><br />
| WaveWatch III ('''WW3'''), <span class="twilightBlue">under development</span><br />
|}<br />
<div style="clear: both"></div><br />
<br />
==Import and Export Fields==<br />
<br />
The <span class="red">export</span> and <span class="red">import</span> fields are configured and specified in the '''YAML''' file [[coupling_esmf_atm.yaml]]. Check the following [[coupling_esmf_atm.yaml|wikiROMS]] page for detailed information. The user has full control of the coupling exchange fields metadata.<br />
<br />
The '''ESMF/NUOPC''' library version '''8.0.0''' or higher supports native nesting capabilities. The '''NUOPC''' layer now allows coupling sets in the <span class="red">import</span> and <span class="red">export</span> states of the '''ESM''' components. A nested grid is represented as another '''ESMF_Grid''' object so that a coupled model can have a set of telescoping meshes of decreasing spatial resolution. The '''NUOPC''' connector can recognize the different grids and exchange fields to a connected '''ESM''' component. Various types of connections are possible, like nest-to-nest, fine-to-coarse, or coarse-to-fine. The user decides which nested grids to connect. The <span class="blue">NUOPC_AddNestedState</span> routine is used to advertise the nested couple sets.<br />
<br />
For example, the ROMS '''NOUPC''' <span class="red">cap</span> module routine <span class="blue">ROMS_SetInitializeP1</span> illustrates how easily the coupled sets is added to the <span class="red">ImportState</span> and <span class="red">ExportState</span> as shown below in the green blocks of code. <br />
<br />
<br />
{|cellpadding="3"<br />
|-<br />
|[[Image:ImportCoupledStates.png]]<br />
|[[Image:ExportCoupledStates.png]]<br />
|}</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6532Regional Ocean Modeling System (ROMS)2022-05-19T13:21:40Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first assignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation, tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Moreover, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
To be continued ...<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
* <span id="DiLorenzo_2000a"></span>Di Lorenzo, E., A. M. Moore, H. G. Arango, B. D. Cornuelle, A. J. Miller, B. Powell, B. S. Chua, and A. F. Bennett, 2007: Weak and strong constraint data assimilation in the inverse Regional Ocean Modeling System (ROMS): Development and application for a baroclinic coastal upwelling system, ''Ocean Modelling'', '''16''', doi:10.1016/j.ocemod.2006.08.002. <br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="MooreAM_2021a"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id="MooreAM_2021b"></span>Moore, A.M., J. Levin, and H.G. Arango, and J. Wilkin, 2021: Assessing the performance of an ocean observing, analysis and forecast System for the Mid-Atlantic Bight using array modes, ''Ocean Modelling'', '''164''', doi:10.1016/j.ocemod.2021.101821<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6531Regional Ocean Modeling System (ROMS)2022-05-18T15:39:09Z<p>Arango: /* ROMS Selected References */</p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first assignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation, tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Moreover, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
* <span id="DiLorenzo_2000a"></span>Di Lorenzo, E., A. M. Moore, H. G. Arango, B. D. Cornuelle, A. J. Miller, B. Powell, B. S. Chua, and A. F. Bennett, 2007: Weak and strong constraint data assimilation in the inverse Regional Ocean Modeling System (ROMS): Development and application for a baroclinic coastal upwelling system, ''Ocean Modelling'', '''16''', doi:10.1016/j.ocemod.2006.08.002. <br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="MooreAM_2021a"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id="MooreAM_2021b"></span>Moore, A.M., J. Levin, and H.G. Arango, and J. Wilkin, 2021: Assessing the performance of an ocean observing, analysis and forecast System for the Mid-Atlantic Bight using array modes, ''Ocean Modelling'', '''164''', doi:10.1016/j.ocemod.2021.101821<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6530Regional Ocean Modeling System (ROMS)2022-05-18T13:18:48Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first assignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation, tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Moreover, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6529Regional Ocean Modeling System (ROMS)2022-05-18T02:35:50Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first ssignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation, tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Moreover, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6528Regional Ocean Modeling System (ROMS)2022-05-18T02:09:37Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first ssignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation, tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Thus, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6527Regional Ocean Modeling System (ROMS)2022-05-18T02:04:54Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first ssignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
[[Image:AdjointGroup.jpg|1000px|center]]<br />
<center>'''''ROMS Adjoint Group:''''' Back, left to right: Arthur J. Miller, Andrew M. Moore, Bruce D. Cornuelle, and Douglas J. Neilson. Front: Hernan G. Arango and Emanuel Di Lorenzo. </center><br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long perturbation tangent linear model ('''TLM''') and adjoint model ('''ADM''') code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Thus, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm. The finite-amplitude, tangent linear code for the Representer model ('''RPM''') was later derived from the '''TLM'''. The '''RPM''' is used in the indirect, representer-based approach for 4-Dimensional Variational ('''4D-Var''') data assimilation.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=File:AdjointGroup.jpg&diff=6526File:AdjointGroup.jpg2022-05-18T01:44:19Z<p>Arango: ROMS Original Adjoint Group</p>
<hr />
<div>== Summary ==<br />
ROMS Original Adjoint Group</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6525Regional Ocean Modeling System (ROMS)2022-05-18T01:42:43Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first ssignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (hereafter, <span class="blue">JCM</span>) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, to meet at University of California, Los Angeles (UCLA) with <span class="blue">JCM</span> and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
In 2000, Terri Paluszkiewicz from the Office of Naval Research (ONR) approached <span class="blue">HGA</span> and <span class="blue">JCM</span> about funding opportunities to develop the next generation of a regional, high-resolution ocean model with accurate numerical algorithms and data assimilation capabilities. As a result, ONR started funding '''ROMS''' development separately to Rutgers (data assimilation and adjoint-based algorithms, applications, maintenance, and community outreach) and UCLA (numerical algorithms and applications). Then, <span class="blue">HGA</span> suggested to ONR that a redesign of '''ROMS''' was required, and a team of experts was needed to do variational data assimilation correctly. During his years at Harvard, <span class="blue">HGA</span> had the opportunity to meet Andrew M. Moore (hereafter, <span class="blue">AMM</span>), now at the University of California, Santa Cruz (UCSC), and work with Arthur J. Miller (hereafter, <span class="blue">AJM</span>) from the Scripps Institute of Oceanography (SIO). <span class="blue">AMM</span> is an expert on adjoint-based modeling and applications. Therefore, the <span class="blue">ROMS</span> data assimilation team included <span class="blue">AMM</span>, <span class="blue">HGA</span>, <span class="blue">AJM</span>, collaborator Bruce D. Cornuelle (hereafter, <span class="blue">BDC</span>), and Ph. D. student Emanuele Di Lorenzo (hereafter, <span class="blue">EDL</span>), and Douglas J. Neilson (Hereafter, <span class="blue">DJW</span>). ONR also funded UCSC and SIO separately for this work. Thus, it started a very successful collaboration with several peer-reviewed publications in journals and books. The chemistry between the '''ROMS''' Data Assimilation ('''ROMS-DA''') group was extraordinary, with lots of fun, good food, and great wine-tasting parties.<br />
<br />
<span class="blue">AMM</span> and <span class="blue">HGA</span> frequently traveled to Scripps, in La Jolla, for an intense week-long tangent linear model ('''TLM''') and adjoint model (<span class="blue">ADM</span>) code writing. <span class="blue">AMM</span> was the master teaching and directing the monkies on how to write '''ROMS-TL''' and '''ROMS-AD''' codes. The '''TLM''' and '''ADM''' were hand-coded from the discrete nonlinear ('''NLM''') code using the recipes of Giering and Kaminski (1998). Therefore, any change to its dynamical and numerical kernels will affect the symmetry of the '''TLM''' and '''ADM''' operators. Thus, the resulting discrete adjoint is exact and is defined relative to the inner product that prescribes the '''L2'''-norm.<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arangohttps://www.myroms.org/wiki/index.php?title=Regional_Ocean_Modeling_System_(ROMS)&diff=6524Regional Ocean Modeling System (ROMS)2022-05-17T19:59:54Z<p>Arango: </p>
<hr />
<div><div class="title">ROMS History</div><br />
In the fall of 1995, Hernan G. Arango (hereafter <span class="blue">HGA</span>) started working at the Ocean Modeling Lab ('''OML'''), Institute of Marine and Coastal Sciences ('''IMCS'''), Rutgers University. His first ssignment from '''OML''''s group leader Dale B. Haigvogel (hereafter <span class="blue">DBH</span>) was to work on the classic baroclinic, wind-driven, double gyre problem with the now-extinct '''S'''-'''C'''oordinate '''R'''utgers '''U'''niversity '''M'''odel ('''SCRUM''', Song and Haidvogel, 1994), which was derived from '''SPEM''' ('''S'''emi-spectral '''P'''rimitive '''E'''quation '''M'''odel; Haidvogel ''et al.'', 1991). The double-gyre application is affectionately known as '''Big Bad Basin''' (nowadays, <span class="mediumOrchid">BASIN</span> test case) by several '''OML''' modelers because of its stability issues in long simulations. After a couple of months of struggling with the <span class="mediumOrchid">BASIN</span> problem, <span class="blue">HGA</span> decided to rewrite '''SCRUM''' from scratch and modify its numerical kernel, including time-stepping, advection schemes, and implementing a split-explicit treatment coupling between barotropic (fast) and barotropic (slow) governing equations in collaboration with '''OML''''s scientists Robert J. Chant and Katherine S. Hedström. In addition, a complete overhaul of the model metadata, input, and output files with NetCDF format. And voilà, the <span class="mediumOrchid">BASIN</span> application ran successfully for several years. The new model became '''SCRUM''' version '''3.0''', which later evolved to '''SCRUM 4.0''' and then becomes the '''R'''egional '''O'''cean '''M'''odeling '''S'''ystem ('''ROMS'''). The acronym '''ROMS''' was inspired by the '''R'''egional '''A'''tmospheric '''M'''odeling '''S'''ystem ('''RAMS'''). Thus, at the time, there was a regional numerical model for the atmosphere and ocean with similar acronyms.<br />
<br />
In 1998, James C. McWilliams (UCLA) started looking for a regional ocean model for the U.S. West Coast and contacted <span class="blue">DBH</span> for possible collaborations. As a result, <span class="blue">DBH</span> and <span class="blue">HGA</span> traveled to Los Angeles, CA, to meet at UCLA with Jim McWilliams and Alexander F. Shchepetkin (hereafter <span class="blue">AFS</span>) or just Sasha, as the '''ROMS''' VIPs know him. The expectation that <span class="blue">HGA</span> and <span class="blue">AFS</span> would get along was very low. However, their mutual respect, skills, and friendship grew. As a result, and to the benefit of the ocean modeling community, '''ROMS''' was officially born. Their first task was to improve its efficiency by implementing a multi-threaded parallel version of its numerical kernel using the shared-memory paradigm native to the IRIX operating system available on SGI supercomputers of the time. The shared-memory compiling directives evolved later to the portable '''OpenMP''' standard implementation in '''ROMS'''. Moreover, <span class="blue">AFS</span> began looking at the ensemble code and started counting operations; multiplications replaced unnecessary divisions when appropriate, and '''DO'''-loops were reorganized and pipelined to reduce memory hits. <span class="blue">HGA</span> learned how to count operations and analyzed optimized Fortran code, which was available at the time, to minimize the need for automatic optimization. As a result, the '''ROMS''' code was very efficient and fast. Nowadays, '''ROMS''' can be compiled with aggressive Fortran optimization flags. Of course, users need to be aware that the bold optimization flags in some compilers could compromise the numerical '''IEEE''' floating-point precision, and new flags are required to retain it, which may slow down the computations.<br />
<br />
<br />
===ROMS Releases===<br />
<br />
The following table shows information about the infrequent tagged '''ROMS''' releases showing its evolution and expansion. It was redesigned completely starting with version '''ROMS 2.0''' and written in '''Fortran-1990''' standard with modules with the '''derived-type''' objects with '''dynamical allocation''', and managed with '''svn''' repositories instead of '''cvs'''. Some features of '''Fortran-1995''' and '''Fortran-2003''' standards were introduced later when available in mostly all compilers. The perturbation Tangent Linear Model ('''TLM'''), finite-amplitude Tangent Linear (Representer) Model ('''RPM'''), and adjoint model ('''ADM''') were released in version '''ROMS 2.3'''. <br />
<br />
<br />
:{| border="1" cellspacing="0" cellpadding="5" align="left"<br />
| align="center" | <span class="violet">Tag Version</span><br />
| align="center" | <span class="violet">svn Revision</span><br />
| align="center" | <span class="violet">Release Date</span><br />
| align="center" | <span class="violet">Number of Files</span><br />
| align="center" | <span class="violet">Lines of Code</span><br />
|-<br />
| align="center" | '''1.1.0'''<br />
| align="center" | '''cvs -r 1.1.0'''<br />
| align="center" | '''1999-01-20'''<br />
| align="center" | '''125'''<br />
| align="center" | '''44,639'''<br />
|-<br />
| align="center" | '''1.2.0'''<br />
| align="center" | '''cvs -r 1.2.0'''<br />
| align="center" | '''1999-01-25'''<br />
| align="center" | '''130'''<br />
| align="center" | '''46,905'''<br />
|-<br />
| align="center" | '''1.3.0'''<br />
| align="center" | '''cvs -r 1.3.0'''<br />
| align="center" | '''1999-03-15'''<br />
| align="center" | '''131'''<br />
| align="center" | '''47,931'''<br />
|-<br />
| align="center" | '''1.4.0'''<br />
| align="center" | '''cvs -r 1.4.0'''<br />
| align="center" | '''1999-06-20'''<br />
| align="center" | '''146'''<br />
| align="center" | '''49,466'''<br />
|-<br />
| align="center" | '''1.5.0'''<br />
| align="center" | '''cvs -r 1.5.0'''<br />
| align="center" | '''2000-02-10'''<br />
| align="center" | '''145'''<br />
| align="center" | '''56,118'''<br />
|-<br />
| align="center" | '''1.6.0'''<br />
| align="center" | '''cvs -r 1.6.0'''<br />
| align="center" | '''2000-01-29'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,066'''<br />
|-<br />
| align="center" | '''1.6.1'''<br />
| align="center" | '''cvs -r 1.6.1'''<br />
| align="center" | '''2000-03-06'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,123'''<br />
|-<br />
| align="center" | '''1.6.2'''<br />
| align="center" | '''cvs -r 1.6.2'''<br />
| align="center" | '''2000-03-17'''<br />
| align="center" | '''144'''<br />
| align="center" | '''56,283'''<br />
|-<br />
| align="center" | '''1.7.0'''<br />
| align="center" | '''cvs -r 1.7.0'''<br />
| align="center" | '''2001-01-19'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,652'''<br />
|-<br />
| align="center" | '''1.7.1'''<br />
| align="center" | '''cvs -r 1.7.1'''<br />
| align="center" | '''2001-02-01'''<br />
| align="center" | '''165'''<br />
| align="center" | '''63,573'''<br />
|-<br />
| align="center" | '''1.7.2'''<br />
| align="center" | '''cvs -r 1.7.2'''<br />
| align="center" | '''2001-04-17'''<br />
| align="center" | '''173'''<br />
| align="center" | '''66,324'''<br />
|-<br />
| align="center" | '''1.8.0'''<br />
| align="center" | '''cvs -r 1.8.0'''<br />
| align="center" | '''2002-01-11'''<br />
| align="center" | '''175'''<br />
| align="center" | '''67,999'''<br />
|- style="background-color: #D0FFD0;"<br />
| align="center" | '''2.0'''<br />
| align="center" | '''2'''<br />
| align="center" | '''2003-06-30'''<br />
| align="center" | '''232'''<br />
| align="center" | '''95,836'''<br />
|-<br />
| align="center" | '''2.1'''<br />
| align="center" | '''3'''<br />
| align="center" | '''2004-05-05'''<br />
| align="center" | '''253'''<br />
| align="center" | '''117,956'''<br />
|-<br />
| align="center" | '''2.2'''<br />
| align="center" | '''4'''<br />
| align="center" | '''2005-05-26'''<br />
| align="center" | '''297'''<br />
| align="center" | '''158,176'''<br />
|-<br />
| align="center" | '''2.3'''<br />
| align="center" | '''8'''<br />
| align="center" | '''2007-02-06'''<br />
| align="center" | '''1,122'''<br />
| align="center" | '''399,457'''<br />
|-<br />
| align="center" | '''3.0'''<br />
| align="center" | '''36'''<br />
| align="center" | '''2007-04-27'''<br />
| align="center" | '''1,313'''<br />
| align="center" | '''460,643'''<br />
|-<br />
| align="center" | '''3.1'''<br />
| align="center" | '''166'''<br />
| align="center" | '''2008-03-17'''<br />
| align="center" | '''1,345'''<br />
| align="center" | '''470,110'''<br />
|-<br />
| align="center" | '''3.2'''<br />
| align="center" | '''338'''<br />
| align="center" | '''2009-03-25'''<br />
| align="center" | '''1,428'''<br />
| align="center" | '''549,318'''<br />
|-<br />
| align="center" | '''3.3'''<br />
| align="center" | '''432'''<br />
| align="center" | '''2009-12-26'''<br />
| align="center" | '''1,559'''<br />
| align="center" | '''603,578'''<br />
|-<br />
| align="center" | '''3.4'''<br />
| align="center" | '''550'''<br />
| align="center" | '''2011-04-19'''<br />
| align="center" | '''1,583'''<br />
| align="center" | '''643,878'''<br />
|-<br />
| align="center" | '''3.5'''<br />
| align="center" | '''568'''<br />
| align="center" | '''2011-09-21'''<br />
| align="center" | '''1,582'''<br />
| align="center" | '''648,098'''<br />
|-<br />
| align="center" | '''3.6'''<br />
| align="center" | '''656'''<br />
| align="center" | '''2013-04-18'''<br />
| align="center" | '''1,638'''<br />
| align="center" | '''684,351'''<br />
|-<br />
| align="center" | '''3.7'''<br />
| align="center" | '''974'''<br />
| align="center" | '''2019-07-25'''<br />
| align="center" | '''1,784'''<br />
| align="center" | '''844,948'''<br />
|-<br />
| align="center" | '''3.8'''<br />
| align="center" | '''995'''<br />
| align="center" | '''2020-01-09'''<br />
| align="center" | '''1,799'''<br />
| align="center" | '''848,245'''<br />
|-<br />
| align="center" | '''3.9'''<br />
| align="center" | '''1054'''<br />
| align="center" | '''2021-03-06'''<br />
| align="center" | '''1,909'''<br />
| align="center" | '''884,508'''<br />
|-<br />
| align="center" | '''4.0'''<br />
| align="center" | '''1108'''<br />
| align="center" | '''2022-02-25'''<br />
| align="center" | '''1,985'''<br />
| align="center" | '''1,066,953'''<br />
|-<br />
| align="center" | <span class="red">current</span><br />
| align="center" | '''1123'''<br />
| align="center" | '''2022-04-25'''<br />
| align="center" | '''1,989'''<br />
| align="center" | '''1,079,102'''<br />
|}<br />
<div style="clear: both"></div><br />
<br />
<br />
On '''May 7, 2022''', '''ROMS''' had '''6970''' registered users at [http://www.myroms.org www.myroms.org]. The histogram below shows the number of new '''ROMS''' users per year and the number of countries (<span class="red">red</span> values on top of the bar) where the new users reside. The tally was started when '''ROMS 2.0''' was released on '''June 30, 2003'''.<br />
<br />
<br />
[[Image:roms_users.png|1000px|center]]<br />
<br />
===ROMS Selected References===<br />
<br />
The following is an alphabetic list of '''essential publications''' describing various aspects of the '''ROMS''' algorithms:<br />
<br />
<br />
* <span id="HaidvogelDB_2000a"></span>Haidvogel, D. B., H. G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A. F. Shchepetkin, 2000: Model evaluation experiments in the North Atlantic Basin: Simulations in nonlinear terrain-following coordinates, ''Dyn. Atmos. Oceans'', '''32''', 239-281. ([http://marine.rutgers.edu/po/Papers/damee_roms1.pdf PDF])<br />
<br />
* <span id="FennelK_2006a"></span>Fennel, K., J. Wilkin, J. Levin, J. Moisan, J. O'Reilly, and D. Haidvogel, 2006: Nitrogen cycling in the Middle Atlantic Bight: Results from a three-dimensional model and implications for the North Atlantic nitrogen budget, ''Global Biogeochem. Cycles'', '''20''', GB3007, doi:10.1029/2005GB002456. ([http://www.myroms.org/Papers/Fennel_etal_gbc.pdf PDF])<br />
<br />
* <span id="GurolS_2014"></span>Gurol, S., A. T. Weaver, A. M. Moore, A. Piacentini, H. G. Arango, and S. Gratton, 2014: B-preconditioned minimization algorithms for variational data assimilation with the dual formulation, ''Q. J. R. Meteorol. Soc.'', '''140''', 539-556, DOI:10.1002/qj.2150.<br />
<br />
* <span id="MooreAM_2021"></span>Moore, A.M. and H.G. Arango, 2021: On the behavior of ocean analysis and forecast error covariance in the presence of baroclinic instability, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101733.<br />
<br />
* <span id=LevinJ_2020a></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2020: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transportin 4D-Var ocean state estimates: Part I – Multiplatform Analysis, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101721.<br />
<br />
* <span id="LevinJ_2020b"></span>Levin, J., H.G. Arango, B. Laughlin, E. Hunter, J. Wilkin, and A.M. Moore, 2021: Observation impacts on the Mid-Atlantic Bight front and cross-shelf transport in 4D-Var ocean state estimates: Part II - The Pioneer Array, ''Ocean Modelling'', '''157''', doi:10.1016/j.ocemod.2020.101731.<br />
<br />
* <span id="MooreAM_2004a"></span>Moore, A. M., H. G. Arango, A. J. Miller, B. D. Cornuelle, E. Di Lorenzo and D. J. Neilson, 2004: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, ''Ocean Modelling'', '''7''', 227-258. ([http://marine.rutgers.edu/po/Papers/Moore_2004_om.pdf PDF])<br />
<br />
* <span id="MooreAM_2011a"></span>Moore, A.M., H.G. Arango, G. Broquet, B.S. Powell, A.T. Weaver, and J. Zavala-Garay, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part I - System overview and formulation, ''Progress in Oceanography'', '''91''', 34-49.<br />
<br />
* <span id="MooreAM_2011b"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part II: Performance and application to the California Current System, ''Progress in Oceanography'', '''91''', 50-73.<br />
<br />
* <span id="MooreAM_2011c"></span>Moore, A.M., H.G. Arango, and G. Broquet, C. Edwards, M. Veneziani, B.S. Powell, D. Foley, J. Doyle, D. Costa, and P. Robinson, 2011: The Regional Ocean Modeling System (ROMS) 4-dimensional variational data assimilation systems, Part III: Observation Impact and Observation Sensitivity in the California Current System, ''Ocean Modelling'', '''91''', 74-94.<br />
<br />
* <span id="MooreAM_2017"></span>Moore, A.M., H.G. Arango, C.A. Edwards, 2017: Reduced-Rank Array Modes of the California Current Observing System, ''J. Geophys. Res. Ocean'', '''122''', doi:10.1002/2017JC013172.<br />
<br />
* <span id="MooreAM_2020"></span>Moore, A.M., J. Zavala-Garay, H.G. Arango, C.A. Edwards, J. Anderson, and T. Hoar, 2020: Regional and basin scale applications of ensemble adjustment Kalman filter and 4D-Var ocean data assimilation systems, ''Progress in Oceanography'', '''189''', doi:10.1016/j.pocean.2020.102450.<br />
<br />
* <span id="ShchepetkinAF_2003a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2003: A method for computing horizontal pressure-gradient force in an oceanic model with a nonaligned vertical coordinate, ''J. Geophys. Res.'', '''108''' (C3), 3090, doi:10.1029/2001JC001047. ([http://marine.rutgers.edu/po/Papers/2001JC001047.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2005a"></span>Shchepetkin, A. F., and J. C. McWilliams, 2005: The Regional Ocean Modeling System: A split-explicit, free-surface, topography following coordinates ocean model, ''Ocean Modelling'', '''9''', 347-404. ([http://marine.rutgers.edu/po/Papers/Shchepetkin_om05.pdf PDF])<br />
<br />
* <span id="ShchepetkinAF_2008b"></span>Shchepetkin, A. F., and J. C. McWilliams, 2008: Computational kernel algorithms for fine-scale, multi-process, long-time oceanic simulations. In: ''Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere'', eds. R. Temam & J. Tribbia, Elsevier Science, ISBN-10: 0444518932, ISBN-13: 978-0444518934.<br />
<br />
* <span id="ShchepetkinAF_2015">Shchepetkin, A. F., 2015: An Adaptive, Courant-number-dependent Implicit Scheme for Vertical Advection in Oceanic Modeling, ''Ocean Modelling'', '''91''', 38-69, doi:10.1016/j.ocemod.2015.03.006.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J. C., Z. Defne, K. Hass, and H. G. Arango, 2013: A wetting and drying scheme for ROMS, ''Computers & Geosciences'', '''58''', 54-61, 10.1016/j.cageo.2013.05.004.<br />
<br />
* <span id="WarnerJC_2013"></span>Warner, J.C., W.R. Geyer, and H.G. Arango, 2010: Using composite grid approach in complex coastal domain to estimate estuarine residence time, ''Computer and Geosciences'', '''36''', 921-935, doi:10.1016/j.cageo.2009.11.008.<br />
<br />
* <span id="WarnerJC_2006a"></span>Warner, J. C., C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango, 2008: Development of a three-dimensional, regional, coupled wave, current, and sediment-transport model, ''Computers & Geosciences'', '''34''', 1284-1306.<br />
<br />
* <span id="WarnerJC_2005a"></span>Warner, J. C., C. R. Sherwood, H. G. Arango, and R. P. Signell, 2005a: Performance of four turbulence closure methods implemented using a generic length scale method, ''Ocean Modelling'', '''8''', 81-113. ([http://marine.rutgers.edu/po/Papers/Warner_2004_om.pdf PDF])<br />
<br />
* <span id="WilkinJL_2006a"></span>Wilkin, J. L., 2006: The summer time heat budget and circulation of southeast New England shelf, ''J. Phys. Oceanog.'', '''36''', 1997-2011. ([http://marine.rutgers.edu/%7Ewilkin/Wilkin_CBLAST_JPO2006.pdf PDF])</div>Arango