ROMS Standard Output
ROMS writes to the standard output, unit 6 by default. It writes a very complete set of diagnostics on startup so that you can check to make sure everything is in order. An example of this output for the BENCHMARK option is below. I will attempt to describe each section here, but it's a wiki, so fix what's wrong:
Setup Information
ROMS starts by outputting some General Configuration and setup information.
Parallel Initialization
We've got four MPI processes here, with the respective Unix pid's.
Node # 0 (pid= 13399) is active.
Node # 2 (pid= 13401) is active.
Node # 3 (pid= 13402) is active.
Node # 1 (pid= 13400) is active.
This is header information, including the title for this case, notes on the operating system, compiler, compiler options, and some information on which files were used.
Model Input Parameters: ROMS/TOMS version 3.0
Wednesday - January 30, 2008 - 3:07:49 PM
-----------------------------------------------------------------------------
Benchmark Test, Idealized Southern Ocean, Large Grid
Operating system : Linux
CPU/hardware : x86_64
Compiler system : pgi
Compiler command : mpif90
Compiler flags : -O3 -tp k8-64 -Mfree
Input Script : External/ocean_benchmark3.in
SVN Root URL : https://www.myroms.org/svn/omlab/branches/kate
SVN Revision : 526M
Local Root : /import/home/u1/uaf/kate/kate_branch
Header Dir : ./ROMS/Include
Header file : benchmark.h
Analytical Dir: /import/home/u1/uaf/kate/kate_branch/ROMS/Functionals
Grid and Input/Output Information
Next ROMS will output grid dimensions and parameters.
Physical Dimensions and Tile Partitioning
We've got one grid, with 2048x256x30 grid points running on four nodes in a 2x2 tiling. This information comes from the ascii input file, so if you don't see this output, ROMS isn't reading the input correctly.
Resolution, Grid 01: 2048x0256x030, Parallel Nodes: 4, Tiling: 002x002
Physical Parameters
Gory details about grid one and all its parameters.
Physical Parameters, Grid: 01
=============================
200 ntimes Number of timesteps for 3-D equations.
150.000 dt Timestep size (s) for 3-D equations.
20 ndtfast Number of timesteps for 2-D equations between
each 3D timestep.
1 ERstr Starting ensemble/perturbation run number.
1 ERend Ending ensemble/perturbation run number.
0 nrrec Number of restart records to read from disk.
T LcycleRST Switch to recycle time-records in restart file.
1000 nRST Number of timesteps between the writing of data
into restart fields.
1 ninfo Number of timesteps between print of information
to standard output.
T ldefout Switch to create a new output NetCDF file(s).
1000 nHIS Number of timesteps between the writing fields
into history file.
5.0000E+02 tnu2(01) Horizontal, harmonic mixing coefficient (m2/s)
for tracer 01: temp
5.0000E+02 tnu2(02) Horizontal, harmonic mixing coefficient (m2/s)
for tracer 02: salt
5.0000E+03 visc2 Horizontal, harmonic mixing coefficient (m2/s)
for momentum.
1.0000E-05 Akt_bak(01) Background vertical mixing coefficient (m2/s)
for tracer 01: temp
1.0000E-05 Akt_bak(02) Background vertical mixing coefficient (m2/s)
for tracer 02: salt
1.0000E-04 Akv_bak Background vertical mixing coefficient (m2/s)
for momentum.
3.0000E-04 rdrg Linear bottom drag coefficient (m/s).
3.0000E-03 rdrg2 Quadratic bottom drag coefficient.
2.0000E-02 Zob Bottom roughness (m).
1.0000E+01 blk_ZQ Height (m) of surface air humidity measurement.
1.0000E+01 blk_ZT Height (m) of surface air temperature measurement.
1.0000E+01 blk_ZW Height (m) of surface winds measurement.
1 lmd_Jwt Jerlov water type.
5.0000E+00 theta_s S-coordinate surface control parameter.
4.0000E-01 theta_b S-coordinate bottom control parameter.
200.000 Tcline S-coordinate surface/bottom layer width (m) used
in vertical coordinate stretching.
1025.000 rho0 Mean density (kg/m3) for Boussinesq approximation.
0.000 dstart Time-stamp assigned to model initialization (days).
0.00 time_ref Reference time for units attribute (yyyymmdd.dd)
0.0000E+00 Tnudg(01) Nudging/relaxation time scale (days)
for tracer 01: temp
0.0000E+00 Tnudg(02) Nudging/relaxation time scale (days)
for tracer 02: salt
0.0000E+00 Znudg Nudging/relaxation time scale (days)
for free-surface.
0.0000E+00 M2nudg Nudging/relaxation time scale (days)
for 2D momentum.
0.0000E+00 M3nudg Nudging/relaxation time scale (days)
for 3D momentum.
0.0000E+00 obcfac Factor between passive and active
open boundary conditions.
10.000 T0 Background potential temperature (C) constant.
35.000 S0 Background salinity (PSU) constant.
1.000 gamma2 Slipperiness variable: free-slip (1.0) or
no-slip (-1.0).
Output/Input Files
The I/O filenames. There aren't many for this case! In real applications this will have file names for your various output, grid, initial conditions, forcing, boundary, etc. files.
Output Restart File: ocean_rst.nc
IO Variable Information File: External/varinfo.dat
Detailed Tile Partition Information
Since this is a parallel application, we may want to know the details on the tiling. Here they are.
Tile partition information for Grid 01: 2048x0256x0030 tiling: 002x002
tile Istr Iend Jstr Jend Npts
0 1 1024 1 128 3932160
1 1025 2048 1 128 3932160
2 1 1024 129 256 3932160
3 1025 2048 129 256 3932160
Maximum halo size in XI and ETA directions:
HaloSizeI(1) = 3108
HaloSizeJ(1) = 417
TileSide(1) = 1030
TileSize(1) = 136990
C-preprocessing Options
ROMS will llist of C preprocessor variables that were defined, starting with the case name (BENCHMARK3).
Activated C-preprocessing Options:
BENCHMARK3 Benchmark Test, Idealized Southern Ocean, Large Grid
ALBEDO Shortwave radiation from albedo equation.
ANA_BSFLUX Analytical kinematic bottom salinity flux.
ANA_BTFLUX Analytical kinematic bottom temperature flux.
ANA_CLOUD Analytical cloud fraction.
ANA_GRID Analytical grid set-up.
ANA_HUMIDITY Analytical surface air humidity.
ANA_INITIAL Analytical initial conditions.
ANA_PAIR Analytical surface air pressure.
ANA_RAIN Analytical rain fall rate.
ANA_SRFLUX Analytical kinematic shortwave radiation flux.
ANA_SSFLUX Analytical kinematic surface salinity flux.
ANA_WINDS Analytical surface wind components.
ASSUMED_SHAPE Using assumed-shape arrays.
BULK_FLUXES Surface bulk fluxes parametererization.
CURVGRID Orthogonal curvilinear grid.
DJ_GRADPS Parabolic Splines density Jacobian (Shchepetkin, 2002).
DOUBLE_PRECISION Double precision arithmetic.
EW_PERIODIC East-West periodic boundaries.
LMD_CONVEC LMD convective mixing due to shear instability.
LMD_MIXING Large/McWilliams/Doney interior mixing.
LMD_NONLOCAL LMD convective nonlocal transport.
LMD_RIMIX LMD diffusivity due to shear instability.
LMD_SKPP KPP surface boundary layer mixing.
LONGWAVE Compute net longwave radiation internally.
MIX_GEO_TS Mixing of tracers along geopotential surfaces.
MIX_S_UV Mixing of momentum along constant S-surfaces.
MPI MPI distributed-memory configuration.
NONLINEAR Nonlinear Model.
NONLIN_EOS Nonlinear Equation of State for seawater.
NORTHERN_WALL Wall boundary at Northern edge.
POWER_LAW Power-law shape time-averaging barotropic filter.
PROFILE Time profiling activated .
!RST_SINGLE Double precision fields in restart NetCDF file.
SALINITY Using salinity.
SOLAR_SOURCE Solar Radiation Source Term.
SOLVE3D Solving 3D Primitive Equations.
SOUTHERN_WALL Wall boundary at Southern edge.
SPLINES Conservative parabolic spline reconstruction.
SPHERICAL Spherical grid configuration.
TS_U3HADVECTION Third-order upstream bias horizontal advection of tracers.
TS_C4VADVECTION Fourth-order centered vertical advection of tracers.
TS_DIF2 Harmonic mixing of tracers.
UV_ADV Advection of momentum.
UV_COR Coriolis term.
UV_U3HADVECTION Third-order upstream bias advection of momentum.
UV_QDRAG Quadratic bottom stress.
UV_VIS2 Harmonic mixing of momentum.
VAR_RHO_2D Variable density barotropic mode.
Configuration and Initialization
Finally ROMS is ready to setup the actual model run.
Vertical S-coordinate
This stuff is actually quite useful for finding out about your vertical coordinate system. You asked for 30 levels, the shallowest water is 500 meters deep and the deepest is 4000 meters deep. This lists the vertical resolution for shallow, deep, and intermediate waters.
INITIAL: Configuring and initializing forward nonlinear model ...
Vertical S-coordinate System:
level S-coord Cs-curve at_hmin over_slope at_hmax
30 0.0000000 0.0000000 0.000 0.000 0.000
29 -0.0333333 -0.0024174 -7.392 -11.622 -15.853
28 -0.0666667 -0.0052838 -14.918 -24.165 -33.412
27 -0.1000000 -0.0087922 -22.638 -38.024 -53.410
26 -0.1333333 -0.0131904 -30.624 -53.707 -76.790
25 -0.1666667 -0.0187972 -38.973 -71.868 -104.763
24 -0.2000000 -0.0260168 -47.805 -93.334 -138.864
23 -0.2333333 -0.0353508 -57.272 -119.136 -181.000
22 -0.2666667 -0.0473981 -67.553 -150.499 -233.446
21 -0.3000000 -0.0628318 -78.850 -188.805 -298.761
20 -0.3333333 -0.0823381 -91.368 -235.460 -379.551
19 -0.3666667 -0.1065031 -105.284 -291.665 -478.045
18 -0.4000000 -0.1356491 -120.695 -358.081 -595.466
17 -0.4333333 -0.1696534 -137.563 -434.456 -731.349
16 -0.4666667 -0.2078237 -155.680 -519.372 -883.063
15 -0.5000000 -0.2489214 -174.676 -610.289 -1045.901
14 -0.5333333 -0.2913811 -194.081 -703.998 -1213.915
13 -0.5666667 -0.3336756 -213.436 -797.368 -1381.301
12 -0.6000000 -0.3746809 -232.404 -888.096 -1543.788
11 -0.6333333 -0.4138998 -250.837 -975.161 -1699.486
10 -0.6666667 -0.4514899 -268.780 -1058.888 -1848.995
9 -0.7000000 -0.4881475 -286.444 -1140.702 -1994.960
8 -0.7333333 -0.5249360 -304.147 -1222.785 -2141.423
7 -0.7666667 -0.5631362 -322.274 -1307.762 -2293.251
6 -0.8000000 -0.6041494 -341.245 -1398.506 -2455.768
5 -0.8333333 -0.6494565 -361.504 -1498.053 -2634.601
4 -0.8666667 -0.7006196 -383.519 -1609.604 -2835.688
3 -0.9000000 -0.7593114 -407.793 -1736.588 -3065.383
2 -0.9333333 -0.8273620 -434.875 -1882.759 -3330.642
1 -0.9666667 -0.9068164 -465.378 -2052.307 -3639.236
0 -1.0000000 -1.0000000 -500.000 -2250.000 -4000.000
Time-stepping
Timestepping details for those who know how to parse it.
Time Splitting Weights: ndtfast = 20 nfast = 29
Primary Secondary Accumulated to Current Step
1-0.0009651193358779 0.0500000000000000-0.0009651193358779 0.0500000000000000
2-0.0013488780126037 0.0500482559667939-0.0023139973484816 0.1000482559667939
3-0.0011514592651645 0.0501156998674241-0.0034654566136461 0.1501639558342179
4-0.0003735756740661 0.0501732728306823-0.0038390322877122 0.2003372286649002
5 0.0009829200513762 0.0501919516143856-0.0028561122363360 0.2505291802792859
6 0.0029141799764308 0.0501428056118168 0.0000580677400948 0.3006719858911026
7 0.0054132615310267 0.0499970966129953 0.0054713292711215 0.3506690825040979
8 0.0084687837865132 0.0497264335364439 0.0139401130576347 0.4003955160405418
9 0.0120633394191050 0.0493029943471183 0.0260034524767397 0.4496985103876600
10 0.0161716623600090 0.0486998273761630 0.0421751148367487 0.4983983377638231
11 0.0207585511322367 0.0478912442581626 0.0629336659689854 0.5462895820219856
12 0.0257765478740990 0.0468533167015507 0.0887102138430845 0.5931428987235363
13 0.0311633730493854 0.0455644893078458 0.1198735868924699 0.6387073880313821
14 0.0368391158442262 0.0440063206553765 0.1567127027366961 0.6827137086867586
15 0.0427031802506397 0.0421643648631652 0.1994158829873358 0.7248780735499238
16 0.0486309868367616 0.0400292058506332 0.2480468698240974 0.7649072794005569
17 0.0544704302037592 0.0375976565087951 0.3025173000278566 0.8025049359093520
18 0.0600380921294286 0.0348741349986072 0.3625553921572852 0.8373790709079591
19 0.0651152103984764 0.0318722303921357 0.4276706025557615 0.8692513013000948
20 0.0694434033194840 0.0286164698722119 0.4971140058752455 0.8978677711723067
21 0.0727201499285570 0.0251442997062377 0.5698341558038026 0.9230120708785446
22 0.0745940258796570 0.0215082922098099 0.6444281816834597 0.9445203630883544
23 0.0746596950216181 0.0177785909158270 0.7190878767050776 0.9622989540041814
24 0.0724526566618460 0.0140456061647461 0.7915405333669236 0.9763445601689275
25 0.0674437485167026 0.0104229733316538 0.8589842818836262 0.9867675335005814
26 0.0590334053485720 0.0070507859058187 0.9180176872321983 0.9938183194064001
27 0.0465456732896125 0.0040991156383901 0.9645633605218107 0.9979174350447902
28 0.0292219798521905 0.0017718319739095 0.9937853403740011 0.9996892670186996
29 0.0062146596259991 0.0003107329813000 1.0000000000000000 0.9999999999999996
ndtfast, nfast = 20 29 nfast/ndtfast = 1.45000
Centers of gravity and integrals (values must be 1, 1, approx 1/2, 1, 1):
1.000000000000 1.060707743385 0.530353871693 1.000000000000 1.000000000000
Power filter parameters, Fgamma, gamma = 0.28400 0.14200
Grid Spacing
Minimum X-grid spacing, DXmin = 6.67292613E+00 km
Maximum X-grid spacing, DXmax = 1.25747428E+01 km
Minimum Y-grid spacing, DYmin = 8.68753316E+00 km
Maximum Y-grid spacing, DYmax = 8.68753316E+00 km
Minimum Z-grid spacing, DZmin = 7.39187981E+00 m
Maximum Z-grid spacing, DZmax = 3.60764219E+02 m
As state, min and max grid spacings.
Sanity Diagnostics
Other diagnostics for determining if all your decisions were sensible.
Minimum barotropic Courant Number = 9.46797803E-02
Maximum barotropic Courant Number = 2.63504236E-01
Maximum Coriolis Courant Number = 2.05621375E-02
Maximum grid stiffness ratios: rx0 = 4.193425E-01 (Beckmann and Haidvogel)
rx1 = 1.224868E+01 (Haney)
Initial basin volumes: TotVolume = 1.65993688308698E+17 m3
MinVolume = 4.30124999569357E+08 m3
MaxVolume = 3.93471529157603E+10 m3
Max/Min = 9.14784143101536E+01
Running
The actual timestepping - finally! Here, we asked for a diagnostics report every timestep for all 200 of them.
NL ROMS/TOMS: started time-stepping:( TimeSteps: 00000001 - 00000200)
STEP time[DAYS] KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME trd
0 0.00000 0.000000E+00 1.962855E+04 1.962855E+04 1.659937E+17 0
1 0.00174 9.462490E-09 1.962855E+04 1.962855E+04 1.659937E+17 0
2 0.00347 3.773820E-08 1.962855E+04 1.962855E+04 1.659937E+17 0
3 0.00521 8.474779E-08 1.962855E+04 1.962855E+04 1.659937E+17 0
4 0.00694 1.504326E-07 1.962855E+04 1.962855E+04 1.659937E+17 0
5 0.00868 2.347116E-07 1.962855E+04 1.962855E+04 1.659937E+17 0
6 0.01042 3.374971E-07 1.962855E+04 1.962855E+04 1.659937E+17 0
7 0.01215 4.586904E-07 1.962855E+04 1.962855E+04 1.659937E+17 0
:
198 0.34375 2.922162E-05 1.962855E+04 1.962855E+04 1.659937E+17 0
199 0.34549 2.913050E-05 1.962855E+04 1.962855E+04 1.659937E+17 0
200 0.34722 2.903620E-05 1.962855E+04 1.962855E+04 1.659937E+17 0
Performance Statistics
Some timing statistics produced by ROMS internally. Some of its percentages are a little odd for parallel runs, but they should make sense for serial runs. Note that the 2D engine is significant. Also note that we've turned off writing of NetCDF files for this case - realistic problems spend a significant fraction in the I/O.
Elapsed CPU time (seconds):
Node # 0 CPU: 3593.181
Node # 1 CPU: 3593.181
Node # 2 CPU: 3593.181
Node # 3 CPU: 3593.181
Total: 14372.725
Nonlinear model elapsed time profile:
Initialization ................................... 1.816 ( 0.0126 %)
Reading of input data ............................ 0.000 ( 0.0000 %)
Processing of input data ......................... 6.522 ( 0.0454 %)
Computation of vertical boundary conditions ...... 1.120 ( 0.0078 %)
Computation of global information integrals ...... 16.315 ( 0.1135 %)
Writing of output data ........................... 0.001 ( 0.0000 %)
Model 2D kernel .................................. 1032.187 ( 7.1816 %)
2D/3D coupling, vertical metrics ................. 24.186 ( 0.1683 %)
Omega vertical velocity .......................... 31.447 ( 0.2188 %)
Equation of state for seawater ................... 118.504 ( 0.8245 %)
Atmosphere-Ocean bulk flux parameterization ...... 70.950 ( 0.4936 %)
KPP vertical mixing parameterization ............. 438.943 ( 3.0540 %)
3D equations right-side terms .................... 198.657 ( 1.3822 %)
3D equations predictor step ...................... 353.069 ( 2.4565 %)
Pressure gradient ................................ 129.359 ( 0.9000 %)
Harmonic mixing of tracers, geopotentials ........ 291.478 ( 2.0280 %)
Harmonic stress tensor, S-surfaces ............... 115.342 ( 0.8025 %)
Corrector time-step for 3D momentum .............. 227.856 ( 1.5853 %)
Corrector time-step for tracers .................. 270.002 ( 1.8786 %)
Total: 3327.755 23.1533
Nonlinear model message Passage profile:
Message Passage: 2D halo exchanges ............... 25.312 ( 0.1761 %)
Message Passage: 3D halo exchanges ............... 13.013 ( 0.0905 %)
Message Passage: 4D halo exchanges ............... 6.765 ( 0.0471 %)
Message Passage: data broadcast .................. 0.004 ( 0.0000 %)
Message Passage: data reduction .................. 216.131 ( 1.5038 %)
Total: 261.224 1.8175
All percentages are with respect to total time = 14372.725
Wrap-up
Wrap-up and report of ending time.
ROMS/TOMS - Output NetCDF summary for Grid 01:
Analytical header files used:
ROMS/Functionals/ana_btflux.h
ROMS/Functionals/ana_cloud.h
ROMS/Functionals/ana_grid.h
ROMS/Functionals/ana_humid.h
ROMS/Functionals/ana_initial.h
ROMS/Functionals/ana_pair.h
ROMS/Functionals/ana_rain.h
ROMS/Functionals/ana_srflux.h
ROMS/Functionals/ana_stflux.h
ROMS/Functionals/ana_tair.h
ROMS/Functionals/ana_winds.h
ROMS/TOMS: DONE... Wednesday - January 30, 2008 - 4:07:42 PM