ROMS Blows up after third year simulation

[novice]

Hi Forum,

I am using ROMS model configured for a climatological run in Indian Ocean. Model is initialized with a restart file from the previous run of third year and one month. My model blows up after the start of simulation for the fourth year first month.
Bathymetry (etopo2) is smoothen by 0.2 r value to avoid any chances of bathymetry related issue.
Kindly suggest to solve the problem.


Following is my model configuration and simulation step....


Model Input Parameters: ROMS/TOMS version 3.7
Thursday - March 6, 2014 - 11:42:24 AM
-----------------------------------------------------------------------------

Trial run for Indian Ocean

Operating system : Linux
CPU/hardware : x86_64
Compiler system : gfortran
Compiler command : /usr/bin/mpif90
Compiler flags : -frepack-arrays -O3 -ffast-math -ffree-form -ffree-line-length-none -ffree-form -ffree-line-length-none -ffree-form -ffree-line-length-none

Input Script :

SVN Root URL : https:://myroms.org/svn/src
SVN Revision : 669M

Local Root : /home/osf123/src0
Header Dir : /home/osf123/src0/test/test_io2
Header file : test_io2.h
Analytical Dir: /home/osf123/src0/test/test_io2

Resolution, Grid 01: 0359x0251x032, Parallel Nodes: 30, Tiling: 006x005


Physical Parameters, Grid: 01
=============================

5184000 ntimes Number of timesteps for 3-D equations.
60.000 dt Timestep size (s) for 3-D equations.
30 ndtfast Number of timesteps for 2-D equations between
each 3D timestep.
1 ERstr Starting ensemble/perturbation run number.
1 ERend Ending ensemble/perturbation run number.
-1 nrrec Number of restart records to read from disk.
T LcycleRST Switch to recycle time-records in restart file.
864 nRST Number of timesteps between the writing of data
into restart fields.
1 ninfo Number of timesteps between print of information
to standard output.
T ldefout Switch to create a new output NetCDF file(s).
43200 nHIS Number of timesteps between the writing fields
into history file.
518400 ndefHIS Number of timesteps between creation of new
history files.
1 ntsAVG Starting timestep for the accumulation of output
time-averaged data.
43200 nAVG Number of timesteps between the writing of
time-averaged data into averages file.
518400 ndefAVG Number of timesteps between creation of new
time-averaged file.
1.0000E-06 Akt_bak(01) Background vertical mixing coefficient (m2/s)
for tracer 01: temp
1.0000E-06 Akt_bak(02) Background vertical mixing coefficient (m2/s)
for tracer 02: salt
1.0000E-05 Akv_bak Background vertical mixing coefficient (m2/s)
for momentum.
5.0000E-06 Akk_bak Background vertical mixing coefficient (m2/s)
for turbulent energy.
5.0000E-06 Akp_bak Background vertical mixing coefficient (m2/s)
for turbulent generic statistical field.
1.0000E-04 rdrg Linear bottom drag coefficient (m/s).
1.0000E-03 rdrg2 Quadratic bottom drag coefficient.
2.0000E-02 Zob Bottom roughness (m).
1 Vtransform S-coordinate transformation equation.
1 Vstretching S-coordinate stretching function.
6.0000E+00 theta_s S-coordinate surface control parameter.
0.0000E+00 theta_b S-coordinate bottom control parameter.
10.000 Tcline S-coordinate surface/bottom layer width (m) used
in vertical coordinate stretching.
1027.000 rho0 Mean density (kg/m3) for Boussinesq approximation.
0.000 dstart Time-stamp assigned to model initialization (days).
0.00 time_ref Reference time for units attribute (yyyymmdd.dd)
0.0000E+00 Tnudg(01) Nudging/relaxation time scale (days)
for tracer 01: temp
0.0000E+00 Tnudg(02) Nudging/relaxation time scale (days)
for tracer 02: salt
0.0000E+00 Znudg Nudging/relaxation time scale (days)
for free-surface.
0.0000E+00 M2nudg Nudging/relaxation time scale (days)
for 2D momentum.
0.0000E+00 M3nudg Nudging/relaxation time scale (days)
for 3D momentum.
0.0000E+00 obcfac Factor between passive and active
open boundary conditions.
F VolCons(1) NLM western edge boundary volume conservation.
F VolCons(2) NLM southern edge boundary volume conservation.
F VolCons(3) NLM eastern edge boundary volume conservation.
F VolCons(4) NLM northern edge boundary volume conservation.
-0.278 T0 Background potential temperature (C) constant.
34.649 S0 Background salinity (PSU) constant.
1.000 gamma2 Slipperiness variable: free-slip (1.0) or
no-slip (-1.0).
T LtracerCLM(01) Processing climatology on tracer 01: temp
T LtracerCLM(02) Processing climatology on tracer 02: salt
T Hout(idFsur) Write out free-surface.
T Hout(idUbar) Write out 2D U-momentum component.
T Hout(idVbar) Write out 2D V-momentum component.
T Hout(idUvel) Write out 3D U-momentum component.
T Hout(idVvel) Write out 3D V-momentum component.
T Hout(idWvel) Write out W-momentum component.
T Hout(idOvel) Write out omega vertical velocity.
T Hout(idTvar) Write out tracer 01: temp
T Hout(idTvar) Write out tracer 02: salt

T Aout(idFsur) Write out averaged free-surface.
T Aout(idUbar) Write out averaged 2D U-momentum component.
T Aout(idVbar) Write out averaged 2D V-momentum component.
T Aout(idUvel) Write out averaged 3D U-momentum component.
T Aout(idVvel) Write out averaged 3D V-momentum component.
T Aout(idWvel) Write out averaged W-momentum component.
T Aout(idOvel) Write out averaged omega vertical velocity.
T Aout(idTvar) Write out averaged tracer 01: temp
T Aout(idTvar) Write out averaged tracer 02: salt

Output/Input Files:

Output Restart File: ocean_rst.nc
Prefix for History Files: ocean_his
Prefix for Averages Files: ocean_avg
Input Grid File: roms_grd.nc
Input Nonlinear Initial File: ocean_rst.nc
Input Forcing File 01: roms_frc.nc
Input Climatology File: roms_clm.nc

Tile partition information for Grid 01: 0359x0251x0032 tiling: 006x005

tile Istr Iend Jstr Jend Npts

0 1 60 1 49 94080
1 61 120 1 49 94080
2 121 180 1 49 94080
3 181 240 1 49 94080
4 241 300 1 49 94080
5 301 359 1 49 92512
6 1 60 50 100 97920
7 61 120 50 100 97920
8 121 180 50 100 97920
9 181 240 50 100 97920
10 241 300 50 100 97920
11 301 359 50 100 96288
12 1 60 101 151 97920
13 61 120 101 151 97920
14 121 180 101 151 97920
15 181 240 101 151 97920
16 241 300 101 151 97920
17 301 359 101 151 96288
18 1 60 152 202 97920
19 61 120 152 202 97920
20 121 180 152 202 97920
21 181 240 152 202 97920
22 241 300 152 202 97920
23 301 359 152 202 96288
24 1 60 203 251 94080
25 61 120 203 251 94080
26 121 180 203 251 94080
27 181 240 203 251 94080
28 241 300 203 251 94080
29 301 359 203 251 92512

Tile minimum and maximum fractional coordinates for Grid 01:
(interior points only)

tile Xmin Xmax Ymin Ymax grid

0 0.50 60.50 0.50 49.50 RHO-points
1 60.50 120.50 0.50 49.50 RHO-points
2 120.50 180.50 0.50 49.50 RHO-points
3 180.50 240.50 0.50 49.50 RHO-points
4 240.50 300.50 0.50 49.50 RHO-points
5 300.50 359.50 0.50 49.50 RHO-points
6 0.50 60.50 49.50 100.50 RHO-points
7 60.50 120.50 49.50 100.50 RHO-points
8 120.50 180.50 49.50 100.50 RHO-points
9 180.50 240.50 49.50 100.50 RHO-points
10 240.50 300.50 49.50 100.50 RHO-points
11 300.50 359.50 49.50 100.50 RHO-points
12 0.50 60.50 100.50 151.50 RHO-points
13 60.50 120.50 100.50 151.50 RHO-points
14 120.50 180.50 100.50 151.50 RHO-points
15 180.50 240.50 100.50 151.50 RHO-points
16 240.50 300.50 100.50 151.50 RHO-points
17 300.50 359.50 100.50 151.50 RHO-points
18 0.50 60.50 151.50 202.50 RHO-points
19 60.50 120.50 151.50 202.50 RHO-points
20 120.50 180.50 151.50 202.50 RHO-points
21 180.50 240.50 151.50 202.50 RHO-points
22 240.50 300.50 151.50 202.50 RHO-points
23 300.50 359.50 151.50 202.50 RHO-points
24 0.50 60.50 202.50 251.50 RHO-points
25 60.50 120.50 202.50 251.50 RHO-points
26 120.50 180.50 202.50 251.50 RHO-points
27 180.50 240.50 202.50 251.50 RHO-points
28 240.50 300.50 202.50 251.50 RHO-points
29 300.50 359.50 202.50 251.50 RHO-points

0 1.00 60.50 0.50 49.50 U-points
1 60.50 120.50 0.50 49.50 U-points
2 120.50 180.50 0.50 49.50 U-points
3 180.50 240.50 0.50 49.50 U-points
4 240.50 300.50 0.50 49.50 U-points
5 300.50 359.00 0.50 49.50 U-points
6 1.00 60.50 49.50 100.50 U-points
7 60.50 120.50 49.50 100.50 U-points
8 120.50 180.50 49.50 100.50 U-points
9 180.50 240.50 49.50 100.50 U-points
10 240.50 300.50 49.50 100.50 U-points
11 300.50 359.00 49.50 100.50 U-points
12 1.00 60.50 100.50 151.50 U-points
13 60.50 120.50 100.50 151.50 U-points
14 120.50 180.50 100.50 151.50 U-points
15 180.50 240.50 100.50 151.50 U-points
16 240.50 300.50 100.50 151.50 U-points
17 300.50 359.00 100.50 151.50 U-points
18 1.00 60.50 151.50 202.50 U-points
19 60.50 120.50 151.50 202.50 U-points
20 120.50 180.50 151.50 202.50 U-points
21 180.50 240.50 151.50 202.50 U-points
22 240.50 300.50 151.50 202.50 U-points
23 300.50 359.00 151.50 202.50 U-points
24 1.00 60.50 202.50 251.50 U-points
25 60.50 120.50 202.50 251.50 U-points
26 120.50 180.50 202.50 251.50 U-points
27 180.50 240.50 202.50 251.50 U-points
28 240.50 300.50 202.50 251.50 U-points
29 300.50 359.00 202.50 251.50 U-points

0 0.50 60.50 1.00 49.50 V-points
1 60.50 120.50 1.00 49.50 V-points
2 120.50 180.50 1.00 49.50 V-points
3 180.50 240.50 1.00 49.50 V-points
4 240.50 300.50 1.00 49.50 V-points
5 300.50 359.50 1.00 49.50 V-points
6 0.50 60.50 49.50 100.50 V-points
7 60.50 120.50 49.50 100.50 V-points
8 120.50 180.50 49.50 100.50 V-points
9 180.50 240.50 49.50 100.50 V-points
10 240.50 300.50 49.50 100.50 V-points
11 300.50 359.50 49.50 100.50 V-points
12 0.50 60.50 100.50 151.50 V-points
13 60.50 120.50 100.50 151.50 V-points
14 120.50 180.50 100.50 151.50 V-points
15 180.50 240.50 100.50 151.50 V-points
16 240.50 300.50 100.50 151.50 V-points
17 300.50 359.50 100.50 151.50 V-points
18 0.50 60.50 151.50 202.50 V-points
19 60.50 120.50 151.50 202.50 V-points
20 120.50 180.50 151.50 202.50 V-points
21 180.50 240.50 151.50 202.50 V-points
22 240.50 300.50 151.50 202.50 V-points
23 300.50 359.50 151.50 202.50 V-points
24 0.50 60.50 202.50 251.00 V-points
25 60.50 120.50 202.50 251.00 V-points
26 120.50 180.50 202.50 251.00 V-points
27 180.50 240.50 202.50 251.00 V-points
28 240.50 300.50 202.50 251.00 V-points
29 300.50 359.50 202.50 251.00 V-points

Maximum halo size in XI and ETA directions:

HaloSizeI(1) = 144
HaloSizeJ(1) = 126
TileSide(1) = 66
TileSize(1) = 3762


Lateral Boundary Conditions: NLM
============================

Variable Grid West Edge South Edge East Edge North Edge
--------- ---- ---------- ---------- ---------- ----------

zeta 1 Closed Chapman Imp Chapman Imp Closed

ubar 1 Closed Flather Flather Closed

vbar 1 Closed Flather Flather Closed

u 1 Closed Radiation Radiation Closed

v 1 Closed Radiation Radiation Closed

temp 1 Closed Radiation Radiation Closed

salt 1 Closed Radiation Radiation Closed

tke 1 Closed Gradient Gradient Closed

Activated C-preprocessing Options:

TEST_IO2 Trial run for Indian Ocean
ANA_BSFLUX Analytical kinematic bottom salinity flux.
ANA_BTFLUX Analytical kinematic bottom temperature flux.
ANA_FSOBC Analytical free-surface boundary conditions.
ANA_M2OBC Analytical 2D momentum boundary conditions.
ASSUMED_SHAPE Using assumed-shape arrays.
AVERAGES Writing out time-averaged nonlinear model fields.
CURVGRID Orthogonal curvilinear grid.
DEBUGGING Internal debugging switch activated.
DOUBLE_PRECISION Double precision arithmetic.
M3CLIMATOLOGY Processing 3D momentum climatology data.
MASKING Land/Sea masking.
MPI MPI distributed-memory configuration.
MY25_MIXING Mellor/Yamada Level-2.5 mixing closure.
NONLINEAR Nonlinear Model.
NONLIN_EOS Nonlinear Equation of State for seawater.
NO_LBC_ATT Not checking NetCDF global attribute NLM_LBC during restart.
POWER_LAW Power-law shape time-averaging barotropic filter.
PRSGRD31 Standard density Jacobian formulation (Song, 1998).
PROFILE Time profiling activated .
QCORRECTION Surface net heat flux correction.
K_GSCHEME Third-order upstream advection of TKE fields.
RHO_SURF Include difference between rho0 and surface density.
!RST_SINGLE Double precision fields in restart NetCDF file.
SALINITY Using salinity.
SOLVE3D Solving 3D Primitive Equations.
TCLIMATOLOGY Processing tracer climatology data.
TS_C4HADVECTION Fourth-order centered horizontal advection of tracers.
TS_C4VADVECTION Fourth-order centered vertical advection of tracers.
UV_ADV Advection of momentum.
UV_COR Coriolis term.
UV_U3HADVECTION Third-order upstream horizontal advection of 3D momentum.
UV_C4VADVECTION Fourth-order centered vertical advection of momentum.
UV_LDRAG Linear bottom stress.
VAR_RHO_2D Variable density barotropic mode.
ZCLIMATOLOGY Processing sea surface height climatology data.

Process Information:

Node # 0 (pid= 10116) is active.
Node # 1 (pid= 10117) is active.
Node # 2 (pid= 10118) is active.
Node # 3 (pid= 10119) is active.
Node # 4 (pid= 10120) is active.
Node # 5 (pid= 10121) is active.
Node # 6 (pid= 10122) is active.
Node # 7 (pid= 10123) is active.
Node # 8 (pid= 10124) is active.
Node # 9 (pid= 10125) is active.
Node # 10 (pid= 10126) is active.
Node # 11 (pid= 10127) is active.
Node # 12 (pid= 10128) is active.
Node # 13 (pid= 10129) is active.
Node # 14 (pid= 10130) is active.
Node # 15 (pid= 10131) is active.
Node # 16 (pid= 10132) is active.
Node # 17 (pid= 10133) is active.
Node # 18 (pid= 10134) is active.
Node # 19 (pid= 10135) is active.
Node # 20 (pid= 10136) is active.
Node # 21 (pid= 10137) is active.
Node # 22 (pid= 10138) is active.
Node # 23 (pid= 10139) is active.
Node # 24 (pid= 10140) is active.
Node # 25 (pid= 10141) is active.
Node # 26 (pid= 10142) is active.
Node # 27 (pid= 10143) is active.
Node # 28 (pid= 10144) is active.
Node # 29 (pid= 10145) is active.

INITIAL: Configuring and initializing forward nonlinear model ...
*******

Vertical S-coordinate System, Grid 01:

level S-coord Cs-curve Z at hmin at hc half way at hmax

32 0.0000000 0.0000000 0.000 0.000 0.000 0.000
31 -0.0312500 -0.0009350 -0.373 -0.313 -2.676 -4.978
30 -0.0625000 -0.0019030 -0.749 -0.625 -5.435 -10.121
29 -0.0937500 -0.0029380 -1.128 -0.938 -8.363 -15.598
28 -0.1250000 -0.0040767 -1.515 -1.250 -11.554 -21.593
27 -0.1562500 -0.0053591 -1.911 -1.563 -15.108 -28.304
26 -0.1875000 -0.0068304 -2.319 -1.875 -19.139 -35.959
25 -0.2187500 -0.0085426 -2.743 -2.188 -23.779 -44.815
24 -0.2500000 -0.0105560 -3.186 -2.500 -29.180 -55.174
23 -0.2812500 -0.0129416 -3.654 -2.813 -35.522 -67.391
22 -0.3125000 -0.0157835 -4.151 -3.125 -43.018 -81.884
21 -0.3437500 -0.0191819 -4.684 -3.438 -51.920 -99.155
20 -0.3750000 -0.0232566 -5.262 -3.750 -62.531 -119.801
19 -0.4062500 -0.0281514 -5.892 -4.063 -75.215 -144.538
18 -0.4375000 -0.0340388 -6.588 -4.375 -90.408 -174.228
17 -0.4687500 -0.0411263 -7.361 -4.688 -108.634 -209.908
16 -0.5000000 -0.0496640 -8.228 -5.000 -130.526 -252.823
15 -0.5312500 -0.0599527 -9.209 -5.313 -156.843 -304.477
14 -0.5625000 -0.0723554 -10.328 -5.625 -188.503 -366.678
13 -0.5937500 -0.0873092 -11.613 -5.938 -226.612 -441.611
12 -0.6250000 -0.1053416 -13.097 -6.250 -272.501 -531.904
11 -0.6562500 -0.1270882 -14.823 -6.563 -327.778 -640.733
10 -0.6875000 -0.1533158 -16.841 -6.875 -394.381 -771.921
9 -0.7187500 -0.1849493 -19.209 -7.188 -474.647 -930.085
8 -0.7500000 -0.2231040 -22.002 -7.500 -571.395 -1120.789
7 -0.7812500 -0.2691252 -25.306 -7.813 -688.026 -1350.747
6 -0.8125000 -0.3246355 -29.226 -8.125 -828.641 -1628.056
5 -0.8437500 -0.3915923 -33.891 -8.438 -998.187 -1962.483
4 -0.8750000 -0.4723564 -39.453 -8.750 -1202.631 -2365.809
3 -0.9062500 -0.5697755 -46.098 -9.063 -1449.170 -2852.242
2 -0.9375000 -0.6872846 -54.048 -9.375 -1746.487 -3438.925
1 -0.9687500 -0.8290268 -63.574 -9.688 -2105.053 -4146.531
0 -1.0000000 -1.0000000 -75.000 -10.000 -2537.500 -5000.000

Time Splitting Weights for Grid 01: ndtfast = 30 nfast = 42
==================================

Primary Secondary Accumulated to Current Step

1-0.0008094437383769 0.0333333333333333-0.0008094437383769 0.0333333333333333
2-0.0014053566728197 0.0333603147912792-0.0022148004111966 0.0666936481246126
3-0.0017877524645903 0.0334071600137066-0.0040025528757869 0.1001008081383191
4-0.0019566842408176 0.0334667517625262-0.0059592371166046 0.1335675599008453
5-0.0019122901320372 0.0335319745705535-0.0078715272486418 0.1670995344713988
6-0.0016548570247459 0.0335957175749547-0.0095263842733877 0.2006952520463536
7-0.0011849025289723 0.0336508794757796-0.0107112868023600 0.2343461315221331
8-0.0005032751608631 0.0336903762267453-0.0112145619632232 0.2680365077488784
9 0.0003887272597151 0.0337071520654408-0.0108258347035081 0.3017436598143192
10 0.0014892209965583 0.0336941944901169-0.0093366137069498 0.3354378543044362
11 0.0027955815694920 0.0336445537902317-0.0065410321374578 0.3690824080946679
12 0.0043042707117221 0.0335513677379153-0.0022367614257356 0.4026337758325831
13 0.0060106451121704 0.0334078920475245 0.0037738836864348 0.4360416678801076
14 0.0079087469427945 0.0332075372104522 0.0116826306292293 0.4692492050905598
15 0.0099910761708920 0.0329439123123590 0.0216737068001213 0.5021931174029188
16 0.0122483446563884 0.0326108764399960 0.0339220514565097 0.5348039938429148
17 0.0146692120341107 0.0322025982847830 0.0485912634906204 0.5670065921276978
18 0.0172400033810439 0.0317136245503127 0.0658312668716643 0.5987202166780105
19 0.0199444086685725 0.0311389577709445 0.0857756755402368 0.6298591744489550
20 0.0227631639997064 0.0304741441486588 0.1085388395399432 0.6603333185976138
21 0.0256737146312911 0.0297153720153352 0.1342125541712342 0.6900486906129490
22 0.0286498597812016 0.0288595815276255 0.1628624139524359 0.7189082721405746
23 0.0316613792205220 0.0279045862015855 0.1945237931729578 0.7468128583421600
24 0.0346736416507075 0.0268492068942347 0.2291974348236653 0.7736620652363948
25 0.0376471948657328 0.0256934188392112 0.2668446296893981 0.7993554840756060
26 0.0405373376992232 0.0244385123436867 0.3073819673886213 0.8237939964192927
27 0.0432936737565710 0.0230872677537126 0.3506756411451924 0.8468812641730054
28 0.0458596469320356 0.0216441452951603 0.3965352880772280 0.8685254094681656
29 0.0481720587108284 0.0201154903974257 0.4447073467880565 0.8886408998655914
30 0.0501605672561820 0.0185097551070648 0.4948679140442384 0.9071506549726561
31 0.0517471682814030 0.0168377361985254 0.5466150823256415 0.9239883911711815
32 0.0528456577069106 0.0151128305891453 0.5994607400325521 0.9391012217603267
33 0.0533610761022577 0.0133513086655816 0.6528218161348098 0.9524525304259084
34 0.0531891349131379 0.0115726061288397 0.7060109510479478 0.9640251365547481
35 0.0522156244733761 0.0097996349650684 0.7582265755213239 0.9738247715198165
36 0.0503158038019030 0.0080591141492892 0.8085423793232269 0.9818838856691057
37 0.0473537721847153 0.0063819206892258 0.8558961515079423 0.9882658063583315
38 0.0431818225418188 0.0048034616164019 0.8990779740497611 0.9930692679747334
39 0.0376397765791564 0.0033640675316746 0.9367177506289175 0.9964333355064080
40 0.0305543017255206 0.0021094083123694 0.9672720523544381 0.9985427438187774
41 0.0217382098544504 0.0010909315881854 0.9890102622088885 0.9996336754069628
42 0.0109897377911118 0.0003663245930371 1.0000000000000004 0.9999999999999999

ndtfast, nfast = 30 42 nfast/ndtfast = 1.40000

Centers of gravity and integrals (values must be 1, 1, approx 1/2, 1, 1):

1.000000000000 1.047601458608 0.523800729304 1.000000000000 1.000000000000

Power filter parameters, Fgamma, gamma = 0.28400 0.18933

Metrics information for Grid 01:
===============================

Minimum X-grid spacing, DXmin = 2.40495901E+01 km
Maximum X-grid spacing, DXmax = 2.77832099E+01 km
Minimum Y-grid spacing, DYmin = 2.40871264E+01 km
Maximum Y-grid spacing, DYmax = 2.77830873E+01 km
Minimum Z-grid spacing, DZmin = 3.73274602E-01 m
Maximum Z-grid spacing, DZmax = 8.53468765E+02 m

Minimum barotropic Courant Number = 2.76139585E-03
Maximum barotropic Courant Number = 2.60205005E-02
Maximum Coriolis Courant Number = 4.36950352E-03


NLM: GET_STATE - Warning - NetCDF global attribute: NLM_LBC
for lateral boundary conditions not checked
in restart file: ocean_rst.nc

NLM: GET_STATE - Read state initial conditions, t = 1044 00:00:00
(Grid 01, File: ocean_rst.nc, Rec=0002, Index=1)
- free-surface
(Min = -1.71556758E+00 Max = 1.29852434E+00)
- vertically integrated u-momentum component
(Min = -1.67233753E+00 Max = 1.80405052E+00)
- vertically integrated v-momentum component
(Min = -1.62756808E+00 Max = 2.88255421E+00)
- u-momentum component
(Min = -3.45645858E+00 Max = 5.06761800E+00)
- v-momentum component
(Min = -4.58061582E+00 Max = 5.81005726E+00)
- potential temperature
(Min = -1.45412661E+02 Max = 5.49312200E+01)
- salinity
(Min = 2.13327691E+01 Max = 4.99393483E+01)
- vertical viscosity coefficient
(Min = 0.00000000E+00 Max = 2.26149188E+02)
- temperature vertical diffusion coefficient
(Min = 0.00000000E+00 Max = 2.97953777E+02)
- salinity vertical diffusion coefficient
(Min = 0.00000000E+00 Max = 2.97953777E+02)
GET_2DFLD - surface u-momentum stress, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.67221825E-04 Max = 4.46894929E-05)
GET_2DFLD - surface v-momentum stress, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -2.47540242E-04 Max = 1.10292631E-04)
GET_2DFLD - surface net heat flux, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -5.78668080E-05 Max = 4.32171632E-05)
GET_2DFLD - sea surface temperature climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 1.82882767E+01 Max = 2.97874126E+01)
GET_2DFLD - surface net heat flux sensitivity to SST, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.67268037E-05 Max = -5.56377139E-06)
GET_2DFLD - surface net freswater flux, (E-P), t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -3.65903823E-07 Max = 1.04064828E-07)
GET_2DFLD - sea surface height climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.80340451E+00 Max = 7.93942796E-01)
GET_3DFLD - potential temperature climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 2.51346192E-01 Max = 3.09622487E+01)
GET_3DFLD - salinity climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 1.87654107E+01 Max = 4.14058170E+01)
GET_3DFLD - u-momentum component climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -4.17097056E+00 Max = 1.67593050E+00)
GET_3DFLD - v-momentum component climatology, t = 315 00:00:00
(Rec=0000011, Index=1, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -2.42818214E+00 Max = 5.62615950E+00)

Basin information for Grid 01:

Maximum grid stiffness ratios: rx0 = 2.326602E-01 (Beckmann and Haidvogel)
rx1 = 2.526081E+00 (Haney)

Initial basin volumes: TotVolume = 1.4037563547E+17 m3
MinVolume = 2.1707404178E+08 m3
MaxVolume = 6.5879531608E+11 m3
Max/Min = 3.0348875927E+03

NL ROMS/TOMS: started time-stepping: (Grid: 01 TimeSteps: 01503361 - 05184000)

GET_2DFLD - surface u-momentum stress, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.68642882E-04 Max = 4.38884107E-05)
GET_2DFLD - surface v-momentum stress, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -2.30629585E-04 Max = 1.16616426E-04)
GET_2DFLD - surface net heat flux, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -6.70591379E-05 Max = 4.01735706E-05)
GET_2DFLD - sea surface temperature climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 1.78758011E+01 Max = 2.96960344E+01)
GET_2DFLD - surface net heat flux sensitivity to SST, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.56755779E-05 Max = -4.63628970E-06)
GET_2DFLD - surface net freswater flux, (E-P), t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_frc.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -2.34185175E-07 Max = 1.15583962E-07)
GET_2DFLD - sea surface height climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -1.78705581E+00 Max = 7.96083745E-01)
GET_3DFLD - potential temperature climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 2.51346192E-01 Max = 3.02954484E+01)
GET_3DFLD - salinity climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = 2.44451369E+01 Max = 4.13800304E+01)
GET_3DFLD - u-momentum component climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -4.40555192E+00 Max = 1.89923644E+00)
GET_3DFLD - v-momentum component climatology, t = 345 00:00:00
(Rec=0000012, Index=2, File: roms_clm.nc)
(Tmin= 15.0000 Tmax= 345.0000)
(Min = -2.43779523E+00 Max = 5.26730740E+00)

STEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed

1503360 1044 00:00:00 2.660592E-02 2.017151E+04 2.017154E+04 1.420723E+17
(354,118,01) 1.231160E-03 1.249742E-02 0.000000E+00 5.858310E+00
DEF_HIS - inquiring history file, Grid 01: ocean_his_0003.nc
DEF_AVG - inquiring average file, Grid 01: ocean_avg_0003.nc
1503361 1044 00:01:00 2.660537E-02 2.017151E+04 2.017154E+04 1.420723E+17
(352,161,22) 1.901020E-03 3.217765E-03 7.221660E-02 5.858197E+00
1503362 1044 00:02:00 2.660572E-02 2.017151E+04 2.017154E+04 1.420723E+17
(352,161,22) 1.902566E-03 3.221427E-03 7.252198E-02 5.858440E+00
1503363 1044 00:03:00 2.660607E-02 2.017151E+04 2.017154E+04 1.420723E+17
(352,161,22) 1.904136E-03 3.225071E-03 7.262802E-02 5.858681E+00
/\
|
|
|
\/
(350,153,32) 1.693027E-03 1.207960E-03 4.237695E-01 1.216910E+01
1616397 1122 11:57:00 2.654959E-02 2.017111E+04 2.017114E+04 1.420670E+17
(350,153,32) 1.715048E-03 1.215153E-03 4.059400E-01 1.216248E+01
1616398 1122 11:58:00 2.655689E-02 2.017111E+04 2.017114E+04 1.420670E+17
(351,150,32) 1.054688E-03 1.170428E-03 4.056764E-01 1.215524E+01

Blowing-up: Saving latest model state into RESTART file

WRT_RST - wrote re-start fields (Index=1,1) into time record = 0000003

Elapsed CPU time (seconds):

Node # 0 CPU: 28044.521

ROMS/TOMS - Output NetCDF summary for Grid 01:
number of time records written in HISTORY file = 00000001
number of time records written in RESTART file = 00000131
number of time records written in AVERAGE file = 00000001

Analytical header files used:

ROMS/Functionals/ana_btflux.h
ROMS/Functionals/ana_fsobc.h
Node # 1 CPU: 28294.884
Node # 2 CPU: 28290.268
Node # 3 CPU: 28289.876
Node # 4 CPU: 28281.900
Node # 5 CPU: 28285.780
Node # 6 CPU: 28297.000
Node # 7 CPU: 28296.656
Node # 8 CPU: 28291.668
Node # 9 CPU: 28296.496
Node # 10 CPU: 28302.321
Node # 11 CPU: 28296.772
Node # 12 CPU: 28290.592
Node # 13 CPU: 28300.409
Node # 14 CPU: 28297.208
Node # 15 CPU: 28285.376
Node # 16 CPU: 28299.889
Node # 17 CPU: 28295.204
Node # 18 CPU: 28300.669
Node # 19 CPU: 28289.200
Node # 20 CPU: 28293.496
Node # 21 CPU: 28301.669
Node # 22 CPU: 28289.380
Node # 23 CPU: 28295.860
Node # 24 CPU: 28292.836
Node # 25 CPU: 28293.640
Node # 26 CPU: 28281.471
Node # 27 CPU: 28281.523
Node # 28 CPU: 28286.896
Node # 29 CPU: 28290.840
ROMS/Functionals/ana_m2obc.h
ROMS/Functionals/ana_nudgcoef.h

ROMS/TOMS: DONE... Thursday - March 6, 2014 - 7:37:37 PM

Kindly help to resolve the problem.
Thanks in advance...

regards,
Sandy

[kate]

You need to look at record 3 of the restart file to see if you can figure out where and how the thing went bad. It either has an extreme velocity or extreme density. Your Courant numbers look OK, so I'm guessing extreme density. Is the problem at the surface, bottom or middle? Edge or interior? Due to temperature, salinity or both?

[novice]

Hi,
Thanks Kate for your quick reply. As you suggested , I checked the third record of my restart file and realized that the temperature has an abnormal range in my eastern boundary, to be more specific off the north east coast of Borneo island and the problem persists for all the levels. In case of salinity also the same location shows some discrepancy. I am a beginner to the ROMS and I would be grateful if you can point out some clues about the source of this problem.

Thanks again for your reply
Sandy

[kate]

I see you are using a pure radiation boundary condition for the 3-D fields. Perhaps it is not numerically well-behaved. Do you have external boundary conditions to nudge to? I use the RadNud combination on the boundaries.

[novice]

Hi,
Thanks Kate. As you suggested I created an external boundary condition file as roms_bry.nc, and changed the option from radiation to radiation with nudging. The model is running now and I will get back to you once it is over.

Thanks again for your support which is really encouraging.
By
Sandy