Trouble restarting with roms 3.6

[rduran]

Im using ROMS 3.6 revision 571

I followed the instructions for restarting as in viewtopic.php?f=14&t=599, in different ways:
1.a) Using a history file with NNREC = -1

1.b) Using the fields of a history file to write a netcdf file using the same script I used for my original initial file (NRREC=-1 and also NRREC=165 since that is the day I was restarting from).


2) Using a restart file with NRREC=-1

I always get an error and I don't know what to do (I didnt save the error for 1.a, but it gave error), the other two errors are:


1.b)

Code: Select all

LBC_GETATT - error while reading global attribute:  NLM_LBC
in input file:  /home/server/student/homes/rduran/MODEL/INITIAL/ore2005_ini_rst165.nc
call from:  checkvars.F
             
             
             
             
Node   ROR: Abnormal termination: NetCDF OUTPUT.
REASON: Attribute not found          
             
             
ROMS/TOMS - Output error ............ exit_flag:   3


ERROR: Abnormal termination: NetCDF OUTPUT.
REASON: Attribute not found

2)

Code: Select all

LBC_GETATT - inconsistent northern boundary for variable: tke  Keyword = Radp<91>v
             in input file:  /home/server/student/homes/rduran/MODEL/INITIAL/RST00.nc
...
 ROMS/TOMS - Configuration error ..... exit_flag:   5


 ERROR: Illegal model configuration.

I made sure that nothing other than NRREC and the name of the initial file changed in my .in file.

These are my boundary options:

Code: Select all

 Lateral Boundary Conditions: NLM
 ============================

 Variable               Grid    West Edge   South Edge  East Edge   North Edge
 ---------              ----    ----------  ----------  ----------  ----------

 zeta                     1     Chapman     Chapman     Closed      Chapman

 ubar                     1     Flather     Flather     Closed      Flather

 vbar                     1     Flather     Flather     Closed      Flather

 u                        1     Rad + Nud   Rad + Nud   Closed      Rad + Nud

 v                        1     Rad + Nud   Rad + Nud   Closed      Rad + Nud

 temp                     1     Rad + Nud   Rad + Nud   Closed      Rad + Nud

 salt                     1     Rad + Nud   Rad + Nud   Closed      Rad + Nud

 tke                      1     Radiation   Radiation   Closed      Radiation

And these are my cppdefs:

Code: Select all

ORE2005             Ore2005 52 vertical coord. daily BC shortened domain
ANA_BSFLUX          Analytical kinematic bottom salinity flux.
ANA_BTFLUX          Analytical kinematic bottom temperature flux.
ASSUMED_SHAPE       Using assumed-shape arrays.
AVERAGES            Writing out time-averaged nonlinear model fields.
DIAGNOSTICS_UV      Computing and writing momentum diagnostic terms.
DJ_GRADPS           Parabolic Splines density Jacobian (Shchepetkin, 2002)
DOUBLE_PRECISION    Double precision arithmetic.
MASKING             Land/Sea masking.
MIX_ISO_TS          Mixing of tracers along isopycnal surfaces.
MIX_GEO_UV          Mixing of momentum along geopotential surfaces.
MPI                 MPI distributed-memory configuration.
MY25_MIXING         Mellor/Yamada Level-2.5 mixing closure.
NONLINEAR           Nonlinear Model.
NONLIN_EOS          Nonlinear Equation of State for seawater.
POWER_LAW           Power-law shape time-averaging barotropic filter.
PROFILE             Time profiling activated .
K_GSCHEME           Third-order upstream advection of TKE fields.
RADIATION_2D        Use tangential phase speed in radiation conditions.
!RST_SINGLE         Double precision fields in restart NetCDF file.
SALINITY            Using salinity.
SOLVE3D             Solving 3D Primitive Equations.
SPLINES             Conservative parabolic spline reconstruction.
TS_U3HADVECTION     Third-order upstream horizontal advection of tracers.
TS_C4VADVECTION     Fourth-order centered vertical advection of tracers.
TS_DIF2             Harmonic mixing of tracers.
TS_PSOURCE          Tracers point sources and sinks.
UV_PSOURCE          Mass point sources and sinks.
UV_VIS2             Harmonic mixing of momentum.
VAR_RHO_2DCTION     Variable density barotropic mode.dvection of 3D momentum.
UV_C4VADVECTION     Fourth-order centered vertical advection of momentum.
Process Information:Quadratic bottom stress.

Any help will be greatly appreciated,

Rodrigo.

[arango]

Yes, that makes sense. In order to restart an application with ROMS 3.6, the restart file needs to be created with ROMS 3.6 or higher. The global attribute NLM_LBC is new to ROMS 3.6. This is discussed in the release notes.

You can still use your old restart file but as initial conditions. The restart file usually have two records. I will have to check which record will use during the initialization option. I think that it uses the first record. You may want to create an initial conditions with the desired restart record. There are lot of NetCDF tools to do so.

[rduran]

Hernan,

Thank you for your time/answer.

Both the HIS and RST files I used were created with ROMS 3.6. This has been an "all-ROMS3.6" run.

Indeed this new global attribute is found with ncdump here is what it looks like:

Restart file

Code: Select all

:NLM_LBC = "\n",
        "EDGE:  WEST   SOUTH  EAST   NORTH  \n",
        "zeta:  Cha    Cha    Clo    Cha    \n",
        "ubar:  Fla    Fla    Clo    Fla    \n",
        "vbar:  Fla    Fla    Clo    Fla    \n",
        "u:     RadNud RadNud Clo    RadNud \n",
        "v:     RadNud RadNud Clo    RadNud \n",
        "temp:  RadNud RadNud Clo    RadNud \n",
        "salt:  RadNud RadNud Clo    RadNud \n",
        "tke:   Rad    Rad    Clo    Rad" ;

History file is identical to above.



Hernan said in the 3.6 release:

I highly recommend you that you always check the standard output file or the global header (NLM_LBC attribute) in the output NetCDF files to confirm that your applications have the expected lateral boundary conditions

If I compare the boundary conditions in the .out file (in my first post above) they are identical to the boundary conditions in the NLM_LBC attribute (HIS and RST) and they are identical to the LBC is in the .in file so I am not sure why ROMS isn't comfortable with that.

Could it be the " character at the end after the last TKE Rad? Doesn't seem like it but I checked for tabs and I am only using vi so Im not sure what it could be.
The error might be indicating a weird character written into RST file:

Code: Select all

LBC_GETATT - inconsistent northern boundary for variable: tke  Keyword = Rad^P<81>v
             in input file:  /home/server/student/homes/rduran/MODEL/INITIAL/RST00.nc

Thanks again,

Rodrigo

[arango]

I don't know where is getting that extra character at the end of TKE fields. There is actually no termination character at the end of the attribute. Although the radiation boundary conditions for the TKE fields are implemented, it is not wise to use them. You may get a lot of unexpected behavior at the open boundary. The vertical parametrization is highly nonlinear. Gradient conditions will be adequate even if you have open boundary conditions.

What computer architecture are you running? I expect that it is Linux.

I tested the restart on the Linux and Macs and is fine for me.

[rduran]

Thanks for the tip I will change from using Rad for TKE.

Code: Select all

wyam.OCE.ORST.EDU 104: uname -a
Linux wyam.OCE.ORST.EDU 2.6.9-55.ELlargesmp #1 SMP Fri Apr 20 16:46:56 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

[arango]

Ok, check your input script ocean.in to see if you added weird characters when editing. I am planning to add a CPP option so users can turn off the consistency check with the global attribute. Then, it is the responsibility of the user to check. You may comment the call to get_lbc in your C-preprocessed file Build/get_state.f90. Then, you need to recompile with: build.sh -noclean

[rduran]

First I tried erasing all spaces and introducing them one by one with spacebar in the boundary conditions of my .in file, and I got the same error, it does seem that the character has been introduced somewhere else.
Please note I have exclusively used vi editor in linux to work with my .in file

Second I rebuild noclean after commenting the lines you mentioned and the model is running!!

Thank you Hernan very much!!!

[rduran]

Im not sure whether this would introduce a character (as mentioned above) but I got this warning when compiling ROMS 3.6 so I thought I would mention it in case it is helpful:

Code: Select all

ROMS/Utility/state_addition.F > /home/server/student/homes/rduran/ore/Build/state_addition.f90
ROMS/Utility/state_addition.F:735: warning: extra tokens at end of #undef directive
ROMS/Utility/state_addition.F:736: warning: extra tokens at end of #undef directive
ROMS/Utility/state_addition.F:737: warning: extra tokens at end of #undef directive
ROMS/Utility/state_addition.F:738: warning: extra tokens at end of #undef directive
ROMS/Utility/state_addition.F:739: warning: extra tokens at end of #undef directive
ROMS/Utility/state_addition.F:740: warning: extra tokens at end of #undef directive

[arango]

I was not able to reproduce your problem. However, I updated the code and made the routine lbc_getatt more robust. See the following trac ticket for more information. I don't know if this will fix your problem. Please let me know.

Notice that I also introduced a new C-preprocessing option NO_LBC_ATT to avoid checking the lateral boundary conditions for consistency with the restart file.

[rduran]

Hernan, will do in about two weeks.

[rduran]

Hernan,

Recap: When I build -noclean after commenting the lines as suggested above things work fine.

Follow up experiments:
I updated the code and compiled the same case and tried running with no further modifications, now I get the following error:

Code: Select all

LBC_GETATT - error while reading global attribute:  NLM_LBC
             in restart file:  /home/server/student/homes/rduran/MODEL/INITIAL/ore2005_335_labels.nc
             call from:  get_state.F
             Probably global attribute was not found ...
             restart file needs to be generated by ROMS version 3.6 or higher

Finally I compiled again after adding a line

Code: Select all

#define NO_LBC_ATT

to my cppdefs
and tried running that ... success!

I would suggest adding a line to the error saying:
Alternatively, consider using NO_LBC_ATT option in your CPPDEFS

[rduran]

HI just want to mention that I tried to restart with a restart file that was created with ROMS 3.6 and again, as above, I am getting problems with characters that don't exist:

Code: Select all

LBC_GETATT - inconsistent northern boundary for variable: tke  Keyword = Gra<90>§ß^N
              in input file:  /home/server/student/homes/rduran/MODEL/INITIAL/RST00.nc

After doing

Code: Select all

:set invlist

in vim to be able to see all invisible characters, here is the corresponding line in my .in file:

Code: Select all

LBC(isMtke) ==   Gra     Gra     Clo     Gra         ! mixing TKE$

so nothing there.

I will have to compile again using NO_LBC_ATT

[arango]

I don't understand why. I have not made changes to this logic since Oct 18, 2011 in revision 572. This is telling me that you incorporated illegal character(s) in your ocean.in or the NetCDF version that you are using incorporate it when building the NLM_LBC attribute.

I can see in your post above some $ characters. Where are they come from?

[rduran]

Hernan,

The command

Code: Select all

:set invlist

in editor Vim shows any invisible characters, the $ is for "enter" (or next line), the point there is that there are no other characters in the in file, if you read a previous post in this thread I have checked that carefully. Re netcdf, then I do not know how to check for problems there.

[kate]

The problem is not in the ocean.in file but rather in the restart file. The routine reporting trouble is lbc_getatt which does:

Code: Select all

!  If restart,  this routine reads lateral boundary conditions         !
!  keywords strings from restart NetCDF file global attribute.         !
!  Then, it checks their values with those provided from input         !
!  script for consistency.

If you do an "ncdump -h" of your restart file, you might see a global attribute like this:

Code: Select all

                :NLM_LBC = "\n",
                        "EDGE:  WEST   SOUTH  EAST   NORTH  \n",
                        "zeta:  Cha    Clo    Cha    Cha    \n",
                        "ubar:  Fla    Clo    Fla    Fla    \n",
                        "vbar:  Fla    Clo    Fla    Fla    \n",
                        "u:     RadNud Clo    RadNud RadNud \n",
                        "v:     RadNud Clo    RadNud RadNud \n",
                        "temp:  RadNud Clo    RadNud RadNud \n",
                        "salt:  RadNud Clo    RadNud RadNud \n",
                        "tke:   Clo    Clo    Clo    Clo" ;

What do you get?

[rduran]

Hi!

Here is a copy-paste of what I get with

Code: Select all

ncdump -h RST00.nc

Code: Select all

:NLM_LBC = "\n",
			"EDGE:  WEST   SOUTH  EAST   NORTH  \n",
			"zeta:  Cha    Cha    Clo    Cha    \n",
			"ubar:  Fla    Fla    Clo    Fla    \n",
			"vbar:  Fla    Fla    Clo    Fla    \n",
			"u:     RadNud RadNud Clo    RadNud \n",
			"v:     RadNud RadNud Clo    RadNud \n",
			"temp:  RadNud RadNud Clo    RadNud \n",
			"salt:  RadNud RadNud Clo    RadNud \n",
			"tke:   Gra    Gra    Clo    Gra" ;

... I get the same as I got the previous time (see 3rd post above) except that I changed the BC for TKE from Rad to Gra per Hernan's advice. Point is: no weird characters there either

[kate]

It's a mystery. I would watch it in the debugger.

Is there a difference between Clo and Gra for something like tke on tracer points?

[wangzc]

hi,
I want to restart form rst.nc. Error:
LBA_GETATT- error while reading global attribute: NLM_LBC
in restart file: ./output/ocean_rst.nc
...
Then,i check restart file and history file.

:NLM_LBC = "\n",
"EDGE: WEST SOUTH EAST NORTH \n",
"zeta: Cha Cha Clo Cha \n",
"ubar: Fla Fla Clo Fla \n",
"vbar: Fla Fla Clo Fla \n",
"u: RadNud RadNud Clo RadNud \n",
"v: RadNud RadNud Clo RadNud \n",
"temp: RadNud RadNud Clo RadNud \n",
"salt: RadNud RadNud Clo RadNud \n",
"tke: Gra Gra Clo Gra" ;

Nothing above can be found, why? ROMS version is 3.6, and, PERFECT_RESTART is defined.