Error with Multifiles

Bug reports, work arounds and fixes

Moderators: arango, robertson

Post Reply
Message
Author
yang
Posts: 56
Joined: Mon Sep 25, 2006 2:37 pm
Location: Institue of oceanology ,Chinese acedemy of scinece

Error with Multifiles

#1 Unread post by yang »

Wind forcing files in *.in file are:

<<***********************
../00_ECS_hindcast_forcing_2000-2013/ECS_2000_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2001_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2002_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2003_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2004_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2005_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2006_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2007_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2008_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2009_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2010_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2011_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2012_Wind_stress@ERA_interim_12hourly.nc ! forcing file 1, grid 1
************************>>

If i use 13 wind forcing files (from *2000.nc to 2012.nc) as multifiles seperated by "|", there is an error as follows.

<<***********************
GET_3DFLD - v-momentum component climatology, t = 15 00:00:00
(Rec=0001, Index=2, File: ECS_hindcast_clm.nc)
(Tmin= 15.0000 Tmax= 349.0000)
(Min = -2.07797614E+00 Max = 4.44236817E+00)

STEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed

0 0 00:00:00 4.738806E-03 2.120127E+04 2.120127E+04 5.919756E+15
(072,119,26) 7.825559E-02 5.547537E-02 0.000000E+00 2.139181E+00
DEF_HIS - creating history file: ./ECS_CT_2000_2013_results/ECS_5x5_CT_2000-2013_his_0001.nc

DEF_VAR - unable to define variable: lon_psi
in NetCDF file: ./ECS_CT_2000_2013_results/ECS_5x5_CT_2000-2013_his_0001.nc

Elapsed CPU time (seconds):

Node #124 CPU: 10.505
Node #126 CPU: 10.505
Node #116 CPU: 10.513
************************>>

However, if i use 12 wind forcing files (from *2000.nc to 2011.nc), everything is ok.


Wind forcing files in *.in file are:

<<***********************
../00_ECS_hindcast_forcing_2000-2013/ECS_2000_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2001_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2002_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2003_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2004_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2005_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2006_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2007_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2008_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2009_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2010_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2011_Wind_stress@ERA_interim_12hourly.nc ! forcing file 1, grid 1
************************>>

The model runs well:


<<***********************
(Tmin= 15.0000 Tmax= 349.0000)
(Min = 2.92174696E+01 Max = 3.49169623E+01)
GET_3DFLD - u-momentum component climatology, t = 15 00:00:00
(Rec=0001, Index=2, File: ECS_hindcast_clm.nc)
(Tmin= 15.0000 Tmax= 349.0000)
(Min = -2.86596983E+00 Max = 3.87822682E+00)
GET_3DFLD - v-momentum component climatology, t = 15 00:00:00
(Rec=0001, Index=2, File: ECS_hindcast_clm.nc)
(Tmin= 15.0000 Tmax= 349.0000)
(Min = -2.07797614E+00 Max = 4.44236817E+00)

STEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed

0 0 00:00:00 4.738806E-03 2.120127E+04 2.120127E+04 5.919756E+15
(072,119,26) 7.825559E-02 5.547537E-02 0.000000E+00 2.139181E+00
DEF_HIS - creating history file: ./ECS_CT_2000_2013_results/ECS_5x5_CT_2000-2013_his_0001.nc
WRT_HIS - wrote history fields (Index=1,1) into time record = 0000001

10 0 01:00:00 4.756717E-03 2.119918E+04 2.119918E+04 5.919355E+15
(205,162,26) 3.689252E-03 6.615482E-03 5.412607E+00 1.809340E+00
20 0 02:00:00 4.798577E-03 2.119832E+04 2.119833E+04 5.919201E+15
************************>>

The file of "../00_ECS_hindcast_forcing_2000-2013/ECS_2012_Wind_stress@ERA_interim_12hourly.nc" is correct, because the model runs well when fored by the following twelve files.

<<***********************
../00_ECS_hindcast_forcing_2000-2013/ECS_2000_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2001_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2002_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2003_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2004_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2005_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2006_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2007_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2008_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2009_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2010_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2012_Wind_stress@ERA_interim_12hourly.nc ! forcing file 1, grid 1
************************>>


There likely is a 12 limitation. When the munber of multiflie is larger than 12, the model can not read it correctly.

Any suggetions will be appreciated.
Last edited by yang on Tue Sep 17, 2013 1:01 am, edited 1 time in total.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: Error with Multifiles

#2 Unread post by kate »

The code that loads up data structures with the 12 or 13 forcing file names is in inp_par.F, in a routine called load_s2d. I don't see anything there that should be a problem at 13 files, but Fname is dimensioned by 100, so more than 100 forcing files would overflow. Do you have 13 each of Uwind, Vwind, Tair, etc?

yang
Posts: 56
Joined: Mon Sep 25, 2006 2:37 pm
Location: Institue of oceanology ,Chinese acedemy of scinece

Re: Error with Multifiles

#3 Unread post by yang »

kate wrote:The code that loads up data structures with the 12 or 13 forcing file names is in inp_par.F, in a routine called load_s2d. I don't see anything there that should be a problem at 13 files, but Fname is dimensioned by 100, so more than 100 forcing files would overflow. Do you have 13 each of Uwind, Vwind, Tair, etc?
Thanks, Kate.

The number of my forcing files is obviously less than 100 as shown below. I am looking through the code. What kind of error will give error information like "DEF_VAR - unable to define variable: lon_psi; in NetCDF file: ./ECS_CT_2000_2013_results/ECS_5x5_CT_2000-2013_his_0001.nc".

NFFILES == 09 ! number of unique forcing files
FRCNAME == ../00_ECS_hindcast_forcing_2000-2013/ECS_swf@OAFLUX_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_swr@OAFLUX_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_shf@OAFLUX_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_dQdSST@OAFLUX_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_SST@Pathfinder_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_SSS@OAFLUX_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_Discharge@Changjiang_monthly.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_tide_2000_v3.nc \ ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2000_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2001_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2002_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2003_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2004_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2005_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2006_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2007_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2008_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2009_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2010_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2011_Wind_stress@ERA_interim_12hourly.nc | ! forcing file 1, grid 1
../00_ECS_hindcast_forcing_2000-2013/ECS_2012_Wind_stress@ERA_interim_12hourly.nc ! forcing file 1, grid 1

Any suggestion is welcome.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: Error with Multifiles

#4 Unread post by kate »

It's an output error and you say it is related to changes in input. Quite mysterious, which is why I like to watch these things in the debugger.

k.alexander
Posts: 54
Joined: Tue Jun 28, 2016 2:08 pm
Location: CCRC (UNSW), ARCCSS, ACE CRC

Re: Error with Multifiles

#5 Unread post by k.alexander »

I just got exactly the same error. Did you ever solve it?

yang
Posts: 56
Joined: Mon Sep 25, 2006 2:37 pm
Location: Institue of oceanology ,Chinese acedemy of scinece

Re: Error with Multifiles

#6 Unread post by yang »

Not yet. I have not solved this problem. If you figure this out, let me know. Thanks in advance.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: Error with Multifiles

#7 Unread post by kate »

def_info.F is what writes the names of the forcing files into the global attributes. There's a string variable which has length 1024 - it could be that you're overflowing that in the call to join_string for all the forcing files of one type. Try changing the size of string to 2048 or so.

yang
Posts: 56
Joined: Mon Sep 25, 2006 2:37 pm
Location: Institue of oceanology ,Chinese acedemy of scinece

Re: Error with Multifiles

#8 Unread post by yang »

kate wrote:def_info.F is what writes the names of the forcing files into the global attributes. There's a string variable which has length 1024 - it could be that you're overflowing that in the call to join_string for all the forcing files of one type. Try changing the size of string to 2048 or so.
Thanks kate. According to your suggestion, this problem can be well solved. Because of the length limit of 1024, the number of forcing files can not be large enough to include many years forcing files. As noted by Kate, the error arises when def_info.F writes the names of the forcing files into the global attributes. To deal with this problem, we can change the 1024 to a appropriate number matching the numerical simulation.

k.alexander
Posts: 54
Joined: Tue Jun 28, 2016 2:08 pm
Location: CCRC (UNSW), ARCCSS, ACE CRC

Re: Error with Multifiles

#9 Unread post by k.alexander »

Thanks Kate, this worked for me too. Interestingly, the error showed up differently for me once I'd switched to 3.7 - I got this instead: viewtopic.php?f=14&t=3749 But the root cause was the same, and I fixed it by increasing the character limit on "string".

User avatar
arango
Site Admin
Posts: 1347
Joined: Wed Feb 26, 2003 4:41 pm
Location: DMCS, Rutgers University
Contact:

Re: Error with Multifiles

#10 Unread post by arango »

Yes, thank you for reporting this. It is difficult to guess the length of that string. I set it to 4096. See the following :arrow: trac ticket for more information.

Post Reply