Trouble in shallow water

Report or discuss software problems and other woes

Moderators: arango, robertson

Post Reply
Message
Author
User avatar
kate
Posts: 3780
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Trouble in shallow water

#1 Post by kate » Wed Mar 26, 2008 10:58 pm

I've been trying to set up a run that includes an area with large tides. It's been interesting and it continues to be interesting. We've added the WET_DRY option and fixed the biology blow-ups. However, there are still intermittent problems I don't understand. I put in some stations in order to monitor the previous tidal blow ups and I've managed to capture something odd. Here is the total water column depth for the "bad" station and some of its neighbors:
Image
What follows next are three different surface temperature timeseries for exactly the same executable, three different runs:
Image
Image
Image
The last of these caused the model to blow up while the first two did not. I believe the time and place of the trouble would have a negative water depth without the WET_DRY option. Is there some place in the tracer code that needs to check against the wet-dry mask? I'm not even sure that mask is reading as dry at the time of the 3D timestep. Has anyone else seen anything like this?

jcwarner
Posts: 855
Joined: Wed Dec 31, 2003 6:16 pm
Location: USGS, USA

#2 Post by jcwarner » Thu Mar 27, 2008 12:45 am

We are spending some more time on the wet/dry option. I was having some issues with boundary conditions and need to finalize that and submit to Hernan. But it looks like you are having other issues. I am not sure that it is related to the wet/dry stuff. Right now wet/dry is limited to be a barotropic process. As the water level in a cell becomes shallow, the transport out of the cell will be prevented if the total depth < Dcrit. We think that there should also be an effect on the baroclinic velcoities but there are not any effects right now. We are looking into that. But at very shallow depths the baroclinic flows are typically small. Is there a large wind stress at this time? What does it mean that there are 3 different time series for the same executable?

User avatar
kate
Posts: 3780
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

#3 Post by kate » Thu Mar 27, 2008 1:09 am

Good question on the wind stress - yes, it's huge in both u(-) and v(+) directions at that one time and place. Those plots look identical between the first two cases though. Does the wind stress get big when the water depth gets that small?

I don't know what it means that I get different answers for different runs. Those were all with the Sun compiler on Linux. A Pathscale run just now matches the second plot. A PGI run blows up much earlier (twice) and I've got one in the queue on the IBM.

Does the model do anything different after writing a restart record? The only difference between the first two plots is that I asked for a restart record just before the crash on one of them. The other one matches the daily restart frequency of the bad run.

User avatar
jivica
Posts: 135
Joined: Mon May 05, 2003 2:41 pm
Location: The University of Western Australia, Perth, Australia

#4 Post by jivica » Thu Mar 27, 2008 10:20 am

What have you used for optimization of compiler?
We had problems when using too aggressive -03 option (default) and had to put -O2
just idea...
Ivica

User avatar
kate
Posts: 3780
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

#5 Post by kate » Fri Mar 28, 2008 11:25 pm

Cutting down to -O1 or -O2 (Pathscale) gives a repeatable temperature spike of 4 degrees. Running on the IBM gives no big spike:
Image
Saving a restart file just before and running from that doesn't give a spike either. Come to think of it, the IBM job was restarted after it ran into a 2 GB file size problem. Here's the 4 degree spike and the corresponding temperature of the under-ice log layer:
Image
Image
I was thinking it might be something to do with the log layer code, but then all the bio tracers get a spike too.

User avatar
kate
Posts: 3780
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

#6 Post by kate » Mon Mar 31, 2008 5:50 pm

I haven't figured out what is going wrong. The IBM without restart gives a negative spike, by the way. Meanwhile, trying Dcrit = 0.5 instead of 0.1 seems to work so that will be my working solution. The mystery remains, however.

Post Reply