WRF quilting support in the WRF-ROMS setup in COAWST

General scientific issues regarding ROMS

Moderators: arango, robertson

Post Reply
Message
Author
Serega.Osipov
Posts: 5
Joined: Thu Oct 01, 2015 4:41 am
Location: KAUST

WRF quilting support in the WRF-ROMS setup in COAWST

#1 Post by Serega.Osipov » Sat Apr 14, 2018 10:33 am

I'm having problems running WRF with quilting specifically in the COAWST setup.
Normally, when running WRF with a large number of cores (such as 1024 and more) saving NetCDF output becomes slow (about 30 sec). This is easily resolved by using quilting (by setting namelist parameters nio_tasks_per_group=8(16) and nio_groups=2(4,6)). This reduces the saving time to about 0.03 seconds which is significant speed up.
I've tested pure WRF with this quilting settings and it is running fine. However, when WRF is coupled to ROMS, turning on quilting causes the run to hang indefinitely (no errors, doesn't crash). I suspect that MCT expects cores to communicate and since some of them in case of quilting are dedicated for the I/O, this causes the hanging. I've also tried to specify nproc_x and nproc_y, but it didn't help.

Is it possible to run WRF-ROMS setup with WRF quilting option enabled?

jcwarner
Posts: 876
Joined: Wed Dec 31, 2003 6:16 pm
Location: USGS, USA

Re: WRF quilting support in the WRF-ROMS setup in COAWST

#2 Post by jcwarner » Sun Apr 22, 2018 10:54 pm

when the wrf + other-model job is submitted, the total number of cpus goes through an mpi_comm_split to allocate the procs for wrf. Then each wrf proc does an MCT init so all the processors for wrf will look to communicate through mct. I dont think the current configuration will allow quilting, but i can run a test to see. i am not sure exactly how wrf determines which procs go to quilting and which to computations. i can take a look but it will be some time til i get to it.

Post Reply