faster MPI on single computer / -bind-to core /Ubuntu 18.04

General scientific issues regarding ROMS

Moderators: arango, robertson

Post Reply
Posts: 86
Joined: Sun Jul 27, 2003 6:49 pm
Location: UNH, USA

faster MPI on single computer / -bind-to core /Ubuntu 18.04

#1 Post by jpringle » Fri Apr 27, 2018 5:37 pm

Dear all --

I hope some of you find this useful. I am running ROMS on a dual processor 32 core AMD EPYC machine. I noticed on the latest version of Ubuntu my smaller machines showed good improvement with ROMS runs with openMP parallelization.

So I upgraded the big machine to Ubuntu 18.04, and my ROMS runs became about 15% slower. This new version has a new kernel (4.15) which is supposed to allocate processes to cores in a way that works better with the EPYC memory architecture (it is NUMA with 16 memory channels). It is not doing so by default.

I returned to the old performance (actually, a few percent faster) by invoking mpirun with the "-bind-to core" option -- I run the model with "mpirun -bind-to core -np 32 ./oceanM"

Hope this helps someone,

Post Reply