ROMS not bit-reproducible for certain tilings

Bug reports, work arounds and fixes

Moderators: arango, robertson

Post Reply
Message
Author
k.alexander
Posts: 54
Joined: Tue Jun 28, 2016 2:08 pm
Location: CCRC (UNSW), ARCCSS, ACE CRC

ROMS not bit-reproducible for certain tilings

#1 Post by k.alexander » Mon Mar 27, 2017 5:13 am

Since updating from 3.6 to the trunk (svn version 820), my configuration of ROMS is no longer bit-reproducible for certain tilings. For example 1x1, 2x2, 16x8 all give identical output (for multiple simulations with the same tiling as well as comparing different tilings) but 32x1, 32x16 diverge. See the full discussion here: viewtopic.php?f=14&t=4470 I am not the only one with this problem (see post by Serega.Osipov on the discussion page) so I don't think it's anything specific to my setup. I don't see any updates since svn 820 that look relevant to this problem, although I haven't yet updated to confirm.

k.alexander
Posts: 54
Joined: Tue Jun 28, 2016 2:08 pm
Location: CCRC (UNSW), ARCCSS, ACE CRC

Re: ROMS not bit-reproducible for certain tilings

#2 Post by k.alexander » Mon Apr 10, 2017 2:33 am

I have confirmed that the same problem exists for the newest version of the code (as of today) with none of my own modifications (CICE coupling, ice shelves, etc.) See viewtopic.php?f=14&t=4470&p=17405#p17405.

k.alexander
Posts: 54
Joined: Tue Jun 28, 2016 2:08 pm
Location: CCRC (UNSW), ARCCSS, ACE CRC

Re: ROMS not bit-reproducible for certain tilings

#3 Post by k.alexander » Tue Apr 11, 2017 4:55 am

I have fixed the problem by switching from openmpi 1.8.4 to openmpi 1.10.2. For ROMS 3.6, openmpi 1.8.4 was bit-reproducible. So something about the new kernel is now sensitive to the version of openmpi. Perhaps there should be a warning to users somewhere in the makefile comments to check their version of openmpi.

Post Reply