id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
630	IMPORTANT: Corrected shared-memory bug	arango	arango	"A shared-memory bug was corrected for variables '''ntstart(ng)''', '''ntfirst(ng)''', and '''ntend(ng)'''. Because of nesting, these variables were declared as '''THREADPRIVATE'''. However, this introduced a parallel bug when calling '''get_state''' to read initial conditions or restart fields. The routine '''get_state''' is only executed by the master thread. As a consequence, the above variables are only initialized in the master thread and their values are unknown to other shared-memory threads.  Similar problem occurs for the '''time(ng)''' variable.

The solution is to remove the '''THREADPRIVATE''' clause for such variables and initialize them by master thread as follows:
{{{
!$OMP MASTER
        ntstart(ng)=INT((time(ng)-dstart*day2sec)/dt(ng))+1
        ntend(ng)=ntimes(ng)
        ntfirst(ng)=ntstart(ng)
!$OMP END MASTER
!$OMP BARRIER
}}}
and introduce a new scalar variable '''io_time''' to update the value of '''time(ng)''' for the other shared-memory threads after initialization or restart:

{{{
#ifdef INI_FILE
!
!  Read in initial conditions from initial NetCDF file.
!
      DO ng=1,Ngrids
!$OMP MASTER
        CALL get_state (ng, iNLM, 1, INI(ng)%name,                      &
     &                  IniRec(ng), Tindex(ng))
!$OMP END MASTER
# ifdef DISTRIBUTE
        CALL mp_bcasti (ng, iNLM, exit_flag)
# endif
!$OMP BARRIER
        IF (exit_flag.ne.NoError) RETURN
        time(ng)=io_time                     ! needed for shared-memory
      END DO
#else
!
!  If restart, read in initial conditions restart NetCDF file.
!
      DO ng=1,Ngrids
        IF (nrrec(ng).ne.0) THEN
!$OMP MASTER
          CALL get_state (ng, 0, 1, INI(ng)%name,                      &
     &                    IniRec(ng), Tindex(ng))
!$OMP END MASTER
# ifdef DISTRIBUTE
          CALL mp_bcasti (ng, iNLM, exit_flag)
# endif
!$OMP BARRIER
          IF (exit_flag.ne.NoError) RETURN
          time(ng)=io_time                   ! needed for shared-memory
        END IF
      END DO
#endif
}}}

Many thanks to Mark Hadfield for [https://www.myroms.org/forum/viewtopic.php?f=19&t=3260 bringing] this to my attention. This was also [https://www.myroms.org/forum/viewtopic.php?t=3265 reported] by Mitsuhiro Kawase."	bug	closed	major	Release ROMS/TOMS 3.7	Nonlinear	3.7	Fixed