OpenMP cpu stats

Post ROMS benchmark results

Moderators: arango, robertson

Post Reply
Message
Author
sspagnol
Posts: 7
Joined: Fri Oct 21, 2005 2:42 pm
Location: Australian Institute of Marine Science

OpenMP cpu stats

#1 Post by sspagnol » Fri Apr 08, 2011 1:37 am

Hi All,

For a test job using OpenMP compiled nonlinear roms with NTIMES=100, DT=60

ROMS info:
Elapsed CPU time (seconds):

Thread # 7 CPU: 561.271
Thread # 5 CPU: 561.271
Thread # 2 CPU: 561.271
Thread # 0 CPU: 561.271
Thread # 3 CPU: 561.271
Thread # 1 CPU: 561.271
Thread # 6 CPU: 561.271
Thread # 4 CPU: 561.279
Total: 4490.173
Timing info:
real 1m11.164s
user 8m27.231s
sys 0m54.202s
Now looks like 561.271/8/8 ~ 1m11.164s.

Googling terms like openmp and cpu_time and gfortran come across topics like

http://stackoverflow.com/questions/5083 ... ed-program
http://coding.derkeiler.com/Archive/For ... 00093.html

So if I mangle Utility/mp_routines.F functin my_wtime, add a

Code: Select all

#if defined _OPENMP
	use omp_lib
#endif
and replace original call to cpu_time with
#if defined _OPENMP
my_wtime=omp_get_wtime()
#else
CALL cpu_time (wtime(1))
my_wtime=wtime(1)
#endif
ROMS info:
Elapsed CPU time (seconds):

Thread # 0 CPU: 71.958
Thread # 1 CPU: 71.958
Thread # 2 CPU: 71.958
Thread # 5 CPU: 71.958
Thread # 3 CPU: 71.958
Thread # 7 CPU: 71.958
Thread # 4 CPU: 71.958
Thread # 6 CPU: 71.958
Total: 575.663
Timing info:
real 1m12.026s
user 8m30.091s
sys 0m57.976s
Now this is better in the sense that at least 71.958 sec/8 ~ 1m12.026s rather than 561.271/8/8 ~ 1m11.164s, but still seems the measuring time thing adds up all thread contributions.

So my questions are
  • [1.]Does the elapsed cpu stats work for other people using openmp and it's some wierdness with this machine/compiler [Linux box running 2.6.18-238.5.1.el5 with gcc version 4.4.4 20100726 (Red Hat 4.4.4-13)].
  • Is there a way to do this correctly in a portable fashion?
  • All this is in aid of trying to find out how to calculate a run time ratio. Would like overwrite/update a textfile along the lines of CSIRO SHOC model 'diag.txt' that is updated every (?) steps which looks like
    Simulation start = 7548.5000 (days) : 2010-09-01 12:00:00
    Simulation stop = 7638.0000 (days) : 2010-11-30 00:00:00
    Simulation time = 7638.0000 (days) : 2010-11-30 00:00:00

    CPU time used this iteration = 1.180 (sec)
    Mean CPU time used / iteration = 1.221 (sec)
    CPU run time ratio = 49.132721
    Window 1 integration time = 8.6 %
    Window 2 integration time = 8.7 %
    Window 3 integration time = 8.7 %
    Window 4 integration time = 8.8 %
    Window 5 integration time = 8.7 %
    Window 6 integration time = 8.5 %
    Window 7 integration time = 8.2 %
    Window 8 integration time = 8.1 %
    Window 9 integration time = 8.0 %
    Window 10 integration time = 8.0 %
    Window 11 integration time = 8.1 %
    Window 12 integration time = 7.7 %
    Elapsed time = 0 day(s) 09:21:31
    Total time ratio = 229.521237
    Time to completion = 0 day(s) 00:00:00
    Percent complete = 100.0%
    Run successful.
    While running it display current simulation time, estimated time to completion, and percentage done. At this stage not even sure where a mythical file/function (Utility/wrt_cpuinfo.F) would get called from, Nonlinear/output.F? And don't even know if the start time is stored anywhere.
Any advice appreciated.

Thanks,
Simon.

User avatar
arango
Site Admin
Posts: 1085
Joined: Wed Feb 26, 2003 4:41 pm
Location: IMCS, Rutgers University
Contact:

Re: OpenMP cpu stats

#2 Post by arango » Fri Apr 08, 2011 7:27 pm

It is good to know. I didn't know that behavior for F90 cpu_time intrinsic function. There is a lot of changes coming in you way in few days. I will be doing some systematic updates to ROMS. I was also really annoyed at the behavior that you described above.

Post Reply