I am trying to understand why I get better times with MPI compared to openMP.
It could be something wrong with my environment/build but wanted to hear if anyone has any experiences comparing openMP and mpi to share.
When I look at the grid partitioning on mpi, I see the tile indices nicely getting chunked up for each of the mpi ranks.
Tile partition information for Grid 01: 2048x0256x0030 tiling: 008x004
tile Istr Iend Jstr Jend Npts
0 1 256 1 64 491520
1 257 512 1 64 491520
2 513 768 1 64 491520
3 769 1024 1 64 491520
4 1025 1280 1 64 491520
5 1281 1536 1 64 491520
6 1537 1792 1 64 491520
7 1793 2048 1 64 491520
8 1 256 65 128 491520
9 257 512 65 128 491520
and the Xmin, Xmax tiles
Tile minimum and maximum fractional coordinates for Grid 01:
(interior points only)
tile Xmin Xmax Ymin Ymax grid
0 0.50 256.50 0.50 64.50 RHO-points
1 256.50 512.50 0.50 64.50 RHO-points
2 512.50 768.50 0.50 64.50 RHO-points
3 768.50 1024.50 0.50 64.50 RHO-points
4 1024.50 1280.50 0.50 64.50 RHO-points
The run from the openMP build has the same values for the I,J min/max but for the X,Y min/max I get:
Tile minimum and maximum fractional coordinates for Grid 01:
(interior points only)
tile Xmin Xmax Ymin Ymax grid
0 -1.50 2051.50 0.50 258.50 RHO-points
1 -2.50 2051.50 0.50 258.50 RHO-points
2 -2.50 2051.50 0.50 258.50 RHO-points
3 -2.50 2051.50 0.50 258.50 RHO-points
4 -2.50 2051.50 0.50 258.50 RHO-points
5 -2.50 2051.50 0.50 258.50 RHO-points
6 -2.50 2051.50 0.50 258.50 RHO-points
7 -2.50 2050.50 0.50 258.50 RHO-points
8 -1.50 2051.50 -0.50 258.50 RHO-points
Why are the X,Y values covering the entire domain in the case of openMP?