Ocean Modeling Discussion

ROMS/TOMS

Search for:
It is currently Mon Jun 17, 2019 11:08 pm




Post new topic Reply to topic  [ 8 posts ] 

All times are UTC

Author Message
PostPosted: Wed Apr 25, 2018 3:47 pm 
Offline

Joined: Thu Mar 08, 2018 2:47 am
Posts: 62
Location: German Research Centre for Geosciences
Hi everyone,

after having tried to find the source of error myself during the whole day, I resign and ask for help here:

I want to compile a test case for parallel processing and use openmpi and ifort. I already switched on USE_MPI, of course, and also pointed to mpiifort instead of mpif90 in the Compiler file due to the default gfortan in mpif90. In the build file I point to the mpi library (parallel fortran netcdf open mpi) and using Netcdf4, I also point to the respective nf-config.
After a while of compilation I receive tons of "undefined reference" errors regarding MPI, mpi or ompi variables.
The errors occur regarding four library links:
libhdf5.so
libnetcdf.so
libpnetcdf.so
libmpi_mpifh.so.12

I am already so confused about where to put which library link that it may well be that I just completely lost track on where I put what. So if someone could supply me with a "map" of where to put which pointer in the build and the Compiler file, or a "receipe" for mpiifort usage, he or she would save my week. :roll:


Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 4:23 pm 
Offline
User avatar

Joined: Wed Jul 02, 2003 5:29 pm
Posts: 3599
Location: IMS/UAF, USA
The supercomputer I use has a "modules" feature. Depending on which modules I load, I can either be in an ifort universe or a gfortran universe. Both want the mpi compiler to be called mpif90. For one:
Quote:
chinook04 674% which mpif90
/opt/scyld/openmpi/1.10.7/intel/bin/mpif90
and for the other:
Quote:
chinook04 329% which mpif90
/usr/local/pkg/mpi/OpenMPI/1.10.3-GCC-5.4.0-2.26/bin/mpif90
Either way, I want
Quote:
USE_MPIF90 ?= on

Anyway, if your computer doesn't have these modules, you want to give the full path for the appropriate mpif90.


Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 8:55 pm 
Offline

Joined: Thu Mar 08, 2018 2:47 am
Posts: 62
Location: German Research Centre for Geosciences
Hi Kate!
Your hint was very valuable, I managed to find the module which provides the openmpi mpif90 environment for intel! Thank you! :D
(For users having the same issue: typing something like "module whatis" gives additonal descriptions of the modules and makes it easier to identify the right one. :idea: )
This ...almost saved my week because now I receive these three library conflict warnings:
Quote:
ld: warning: libmpi_usempi_ignore_tkr.so.6, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi_usempi_ignore_tkr.so.0
ld: warning: libmpi_mpifh.so.12, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi_mpifh.so.2
ld: warning: libmpi.so.12, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi.so.1

I figured that libmpi_usempi_ignore_tkr.so.0, libmpi_mpifh.so.2 and libmpi.so.1 are set by the openmpi library. I have this one now (the only appropiate one I found):
Quote:
which mpif90
/sw/rhel6-x64/mpi/openmpi-1.8.4-intel14/bin/mpif90

So apparently mpif90 and fortran netcdf for openmpi do not match. I don't really have many options to choose here for this combination, though. Maybe I am still misunderstanding something... :?:


Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 9:04 pm 
Offline
User avatar

Joined: Wed Jul 02, 2003 5:29 pm
Posts: 3599
Location: IMS/UAF, USA
They are only warnings. Did it build an executable? Can you try running it?

Also, you want to make sure netcdf-fortran was compiled with the same compiler you are using to compile ROMS. Always.


Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 9:17 pm 
Offline

Joined: Thu Mar 08, 2018 2:47 am
Posts: 62
Location: German Research Centre for Geosciences
I just solved it myself. I saw that you have a much newer openmpi so I looked again and found openmpi-1.10.7_hpcx-intel14. I thought it would be a wrong one since it comes with the hpcx suffix but the compiling worked without error. So thank you again! :D
HOWEVER, I only have an oceanG and no oceanM. :?: I want to run the DAMEE_4 test case. I know this one is old but it's only for computation estimates... This one is made for parallel processing, too, right? It comes with default 2 by 2 tiles domain decomposition so I thought it should work in parallel mode, too...


Last edited by c.drinkorn on Wed Apr 25, 2018 9:23 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 9:18 pm 
Offline

Joined: Thu Mar 08, 2018 2:47 am
Posts: 62
Location: German Research Centre for Geosciences
kate wrote:
They are only warnings. Did it build an executable? Can you try running it?

Also, you want to make sure netcdf-fortran was compiled with the same compiler you are using to compile ROMS. Always.


I missed this one... Ok, well it built an oceanG. But I was expecting an oceanM. :|

Regarding the netcdf fortran library: Yes, I use the openmpi parallel netcdf fortran library.


Top
 Profile  
Reply with quote  
PostPosted: Wed Apr 25, 2018 10:41 pm 
Offline
User avatar

Joined: Wed Jul 02, 2003 5:29 pm
Posts: 3599
Location: IMS/UAF, USA
The oceanG means you compiled it with USE_DEBUG=on. You get oceanG with or without MPI when in debug mode.


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 26, 2018 7:34 am 
Offline

Joined: Thu Mar 08, 2018 2:47 am
Posts: 62
Location: German Research Centre for Geosciences
Yes, I know. But I did comment out USE_DEBUG=on...
However, I found that in the makefile it was not. When I did it I got an oceanM. Happy end. :D
Thank you for your help, Kate!


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group