Compiling using MPI anf ifort -> mpiifort

Discussion on computers, ROMS installation and compiling

Moderators: arango, robertson

Post Reply
Message
Author
c.drinkorn
Posts: 83
Joined: Thu Mar 08, 2018 2:47 am
Location: German Research Centre for Geosciences

Compiling using MPI anf ifort -> mpiifort

#1 Post by c.drinkorn » Wed Apr 25, 2018 3:47 pm

Hi everyone,

after having tried to find the source of error myself during the whole day, I resign and ask for help here:

I want to compile a test case for parallel processing and use openmpi and ifort. I already switched on USE_MPI, of course, and also pointed to mpiifort instead of mpif90 in the Compiler file due to the default gfortan in mpif90. In the build file I point to the mpi library (parallel fortran netcdf open mpi) and using Netcdf4, I also point to the respective nf-config.
After a while of compilation I receive tons of "undefined reference" errors regarding MPI, mpi or ompi variables.
The errors occur regarding four library links:
libhdf5.so
libnetcdf.so
libpnetcdf.so
libmpi_mpifh.so.12

I am already so confused about where to put which library link that it may well be that I just completely lost track on where I put what. So if someone could supply me with a "map" of where to put which pointer in the build and the Compiler file, or a "receipe" for mpiifort usage, he or she would save my week. :roll:

User avatar
kate
Posts: 3718
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: Compiling using MPI anf ifort -> mpiifort

#2 Post by kate » Wed Apr 25, 2018 4:23 pm

The supercomputer I use has a "modules" feature. Depending on which modules I load, I can either be in an ifort universe or a gfortran universe. Both want the mpi compiler to be called mpif90. For one:
chinook04 674% which mpif90
/opt/scyld/openmpi/1.10.7/intel/bin/mpif90
and for the other:
chinook04 329% which mpif90
/usr/local/pkg/mpi/OpenMPI/1.10.3-GCC-5.4.0-2.26/bin/mpif90
Either way, I want
USE_MPIF90 ?= on
Anyway, if your computer doesn't have these modules, you want to give the full path for the appropriate mpif90.

c.drinkorn
Posts: 83
Joined: Thu Mar 08, 2018 2:47 am
Location: German Research Centre for Geosciences

Re: Compiling using MPI anf ifort -> mpiifort

#3 Post by c.drinkorn » Wed Apr 25, 2018 8:55 pm

Hi Kate!
Your hint was very valuable, I managed to find the module which provides the openmpi mpif90 environment for intel! Thank you! :D
(For users having the same issue: typing something like "module whatis" gives additonal descriptions of the modules and makes it easier to identify the right one. :idea: )
This ...almost saved my week because now I receive these three library conflict warnings:
ld: warning: libmpi_usempi_ignore_tkr.so.6, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi_usempi_ignore_tkr.so.0
ld: warning: libmpi_mpifh.so.12, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi_mpifh.so.2
ld: warning: libmpi.so.12, needed by /sw/rhel6-x64/netcdf/netcdf_fortran-4.4.3-parallel-openmpi-intel14/lib/libnetcdff.so, may conflict with libmpi.so.1
I figured that libmpi_usempi_ignore_tkr.so.0, libmpi_mpifh.so.2 and libmpi.so.1 are set by the openmpi library. I have this one now (the only appropiate one I found):
which mpif90
/sw/rhel6-x64/mpi/openmpi-1.8.4-intel14/bin/mpif90
So apparently mpif90 and fortran netcdf for openmpi do not match. I don't really have many options to choose here for this combination, though. Maybe I am still misunderstanding something... :?:

User avatar
kate
Posts: 3718
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: Compiling using MPI anf ifort -> mpiifort

#4 Post by kate » Wed Apr 25, 2018 9:04 pm

They are only warnings. Did it build an executable? Can you try running it?

Also, you want to make sure netcdf-fortran was compiled with the same compiler you are using to compile ROMS. Always.

c.drinkorn
Posts: 83
Joined: Thu Mar 08, 2018 2:47 am
Location: German Research Centre for Geosciences

Re: Compiling using MPI anf ifort -> mpiifort

#5 Post by c.drinkorn » Wed Apr 25, 2018 9:17 pm

I just solved it myself. I saw that you have a much newer openmpi so I looked again and found openmpi-1.10.7_hpcx-intel14. I thought it would be a wrong one since it comes with the hpcx suffix but the compiling worked without error. So thank you again! :D
HOWEVER, I only have an oceanG and no oceanM. :?: I want to run the DAMEE_4 test case. I know this one is old but it's only for computation estimates... This one is made for parallel processing, too, right? It comes with default 2 by 2 tiles domain decomposition so I thought it should work in parallel mode, too...
Last edited by c.drinkorn on Wed Apr 25, 2018 9:23 pm, edited 1 time in total.

c.drinkorn
Posts: 83
Joined: Thu Mar 08, 2018 2:47 am
Location: German Research Centre for Geosciences

Re: Compiling using MPI anf ifort -> mpiifort

#6 Post by c.drinkorn » Wed Apr 25, 2018 9:18 pm

kate wrote:They are only warnings. Did it build an executable? Can you try running it?

Also, you want to make sure netcdf-fortran was compiled with the same compiler you are using to compile ROMS. Always.
I missed this one... Ok, well it built an oceanG. But I was expecting an oceanM. :|

Regarding the netcdf fortran library: Yes, I use the openmpi parallel netcdf fortran library.

User avatar
kate
Posts: 3718
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: Compiling using MPI anf ifort -> mpiifort

#7 Post by kate » Wed Apr 25, 2018 10:41 pm

The oceanG means you compiled it with USE_DEBUG=on. You get oceanG with or without MPI when in debug mode.

c.drinkorn
Posts: 83
Joined: Thu Mar 08, 2018 2:47 am
Location: German Research Centre for Geosciences

Re: Compiling using MPI anf ifort -> mpiifort

#8 Post by c.drinkorn » Thu Apr 26, 2018 7:34 am

Yes, I know. But I did comment out USE_DEBUG=on...
However, I found that in the makefile it was not. When I did it I got an oceanM. Happy end. :D
Thank you for your help, Kate!

Post Reply