Hi everyone,
I'm trying to compile ROMS 3 on my Intel Mac using gfortran and have it use both cores. I ran into a brick wall using OpenMP (something about threadprivate not available on the Mac platform), now I'm trying the MPICH2 path (had trouble compiling MPICH1 on OS X Leopard). Has anyone had experience with setup similar to mine?
I used macports to obtain gcc42 (which contains gfortran). I installed netcdf 3.6.2 with the default flags. I've been reading about this -ff2c issue on the forum although I don't understand it. I just downloaded source for MPICH2 and it compiled fine, I'll install it as soon as I deinstall openmpi in macports.
Any help or advice is welcome.
Thanks,
Donglai
ROMS 3, OS X 10.5.1 Intel, gfortran 4.2, MPI2 & netcdf 3
Hi, I'm encountering the following error when trying to compile MPI enabled version of ROMS on my Mac OS X 10.5.1 Core 2 machine:
mpif90 -frepack-arrays -O3 -ffast-math Build/master.o Build/ocean_control.o Build/ocean_coupler.o Build/propagator.o -o ../runs/oceanM Build/libNLM.a Build/libANA.a Build/libICE.a Build/libUTIL.a Build/libMODS.a -L/usr/local/lib -lnetcdf
Undefined symbols:
"___netcdf__nf90_inquire", referenced from:
_opencdf_ in libUTIL.a(opencdf.o)
"___netcdf__nf90_sync", referenced from:
_wrt_rst_ in libUTIL.a(wrt_rst.o)
_wrt_avg_ in libUTIL.a(wrt_avg.o)
_wrt_diags_ in libUTIL.a(wrt_diags.o)
_wrt_his_ in libUTIL.a(wrt_his.o)
_wrt_info_ in libUTIL.a(wrt_info.o)
...CUT MANY LINES...
"___netcdf__nf90_strerror", referenced from:
_close_io_ in libUTIL.a(close_io.o)
"___netcdf__nf90_def_var_manydims", referenced from:
_def_var_ in libUTIL.a(def_var.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [../runs/oceanM] Error 1
I have netcdf 3.6.2, MPICH2 and gfortran4.2 installed. Obviously the problem is with netcdf but I'm not sure what's wrong.
Thanks!
Donglai
mpif90 -frepack-arrays -O3 -ffast-math Build/master.o Build/ocean_control.o Build/ocean_coupler.o Build/propagator.o -o ../runs/oceanM Build/libNLM.a Build/libANA.a Build/libICE.a Build/libUTIL.a Build/libMODS.a -L/usr/local/lib -lnetcdf
Undefined symbols:
"___netcdf__nf90_inquire", referenced from:
_opencdf_ in libUTIL.a(opencdf.o)
"___netcdf__nf90_sync", referenced from:
_wrt_rst_ in libUTIL.a(wrt_rst.o)
_wrt_avg_ in libUTIL.a(wrt_avg.o)
_wrt_diags_ in libUTIL.a(wrt_diags.o)
_wrt_his_ in libUTIL.a(wrt_his.o)
_wrt_info_ in libUTIL.a(wrt_info.o)
...CUT MANY LINES...
"___netcdf__nf90_strerror", referenced from:
_close_io_ in libUTIL.a(close_io.o)
"___netcdf__nf90_def_var_manydims", referenced from:
_def_var_ in libUTIL.a(def_var.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [../runs/oceanM] Error 1
I have netcdf 3.6.2, MPICH2 and gfortran4.2 installed. Obviously the problem is with netcdf but I'm not sure what's wrong.
Thanks!
Donglai
The thing to do is:
to see if the symbol looks anything like ___netcdf__nf90_strerror. I'm guessing it will have a different number of underscores. You need to match the underscoring options used to compile netcdf with those used to compile ROMS.
Code: Select all
nm libnetcdf.a | grep nf90_strerrorLooking in libnetcdff.a I do see lines like:
00000040 T ___netcdf__nf90_strerror
I compiled the netcdf 3.6.2 using the default flags (except for adding --enable-shared --enable-separate-fortran)
I don't know what to change in my ROMS makefile or Darwin-gfortran.mk (which is a copy of Linux-gfortran.mk) to correct this issue.
Thanks,
Donglai
00000040 T ___netcdf__nf90_strerror
I compiled the netcdf 3.6.2 using the default flags (except for adding --enable-shared --enable-separate-fortran)
I don't know what to change in my ROMS makefile or Darwin-gfortran.mk (which is a copy of Linux-gfortran.mk) to correct this issue.
Thanks,
Donglai
OKAY, I think got it to work! The default upwelling case is happily running utilizing both cores. 
To fix the previous linking errors, I had to use the -fno-second-underscore flag when compiling netcdf with gfortran. I also turned off the --enable-shared flag like a previous posting suggested. Not sure if that made any difference.
So ROMS 3, OS X Leopard (10.5.1), gfortran 4.2.3, MPICH2 (1.0.6p1) and netcdf 3.6.2 do work together. If anyone is interested in running ROMS with a similar setup, I'd be happy to share my experience.
Donglai
To fix the previous linking errors, I had to use the -fno-second-underscore flag when compiling netcdf with gfortran. I also turned off the --enable-shared flag like a previous posting suggested. Not sure if that made any difference.
So ROMS 3, OS X Leopard (10.5.1), gfortran 4.2.3, MPICH2 (1.0.6p1) and netcdf 3.6.2 do work together. If anyone is interested in running ROMS with a similar setup, I'd be happy to share my experience.
Donglai
I just want to clarify. After trying to build ROMS 3 on a second Mac Intel with almost identical configuration, I realized the important things that make this work are simply the follow:
1. get gcc42 from macports (i.e. sudo port install gcc42)
2. make sure /opt/local/bin, /opt/local/sbin, /usr/local/bin goes before the rest of the PATH (i.e. export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH)
3. compile MPICH2 using
./configure --enable-f90
make
sudo make install
4. compile netcdf, set the following environment variables before runing configure:
export FCFLAGS=-fno-second-underscore
export FFLAGS=-fno-second-underscore
export F90FLAGS=-fno-second-underscore
then
./configure
make
sudo make install
do not add any flags to configure (i.e. --enable-shared, --enable-separate-fortran, etc. any of them will mess you up)
Finally look in your /usr/local/lib to make sure you have libnetcdf.a and libnetcdf.la, but NOT libnetcdf.dylib. If you have libnetcdf.dylib, delete it.
You should be good to go to compile ROMS. Good luck!
Donglai
1. get gcc42 from macports (i.e. sudo port install gcc42)
2. make sure /opt/local/bin, /opt/local/sbin, /usr/local/bin goes before the rest of the PATH (i.e. export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH)
3. compile MPICH2 using
./configure --enable-f90
make
sudo make install
4. compile netcdf, set the following environment variables before runing configure:
export FCFLAGS=-fno-second-underscore
export FFLAGS=-fno-second-underscore
export F90FLAGS=-fno-second-underscore
then
./configure
make
sudo make install
do not add any flags to configure (i.e. --enable-shared, --enable-separate-fortran, etc. any of them will mess you up)
Finally look in your /usr/local/lib to make sure you have libnetcdf.a and libnetcdf.la, but NOT libnetcdf.dylib. If you have libnetcdf.dylib, delete it.
You should be good to go to compile ROMS. Good luck!
Donglai
Donglai, Thanks for posting this information, it's been very useful.
Have you considered using the Intel Fortran Compiler rather than gfortran?
I also wanted to ask if you had compared running ROMS compiled without MPI to see what performance improvement you get from using both cores?
Presumably if you followed the same method on MacPro with 8 cores, you could utilise all 8 to get a quicker run?
Thanks
Alan
Have you considered using the Intel Fortran Compiler rather than gfortran?
I also wanted to ask if you had compared running ROMS compiled without MPI to see what performance improvement you get from using both cores?
Presumably if you followed the same method on MacPro with 8 cores, you could utilise all 8 to get a quicker run?
Thanks
Alan
Hi Alan,
I have not tried the Intel Fortran Compiler. Currently I have no plan to adopt it simply because I'm happy with my configuration so far.
Although I haven't compared the duo core versus single core performance, I'm pretty sure the performance nearly doubles and would likely double again with four cores. I believe ROMS does a good job of parallelizing but it is limited by the communication bandwidth between the cores. I can't comment how how well this performance scaling hold as you go above 4 cores.
I don't have a 4x2 core machine but I would apply the same procedure if I had one. I think the performance should be what you expect from an 8 core machine. The comms between CPUs is pretty darn fast.
Cheers,
Donglai
I have not tried the Intel Fortran Compiler. Currently I have no plan to adopt it simply because I'm happy with my configuration so far.
Although I haven't compared the duo core versus single core performance, I'm pretty sure the performance nearly doubles and would likely double again with four cores. I believe ROMS does a good job of parallelizing but it is limited by the communication bandwidth between the cores. I can't comment how how well this performance scaling hold as you go above 4 cores.
I don't have a 4x2 core machine but I would apply the same procedure if I had one. I think the performance should be what you expect from an 8 core machine. The comms between CPUs is pretty darn fast.
Cheers,
Donglai
amhsepa wrote:Donglai, Thanks for posting this information, it's been very useful.
Have you considered using the Intel Fortran Compiler rather than gfortran?
I also wanted to ask if you had compared running ROMS compiled without MPI to see what performance improvement you get from using both cores?
Presumably if you followed the same method on MacPro with 8 cores, you could utilise all 8 to get a quicker run?
Thanks
Alan
Re: ROMS 3, OS X 10.5.1 Intel, gfortran 4.2, MPI2 & netcdf 3
Quick installation update:
This configuration runs fine with ROMS3 Revision 203, OS X 10.5.4, gcc/gfortran 4.2.4, MPICH2 1.0.7 and netcdf4.0. To install netcdf4 you'll need HDF5 1.8.1. Make sure you compile it all with the same c/fortran compiler (gfortran in this case).
You don't need to enable netcdf4 flag in the ROMS .in file.
Cheers,
Donglai
This configuration runs fine with ROMS3 Revision 203, OS X 10.5.4, gcc/gfortran 4.2.4, MPICH2 1.0.7 and netcdf4.0. To install netcdf4 you'll need HDF5 1.8.1. Make sure you compile it all with the same c/fortran compiler (gfortran in this case).
You don't need to enable netcdf4 flag in the ROMS .in file.
Cheers,
Donglai
Re: ROMS 3, OS X 10.5.1 Intel, gfortran 4.2, MPI2 & netcdf 3
Hi, I am knew to ROMS and mpi on mac os x.
It seems possible to run roms3 in parallel on a macbook.
I followed all the previous steps of donglai
I have a gfortran 4.2 compiler
mpich2 version 1.2.1p1
netcdf 3.6.3
on a mac os x 10.5.8 on macbook pro intel Core Duo 2GHz
when I try to compile ROMS with my build.bash I have the following error
cd /Users/admin/ROMS/Projects/ReefRun_local/Build; /usr/bin/gfortran -c -frepack-arrays -O3 -ffast-math -I/opt/gfortran/mct/include mp_exchange.f90
mp_exchange.f90:106.30:
Null_Value=MPI_PROC_NULL
1
Error: Symbol 'mpi_proc_null' at (1) has no IMPLICIT type
make: *** [/Users/admin/ROMS/Projects/ReefRun_local/Build/mp_exchange.o] Error 1
Any one could help me ?
cheers
Emmanuel
It seems possible to run roms3 in parallel on a macbook.
I followed all the previous steps of donglai
I have a gfortran 4.2 compiler
mpich2 version 1.2.1p1
netcdf 3.6.3
on a mac os x 10.5.8 on macbook pro intel Core Duo 2GHz
when I try to compile ROMS with my build.bash I have the following error
cd /Users/admin/ROMS/Projects/ReefRun_local/Build; /usr/bin/gfortran -c -frepack-arrays -O3 -ffast-math -I/opt/gfortran/mct/include mp_exchange.f90
mp_exchange.f90:106.30:
Null_Value=MPI_PROC_NULL
1
Error: Symbol 'mpi_proc_null' at (1) has no IMPLICIT type
make: *** [/Users/admin/ROMS/Projects/ReefRun_local/Build/mp_exchange.o] Error 1
Any one could help me ?
cheers
Emmanuel
Re: ROMS 3, OS X 10.5.1 Intel, gfortran 4.2, MPI2 & netcdf 3
Hi, I can try to help a little. There are instructions to install HDF5 and NetCDF on a Mac at http://scicompideas.blogspot.com/2009/1 ... etcdf.html. He is using a Mac with 64-bit Snow Leopard on it but I imagine the instructions should still be close if not the same for your Mac operating system.
For completeness, I will go ahead and copy and paste in some instructions I have written to myself for reference, for after zlib, szlib, gfortran, hdf5, and netcdf, and make, perl and svn are all installed:
Got ROMS using SVN:
svn checkout --username [your ROMS account username] https://www.myroms.org/svn/src/trunk ~/ROMS
Added environment variables in .bashrc file:
export NETCDF_INCDIR='/usr/local/include'
export NETCDF_LIBDIR='/usr/local/lib'
export HDF5_LIBDIR='/usr/local/lib'
Set in makefile
USE_MPI ?= on
USE_MPIF90 ?= on
USE_NETCDF4 ?= on
FORT ?= gfortran
Added flags to lib in Compilers/Linux-gfortran.mk from
ifdef USE_NETCDF4
LIBS += -L$(HDF5_LIBDIR) -lhdf5_hl -lhdf5 -lz
endif
to
ifdef USE_NETCDF4
LIBS += -L$(HDF5_LIBDIR) -lhdf5_hl -lhdf5 -lsz -lz
endif
ifdef USE_NETCDF4,
NETCDF_INCDIR ?= /usr/local/include,
NETCDF_LIBDIR ?= /usr/local/lib,
HDF5_LIBDIR ?= /usr/local/lib
Hopefully that makes sense. I think that it should work for parallel but I may have just done it in serial (I mostly run on a bigger workstation). Sorry, I can't really speak directly to the error listed, but maybe if you try reinstalling some of the programs from scratch something will work out. Good luck!
For completeness, I will go ahead and copy and paste in some instructions I have written to myself for reference, for after zlib, szlib, gfortran, hdf5, and netcdf, and make, perl and svn are all installed:
Got ROMS using SVN:
svn checkout --username [your ROMS account username] https://www.myroms.org/svn/src/trunk ~/ROMS
Added environment variables in .bashrc file:
export NETCDF_INCDIR='/usr/local/include'
export NETCDF_LIBDIR='/usr/local/lib'
export HDF5_LIBDIR='/usr/local/lib'
Set in makefile
USE_MPI ?= on
USE_MPIF90 ?= on
USE_NETCDF4 ?= on
FORT ?= gfortran
Added flags to lib in Compilers/Linux-gfortran.mk from
ifdef USE_NETCDF4
LIBS += -L$(HDF5_LIBDIR) -lhdf5_hl -lhdf5 -lz
endif
to
ifdef USE_NETCDF4
LIBS += -L$(HDF5_LIBDIR) -lhdf5_hl -lhdf5 -lsz -lz
endif
ifdef USE_NETCDF4,
NETCDF_INCDIR ?= /usr/local/include,
NETCDF_LIBDIR ?= /usr/local/lib,
HDF5_LIBDIR ?= /usr/local/lib
Hopefully that makes sense. I think that it should work for parallel but I may have just done it in serial (I mostly run on a bigger workstation). Sorry, I can't really speak directly to the error listed, but maybe if you try reinstalling some of the programs from scratch something will work out. Good luck!