ROMS 3.1 - OSX 10.5.2 Intel - ifort 10.1.006 - netcdf 3.6.2:

Discussion on computers, ROMS installation and compiling

Moderators: arango, robertson

Post Reply
Message
Author
User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

ROMS 3.1 - OSX 10.5.2 Intel - ifort 10.1.006 - netcdf 3.6.2:

#1 Unread post by amhsepa »

Hi,

I'm looking for some advice on using the ifort (10.1.006) compiler to build ROMS 3.1. I've carried out the following steps:

1 - Installed ifort as per instructions and modified .bash_login to include all the necessary scripts.

2 - Successfully built and tested netcdf-3.6.2 using ifort. In the first case I used the Mac OS X default gcc and g++, with no Flags to the compilers. In the second case I used the Intel C/C++ compiler and followed the instructions on the Intel website: http://www.intel.com/support/performanc ... 027812.htm

3 - In both of the above cases identical things happen when I try to compile a basic serial upwelling test case. When using Debug on, ROMS compiles fine with no obvious errors. The resulting oceanG runs fine, with the usual slow speed. When I set debug to off, the code seems to compile ok, and I get lots of LOOP WAS VECTORIZED type comments. The resulting oceanS does not run and crashes, giving the following error:

Model Input Parameters: ROMS/TOMS version 3.0
Saturday - March 29, 2008 - 3:38:59 PM
-----------------------------------------------------------------------------
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
oceanS 00219873 Unknown Unknown Unknown
oceanS 00218D0F Unknown Unknown Unknown
oceanS 001DFFF8 Unknown Unknown Unknown
oceanS 001B0870 Unknown Unknown Unknown
oceanS 001B73A0 Unknown Unknown Unknown
libSystem.B.dylib 91EEA5EB Unknown Unknown Unknown
Unknown FFFFFFFF Unknown Unknown Unknown

4 - The compiler make file I'm using is Darwin-ifort.mk, which is a copy of the Linux-fort.mk. This worked ok for me in the past when using gfortran.

5 - I examined the Flags sent to ifort when debug is set to off. I've tried various combinations of these, based on the information in the ifort manual.

6 - I can only get the oceanS executable to work when I send the -O0 compiler option to ifort. This turns all code optimizations off. oceanS runs fine but just as slow as when debug was on.

7 - I did see in the forums that segmentation fault can be related to the stack size. I tried altering this but no success.

8 - I carried out the above steps using the new version of gfortran (binary) from the following excellent site:http://hpc.sourceforge.net/
netcdf and roms both compile great first time and run fast in serial. It may be worth mentioning I had problems compiling netcdf using the new versions of gcc and g++. These problems dissapear when using the default install, older, versions of gcc and G++.

I'd be really grateful if anyone has any tips on using ifort to compile ROMS. I've read that there are performance gains in using ifort to compile code on Intel chips. Any pointers would be much appreciated.

Thanks in advance.

Alan

User avatar
hetland
Posts: 81
Joined: Thu Jul 03, 2003 3:39 pm
Location: TAMU,USA

#2 Unread post by hetland »

I had exactly the same problem. The solution appears to be to get the latest version of the compiler (10.1.012). Intel has had some pretty sever problems with Leopard, and I'm not sure whether to blame Apple or Intel. Either way, it has not been good. See:
http://softwarecommunity.intel.com/isn/ ... hread.aspx

In any case, gfortran (available with a clickable installer here: http://r.research.att.com/tools/) has worked very well for me through the transition. It is about 10% slower, but compiles quickly, and without segfaulting...
Last edited by hetland on Mon Mar 31, 2008 5:46 pm, edited 1 time in total.

User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

#3 Unread post by amhsepa »

Thanks very much for getting back so quickly, that's excellent info. It's a real shame there's been so many problems. I only purchased the Intel compiler last week, so I’m a bit puzzled as to why it let me download such an old version.

Thanks once again.

Alan

User avatar
hetland
Posts: 81
Joined: Thu Jul 03, 2003 3:39 pm
Location: TAMU,USA

#4 Unread post by hetland »

For the record, here are some build notes for netcdf and openmp on Leopard. I used ifort 10.1.012 and gfortran 4.3.0.

Nothing too surprising here, but I finally decided to write all this down after I kept on forgetting which options needed to be specified.




Notes for making netcdf3 and netcdf4:
------------------------------------

These libraries will be put into:

/opt/netcdf3/ifort
/opt/netcdf3/gfortran

/opt/netcdf4/ifort
/opt/netcdf4/gfortran

/opt/hdf5/ifort
/opt/hdf5/gfortran



NETCDF3 [version 3.6.2]
-------

To start with, this site has a long list of options for various compilers and systems:

http://www.unidata.ucar.edu/software/ne ... uilds.html

First, netcdf3 with ifort uses the environmental variables:

export FC=ifort
export FFLAGS='-O3'
./configure --prefix=/opt/netcdf3/ifort

Some options for FFLAGS might include: -O3 -xT -ip -no-prec-div -static (based on a build from the other builds site.) This is similar to the -fast option, which sets: -ipo, -O3, -no-prec-div, -static, and -xP. It seems that -ipo causes configure to fail, so it is left out of the options
-xT builds for 64bit duo processors, -xP for 32 bit. The default is fine, so nothing is set.
-static builds static libs (not needed, as make knows when to do this is seems)
-no-prec-div uses faster division algorithms at the expense of accuracy. Why do this?
So the only flag left standing is '-O3'. This could probably be set to '-O2' without much trouble.

To make netcdf3 gfortran, simply do

export FC=gfortran
export FFLAGS='-O3'
./configure --prefix=/opt/netcdf3/gfortran

Both can be made in parallel with make -j, and both pass make check.



HDF5 [version 1.8.0]
----

Environmental settings used are similar to netcdf3:

export F77=ifort
export F90=ifort
export FFLAGS='-O3'
./configure --prefix=/opt/hdf5/ifort --enable-hl --enable-fortran

The key is the --enable-hl flag, which allows the high-level language API needed by netcdf4. This cannot be built in full parallel, but make -j 2 seems to work fine. Passes make check, after a long, long time. For gfortran:

export F77=gfortran
export F90=gfortran
export FFLAGS='-O3'
./configure --prefix=/opt/hdf5/gfortran --enable-hl --enable-fortran

Also passes make check

NETCDF4 [version snapshot2008031713]
-------

Again, environmental settings used are similar to netcdf3:

export F77=ifort
export F90=ifort
export FFLAGS='-O3'
./configure --prefix=/opt/netcdf4/ifort -enable-netcdf-4 --with-hdf5=/opt/hdf5/ifort

The key is here is obviously the --enable-netcdf-4 command, otherwise netcdf4 is not really built. Here parallel make -j seems to work fine. Passes make check. For gfortran, no surprises:

export F77=gfortran
export F90=gfortran
export FFLAGS='-O3'
./configure --prefix=/opt/netcdf4/gfortran -enable-netcdf-4 --with-hdf5=/opt/hdf5/gfortran

Also passes make check.




Notes on making openmpi [version 1.2.5]
------------------------------

export F77=ifort
export FC=ifort
export FFLAGS='-O3'
export FCFLAGS='-O3'
./configure --prefix=/opt/openmpi/ifort --disable-mpi-profile


for gfortran

export F77=gfortran
export FC=gfortran
export FFLAGS='-O3'
export FCFLAGS='-O3'
./configure --prefix=/opt/openmpi/gfortran --disable-mpi-profile

Note that the --disable-mpi-profile is because of a poorly linked file in the profiling directory (the link refers to the parent directory instead of a file).
Last edited by hetland on Mon Mar 31, 2008 5:47 pm, edited 1 time in total.

User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

#5 Unread post by amhsepa »

Cheers for more useful notes!

I wanted to ask about the ifort version numbers. I've looked at the available downloads and can see: (the characters in brackets are the various minor versions)

10.0.(CESR, 016, 017 and 020)

10.1.(CESR, 006,007,008 and 012)

The most recent one appears to be 10.1.012. Would you recommend this or 10.0.20?

Thanks

Alan

User avatar
hetland
Posts: 81
Joined: Thu Jul 03, 2003 3:39 pm
Location: TAMU,USA

#6 Unread post by hetland »

I mean 10.1.012, of course...

I have edited the above posts so that no one will ever know I was wrong. Sneaky, eh?

User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

#7 Unread post by amhsepa »

Ah...good...after peering at compiler output forever...the numbers were starting to blur! I'd better edit my posts to preserve continuity..

Thanks again for all the help and notes. Looking forward to testing it out.

User avatar
enrique
Posts: 12
Joined: Fri Apr 02, 2004 4:45 pm
Location: IMCS, Rutgers University, USA

#8 Unread post by enrique »

Well, I just installed the new ifort and when trying to compile netCDF
following Rob's environment suggestions I get:

checking if Fortran "byte" is C "signed char"... configure: error: Could not link conftestf.o and conftest.o

Any ideas?

User avatar
hetland
Posts: 81
Joined: Thu Jul 03, 2003 3:39 pm
Location: TAMU,USA

#9 Unread post by hetland »

Versions? Which ifort, which netcdf?

powellb
Posts: 11
Joined: Sat Feb 14, 2004 5:16 am
Location: University of Hawaii

ifort 10.1.006

#10 Unread post by powellb »

As Rob mentioned, the problem is with ifort 10.1.006. This had a bug in the optimizer which would cause segmentation faults when unrolling loops that access arrays. This was fixed in the 008 release.

Basically, use the latest. You cannot use any release prior to 10.1 on Leopard.

Enrique, the following will setup netcdf 3.6.2 properly using ifort:

1) make distclean
2) ./configure FC=ifort F77=ifort --enable-f90 --prefix=/usr/local/netcdf-3.6.2-intel

If you still have a problem, let me know. I haven't tried the official HDF5 and latest Netcdf4 builds (but I didn't have problems with the alphas).
Last edited by powellb on Tue Apr 01, 2008 6:50 pm, edited 2 times in total.

powellb
Posts: 11
Joined: Sat Feb 14, 2004 5:16 am
Location: University of Hawaii

Forgot

#11 Unread post by powellb »

I forgot that due to some fancy arrangements that ifort 10.1 does to numerical calculations, it can wreak havoc on the symmetry between the tangent-linear and adjoint models.

I have added the following option to the Darwin-ifort.mk in my ROMS development branch (it should make it into the public branch in a while):

-assume protect_parens

This prevents the mathematical optimizer from trying to do some operations out of order. It should not have an impact on the nonlinear model.


UPDATE: This bug has been corrected in ifort, and there is no longer a need to specify this option.
Last edited by powellb on Wed Apr 02, 2008 11:48 pm, edited 2 times in total.

User avatar
enrique
Posts: 12
Joined: Fri Apr 02, 2004 4:45 pm
Location: IMCS, Rutgers University, USA

#12 Unread post by enrique »

I downloaded today ifort 10.1.012 and am trying netcdf 3.6.2

User avatar
enrique
Posts: 12
Joined: Fri Apr 02, 2004 4:45 pm
Location: IMCS, Rutgers University, USA

#13 Unread post by enrique »

Brian wins the "Hacker-of-the-day" title.

I needed to source .... intel/fc/10.1.012/bin/ifortvars.csh

somehow I did not have to do this before

User avatar
hetland
Posts: 81
Joined: Thu Jul 03, 2003 3:39 pm
Location: TAMU,USA

#14 Unread post by hetland »

Usually temporary environmental variable definitions go before the command (e.g., ./configure). Look at the script to make sure it is using the right fortran compiler. Also, try a which ifort command to make sure you are using the one you think (you will have to follow a few softlinks to get to the answer...)

User avatar
enrique
Posts: 12
Joined: Fri Apr 02, 2004 4:45 pm
Location: IMCS, Rutgers University, USA

#15 Unread post by enrique »

The issue has to do with linked dynamic libraries. Ifort
has specific ones it wants ahead of others. This is
what fixes things:

if !($?LD_LIBRARY_PATH) then
setenv LD_LIBRARY_PATH /opt/intel/fc/10.1.012/lib
else
setenv LD_LIBRARY_PATH /opt/intel/fc/10.1.012/lib:${LD_LIBRARY_PATH}
endif

# DYLD_LIBRARY_PATH is used on MAC OS*
if !($?DYLD_LIBRARY_PATH) then
setenv DYLD_LIBRARY_PATH /opt/intel/fc/10.1.012/lib
else
setenv DYLD_LIBRARY_PATH /opt/intel/fc/10.1.012/lib:${DYLD_LIBRARY_PATH}
endif


[/b]

powellb
Posts: 11
Joined: Sat Feb 14, 2004 5:16 am
Location: University of Hawaii

#16 Unread post by powellb »

powellb wrote:I forgot that due to some fancy arrangements that ifort 10.1 does to numerical calculations, it can wreak havoc on the symmetry between the tangent-linear and adjoint models.

I have added the following option to the Darwin-ifort.mk in my ROMS development branch (it should make it into the public branch in a while):

-assume protect_parens

This prevents the mathematical optimizer from trying to do some operations out of order. It should not have an impact on the nonlinear model.
After installing and testing the latest ifort 10.1.012 on linux and mac, I found that they have resolved the numerical bug I listed above. You no longer need -assume protect_parens.

User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

#17 Unread post by amhsepa »

Got ROMS working with ifort 10.1.012, thanks to all for the useful info.

Intel have now released 10.1.014, although I've not tried it yet.

User avatar
amhsepa
Posts: 7
Joined: Sat Nov 03, 2007 3:40 pm
Location: SEPA, Glasgow, Scotland. UK
Contact:

#18 Unread post by amhsepa »

Hi. Some issues arise when you have both ifort and gfortran installed on your system (at least they do on mine!) Despite setting all the correct Environment Variables, the configure files of netcdf and openmpi both want to use gfortran, even when you specifiy ifort as your compiler. I guess this could be related to the PATH variable, but I found the following workaround successful:

- Alter the configure file and put ifort before gfortran in some of the script lines.

For netcdf you only need edit lines 4318, 4319, 4322 and 4323. For openmpi I edited every line where ifort was referenced.

I also had an issue with HDF5 where it wouldn't recognise environment variables and threw an error when trying to configure the fortran compiler.

I found the following information useful at this site:

http://www.tuinder.net/~tuinder/howto/h ... ifort.html

Specifically the command:

env F9X=ifort ./configure --enable-fortran (You can set other configuration options, if required).

This solved the compiler problem, but you need to set the environment variable right before the configure (not as a separate command). I didn't use any of the other options specified.

Post Reply