different compiler commands

[DTokarev]

Hi,
I used ifort v.10 and netcdf-3.6.3.
compiler command in the _.out file is:

Code: Select all

 Operating system : Linux
 CPU/hardware     : x86_64
 Compiler system  : ifort
 Compiler command : ifort -fpp2 -openmp -pc80 -axP -align dcommon -auto -stack_temp
 Compiler flags   : -O3 -IPF_fma -ip -O3 -IPF_fma -ip -ip -O3 -xW -free

Now I have installed ifort v.11.1 and netcdf-4.0.1.
Compiler command: ????

Code: Select all

 
 Operating system : Linux
 CPU/hardware     : x86_64
 Compiler system  : ifort
 Compiler command : /opt/intel/Compiler/11.1/056/bin/intel64/ifort
 Compiler flags   : -heap-arrays -fp-model precise -openmp -fpp -ip -O3 -xW -free

No changes in the makefile and build.bash files.
Any explanation?

Thanks in advance

[kate]

This could be coming from ifort, though I don't really know. Looking in Compilers/Linux-ifort.mk, I see:

Code: Select all

FFLAGS := -heap-arrays -fp-model precise

If that didn't change, ifort could have been modifying the environment on you.

[shchepet]

Intels keeps modifying both compiler and compiler usage, along with hardware, so compiler
flags are generally chanhing all the time.

Compiler setting and flags like

Compiler command : ifort -fpp2 -openmp -pc80 -axP -align dcommon -auto -stack_temp

Compiler flags : -O3 -IPF_fma -ip -xW -free

are originated from me back in 2003-2004, and now these particular settings are (oh, boy!) way-way
long-time-ago obsolete: both compilers and CPUs have evolved since then.

Frankly, it would be very hard to recommend any specific settings, but the principles how to choose
them are quite clear.

First:

You have two groups of flags: what is specified in "Compiler command" is generally fixed (you
do not chenge them on daily basis) and depends on processor you have and the desired precision, etc.

The second group is optimization flags: these are generally more volatile, and you tend to change
them if you work on code (debugging, for example), or to reduce compilation time, if you have to
recompile very often, or set/unset various traps, etc.

Secondly:

Know the type of CPU you use and your compiler version. Intel keeps expanding its instruction
sets, so the general rule is that if you compile your code relying on defaults, you are most likely
under utilizing full potential of your hardware. Conversely, an executable file optimally-compiled
for a specific processor will not run on and earlier processor.

For example, -xW flag above tells optimize for Pentium 4 "Northwood" or first-generation Opteron CPUs.
This is sub-optimal, if your CPU is newer (say, Pentium 4 "Prescott", Pentium D, or a Core 2, not
mentioning i7) than that. -xP stands for "Prescott" (remember that exciting time in spring 2004?).

Similarly, if you get a third party package, like netCDF, ncview, MPI, or whatever, it is very
likely that the usual sequence

configure --- make --- make install

will use all-default settings (e.g., a long-standing Linux tradition is to ise "-O2 -g", resulting
in generally slower executables).

Glance at "man ifort" for your specific version and understand what is the meaning of each flag.
The man page is very long, so it is not likely that you will get everything at once, but at least
find how to choose proper CPU. Intel also can do "hard" and "soft" tuning for targeted CPUs, for
example, the above flags -axP and -xW are "soft" (meaning that optimize for, but still keep
compatible) and "hard" (use that CPU only) settings; However, if you go through "man ifort" for
a 11.1.something version of compiler, they are replaced with new flags, abet of the same
functionality.

Example: say Ifort 11.1.056 compiler, Intel Xeon 5420 CPU, shared-memory code for OpenMP
using more than 2 GBytes of memory:

OMP_FLAG = -fpp2 -openmp

LARGE_MEM_FLAG = -mcmodel=medium -i-dynamic

CFT = ifort $(OMP_FLAG) -pc80 -xSSE4.1 -w95 -align dcommon -auto -stack_temps

FFLAGS = -O3 -IPF_fma -ip $(LARGE_MEM_FLAG)

(note that -ip above may be troublesome; I observed some roundoff-level miss-behavior, so use with
caution)

More restrictive floating-point arithmetic: same as above, except

FFLAGS = -O2 -mp -fpe0 -fp-model strict

Full debugging:

FFLAGS = -g -check all -CA -CB -CS

[DTokarev]

Hi all,
Thanks for explanation, but...

I see compiler command with no installation directory in the first case

Code: Select all

Compiler command : ifort -fpp2 -openmp -pc80 -axP -align dcommon -auto -stack_temp

and ifort command from installation point only in the second one:

Code: Select all

Compiler command : /opt/intel/Compiler/11.1/056/bin/intel64/ifort

I use identical Linux_ifort.mk files and configure - make check - make install
commands for all third party packages.

Is this OK?
Thanks in advance

[shchepet]

What is in your .cshrc file in home directory of the machine you use?

A typical practice in supercomputer centers to install a new version of the compiler
when it becomes available, but keep the old one, for just in case. This is also what
we are doing here at home. As the result, at any given time a computer may have several,
up to a dozen versions of compilers installed, all of which are functional.

So it is not a good idea to hard-code the actual PATH to compiler in your Makefiles
and compilation scripts, like

Code: Select all

Compiler command : /opt/intel/Compiler/11.1/056/bin/intel64/ifort

Specify just "ifort" instead, but in your .cshrc make entries like

Code: Select all

set IFC_ROOT = /opt/intel/Compiler/11.1/056
setenv PATH $IFC_ROOT/bin/intel64:$PATH
setenv LD_LIBRARY_PATH $IFC_ROOT/lib/intel64
setenv MANPATH $IFC_ROOT/man:$MANPATH

This will allow you to switch between the version easily.
You can check which compiler you use by typing "which ifort" and it tells
you the absolute path to it.

[kate]

This could be due to a ROMS update too. The current version of my files in Compilers has this:

Code: Select all

#
# Use full path of compiler.
#
               FC := $(shell which ${FC})
               LD := $(FC)

so I get for instance:

Code: Select all

cd Build; /usr/local/pkg/pgi/current/linux86-64/6.1/bin/pgf90 -c  -O3 -tp k8-64 exchange_2d.f90

[DTokarev]

Hi kate and shchepet,

This is my .cshrc file after changes:

Code: Select all

# .cshrc

# User specific aliases and functions

alias rm 'rm -i'
alias cp 'cp -i'
alias mv 'mv -i'
set path = (\
/usr/bin \
/usr/lib \
/usr/include \
/usr/local/bin \
/usr/local/lib \
/usr/local/include \
.)

export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
set IFC_ROOT = /opt/intel/Compiler/11.1/056
setenv PATH $IFC_ROOT/bin/intel64:$PATH
setenv LD_LIBRARY_PATH $IFC_ROOT/lib/intel64
setenv MANPATH $IFC_ROOT/man:$MANPATH

When I try to compile the model I've received next error:

Code: Select all

rm -f -r /usr/local/Compilers/make_macros.mk
cd /usr/local/Build;  -c -heap-arrays -fp-model precise -openmp -fpp -ip -O3 -xW mod_kinds.f90
/bin/sh: -c: command not found
make: *** [/usr/local/Build/mod_kinds.o] Error 127
make: *** Waiting for unfinished jobs....
ROMS/Bin/cpp_clean /usr/local/Build/master.f90

Thanks in advance

[kate]

The command not found is your compiler. What happens when you type "which ifort"?

[shchepet]

Code: Select all

rm -f -r /usr/local/Compilers/make_macros.mk
cd /usr/local/Build;  -c -heap-arrays -fp-model precise -openmp -fpp -ip -O3 -xW mod_kinds.f90
/bin/sh: -c: command not found

from Kate

The command not found is your compiler. What happens when you type "which ifort"?

No, the error message says "-c: command not found". It does not say "ifort: command not found".

This error does NOT come from .cshrc, but is from the compilation script or Makefile -- the
macro definition which should assume value "ifort" somehow ended up being blank.
And this is what needs to be checked.

[DTokarev]

Hi,


If I add "source" line into the .buid.sh script I can to compile the model:

Code: Select all

# Other user defined environmental variables. See the ROMS makefile for
# details on other options the user might want to set here. Be sure to
# leave the switched meant to be off set to an empty string or commented
# out. Any string value (including off) will evaluate to TRUE in
# conditional if-stamentents.

 source /opt/intel/Compiler/11.1/056/bin/ifortvars.csh intel64

#setenv USE_MPI             on
#setenv USE_MPIF90          on
 setenv FORT                ifort

 setenv USE_OpenMP          on

#setenv USE_DEBUG           on
 setenv USE_LARGE           on
 setenv USE_NETCDF4         on

But in this case I've received error message:

Code: Select all

./oceanO < ROMS/External/ocean_esh.in
./oceanO: error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory

If I compile with build.bash (with shell changing) all OK
Thanks for your help

[shchepet]

Code: Select all

./oceanO < ROMS/External/ocean_esh.in
./oceanO: error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory

This is easy to fix: libiomp5.so belongs to intel compiler, and is
located in /opt/intel/Compiler/11.1/056/lib/intel64 for the compiler
you use. When you add line

source /opt/intel/Compiler/11.1/056/bin/ifortvars.csh intel64

into ROMS build script, the settings that command puts in are valid
only WITHIN the scope of compile script, but not for your login shell,
so as the result you can compile, but you cannot run.

Because executable file oceanO relies on shared libraries, but the
path to them is not hardcoded into it (this is a good normal standard
policy which makes executables be portable when moving from from one
computer to another, and also making sure that previously compiled
executables still working if you upgrade your compiler, while removing
the old one), you must tell where the library resigns. Therefore
environmental variable LD_LIBRARY_PATH of your login shell should
contain library directory of intel compiler you use, which is in your
case is /opt/intel/Compiler/11.1/056/lib/intel64.