Few performance optimizations

[arango]

Implemented a few performance optimizations in mpdata_adiff.F and several lmd_*.F routines, and added an option to use mpi_allreduce in distribute.F. The use of mpi_allreduce is usually more efficient in reducing communications. This will improve the performance on floats, stations, and variational data assimilation. That is, all routines that use mp_collect.

Notice at the top of distribute.F there are several local cpp options to use. They are intended to do the same tasks but at different levels of performance.

The ones that we specified by default are:

# define BOUNDARY_ALLREDUCE /* use mpi_allreduce in mp_boundary */
# undef  COLLECT_ALLGATHER  /* use mpi_allgather in mp_collect  */
# define COLLECT_ALLREDUCE  /* use mpi_allreduce in mp_collect  */
# define REDUCE_ALLGATHER   /* use mpi_allgather in mp_reduce   */
# undef  REDUCE_ALLREDUCE   /* use mpi_allreduce in mp_reduce   */

I highly recommend that you do not modify these definitions unless you know what you are you doing. It requires extensive knowledge and expertise on MPI communications. Some of the advanced MPI communication routines perform differently on several computers. These routines are usually optimized by the vendor for a particular architecture hardware.

Also notice that the vectorization directive in lmd_swfrac.F

!!DIR$ VECTOR ALWAYS

is commented out with an extra ! and will be removed during c-preprocessing. All the code lines that start with !! will be removed by cpp_clean. This directive may not be the same for all compilers.

Many thanks to Xavier Vigouroux, Remi Revire, and others at BULL high performance computers for suggesting these optimizations.