External Libraries - Revision history

Robertson at 18:42, 18 June 2025

2025-06-18T18:42:51Z

Robertson at 18:32, 5 May 2025

2025-05-05T18:32:28Z

← Older revision		Revision as of 18:32, 5 May 2025
Line 38:		Line 38:
	Open-source, machine-independent libraries for storing self-describing scientific data into portable, scalable, appendable, and sharable computer files. It provides '''API''' in several computer languages to create files, to write data into a file, to read data from the file, and access information about the dataset dimensions, variables, and attributes.		Open-source, machine-independent libraries for storing self-describing scientific data into portable, scalable, appendable, and sharable computer files. It provides '''API''' in several computer languages to create files, to write data into a file, to read data from the file, and access information about the dataset dimensions, variables, and attributes.

	ROMS uses '''NetCDF''' for all its input and output data management. Its '''NetCDF3''' or '''NetCDF4''' type files can be processed using the standard library developed and distributed [http://doi.org/10.5065/D6H70CW6 UCAR/Unidata], the Parallel-IO ('''PIO''') developed at ANL/NCAR ([https://doi.org/10.1177/1094342011428143 Dennis ''et al''., 2012]; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE Hartnett and Edwards, 2021])~~, or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library available in [https://e3sm.org/scorpio-parallel-io-library E3SM]~~. We have migrated to the Fortran-90 '''API''' interface, which consists of a library and one or two module files. See the [[makefile]] and [[build_roms\| build script]] for how to tell ROMS where these are located. You will have to use a copy of the library and module files that were compiled using the same compiler you are using to compile ROMS - this is especially true for the module files.		ROMS uses '''NetCDF''' for all its input and output data management. Its '''NetCDF3''' or '''NetCDF4''' type files can be processed using the standard library developed and distributed [http://doi.org/10.5065/D6H70CW6 UCAR/Unidata], the Parallel-IO ('''PIO''') developed at ANL/NCAR ([https://doi.org/10.1177/1094342011428143 Dennis ''et al''., 2012]; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE Hartnett and Edwards, 2021]). We have migrated to the Fortran-90 '''API''' interface, which consists of a library and one or two module files. See the [[makefile]] and [[build_roms\| build script]] for how to tell ROMS where these are located. You will have to use a copy of the library and module files that were compiled using the same compiler you are using to compile ROMS - this is especially true for the module files.

	===Building NetCDF-4/HDF5===		===Building NetCDF-4/HDF5===
Line 76:		Line 76:
	Open-source parallel library for reading and writing distributed arrays to several scientific data formats like '''pNetCDF''', '''NetCDF3''', and '''NetCDF4/HDF5'''. '''PIO''' provides the flexibility to regulate the number of I/O tasks through data rearrangement between computational and I/O processes to improve performance and memory usage. It allows both '''synchronous''' and '''asynchronous''' I/O. In '''synchronous mode''', a subset of processes performs both I/O and computations (MPI intra-communications). Alternatively, in '''asynchronous mode''', the I/O is carried out by a set of disjointed and dedicated processes (MPI inter-communications).		Open-source parallel library for reading and writing distributed arrays to several scientific data formats like '''pNetCDF''', '''NetCDF3''', and '''NetCDF4/HDF5'''. '''PIO''' provides the flexibility to regulate the number of I/O tasks through data rearrangement between computational and I/O processes to improve performance and memory usage. It allows both '''synchronous''' and '''asynchronous''' I/O. In '''synchronous mode''', a subset of processes performs both I/O and computations (MPI intra-communications). Alternatively, in '''asynchronous mode''', the I/O is carried out by a set of disjointed and dedicated processes (MPI inter-communications).

	The '''PIO~~''' or '''SCORPIO~~''' library is intended for ROMS Message Passing Interface (MPI) applications running on a large number of processes in an HPC computer with a '''Parallel File System''' (Lustre, GPFS, and so on). It uses the '''MPI-IO''' interface to facilitate the partitioning of data across computational or dedicated I/O processes. ~~Both~~ [https://github.com/NCAR/ParallelIO PIO~~] and [https://github.com/E3SM-Project/scorpio SCORPIO~~] will work with ROMS [https://www.myroms.org/projects/src/ticket/884 svn revision 1064] (May 10, 2021) or newer ~~but we recommend '''PIO''' for its better performance~~. We recommend you read the manuals for all the build options, but below we show an example build ~~process~~ for ~~both~~ '''PIO~~''' and '''SCORPIO~~'''.		The '''PIO''' library is intended for ROMS Message Passing Interface (MPI) applications running on a large number of processes in an HPC computer with a '''Parallel File System''' (Lustre, GPFS, and so on). It uses the '''MPI-IO''' interface to facilitate the partitioning of data across computational or dedicated I/O processes. [https://github.com/NCAR/ParallelIO PIO] will work with ROMS [https://www.myroms.org/projects/src/ticket/884 svn revision 1064] (May 10, 2021) or newer. We recommend you read the manuals for all the build options, but below we show example build processes for '''PIO'''.

	~~===Building PIO (autotools)===~~

	'''PIO''' can use NetCDF (version 4.6.1+) and/or PnetCDF (version 1.9.0+) for I/O. NetCDF may be built with or without netCDF-4 features. NetCDF is required for '''PIO''', PnetCDF is optional. {{note}}However, the ROMS build system assumes that '''PIO''' is built with PnetCDF support and will need customization if '''PIO''' is built without it.		'''PIO''' can use NetCDF (version 4.6.1+) and/or PnetCDF (version 1.9.0+) for I/O. NetCDF may be built with or without netCDF-4 features. NetCDF is required for '''PIO''', PnetCDF is optional. {{note}}However, the ROMS build system assumes that '''PIO''' is built with PnetCDF support and will need customization if '''PIO''' is built without it.
Line 86:		Line 84:
	'''PIO''' recently introduced a NetCDF integration option that allows standard NetCDF library calls to be implemented with the '''PIO''' library but ROMS Parallel I/O is not currently implemented this way. In order to build '''PIO''' with NetCDF integration, you will need NetCDF version 4.7.4 or higher. However, the Fortran API integration has not been implemented in ROMS.		'''PIO''' recently introduced a NetCDF integration option that allows standard NetCDF library calls to be implemented with the '''PIO''' library but ROMS Parallel I/O is not currently implemented this way. In order to build '''PIO''' with NetCDF integration, you will need NetCDF version 4.7.4 or higher. However, the Fortran API integration has not been implemented in ROMS.

	The '''PIO''' library can be built with the classic Autotools or Cmake. ~~For '''~~PIO~~''' we show the~~ Autotools ~~procedure but will show Cmake for '''SCORPIO''' in the next section.~~		The '''PIO''' library can be built with the classic Autotools or Cmake. Both are shown below.

			===Building PIO With Autotools===

	*First download the '''PIO''' source code as a gzipped tarball or via ''git clone'':<div class="box">> wget https://github.com/NCAR/ParallelIO/releases/download/pio2_5_4/pio-2.5.4.tar.gz</div>or<div class="box">> git clone https://github.com/NCAR/ParallelIO<br />> cd ParallelIO<br />> git checkout tags/pio2_5_4<br />> autoconf -i</div>		*First download the '''PIO''' source code as a gzipped tarball or via ''git clone'':<div class="box">> wget https://github.com/NCAR/ParallelIO/releases/download/pio2_5_4/pio-2.5.4.tar.gz</div>or<div class="box">> git clone https://github.com/NCAR/ParallelIO<br />> cd ParallelIO<br />> git checkout tags/pio2_5_4<br />> autoconf -i</div>
Line 96:		Line 96:
	*Build, test, and install '''PIO''':<div class="box">> make -j 5<br />> make check<br />> make install</div>{{warning}} '''Warning:''' If you are using the Intel compilers, you will probably need to remove "'''test_darray_async_many'''" from line 50 of '''test/cunit/run_tests.sh''' for ''make check'' to complete without errors. Details [https://github.com/NCAR/ParallelIO/issues/1867 here].		*Build, test, and install '''PIO''':<div class="box">> make -j 5<br />> make check<br />> make install</div>{{warning}} '''Warning:''' If you are using the Intel compilers, you will probably need to remove "'''test_darray_async_many'''" from line 50 of '''test/cunit/run_tests.sh''' for ''make check'' to complete without errors. Details [https://github.com/NCAR/ParallelIO/issues/1867 here].

	===Building ~~SCORPIO (~~Cmake)===		===Building PIO With Cmake===

	{{note}} '''Note:''' '''SORPIO''' recently released version 1.2.0 and 1.2.1 but ROMS has not been tested with these versions. It is likely that the new version will work fine with ROMS but use these new versions at your own risk.

	~~'''SCORPIO''' can use NetCDF (version 4.3.3+), PnetCDF (version 1.6.0+) or ADIOS (version 2.6.0+) for I/O. {{note}} ADIOS is not implemented in ROMS.~~

	Ideally, the NetCDF version should be built with MPI, which requires that it be linked with an MPI-enabled version of HDF5. Optionally, NetCDF can be built with DAP support, which introduces a dependency on CURL. Additionally, HDF5, itself, introduces dependencies on LIBZ and (optionally) SZIP. If your NetCDF is built with DAP support, you will need to edit '''cmake/TryNetCDF_DAP.c''', changing line 8 from <div class="box">#if NC_HAS_DAP==1</div> to <div class="box">#if NC_HAS_DAP==1 \|\| NC_HAS_DAP2==1 \|\| NC_HAS_DAP4==1</div>

	*~~Download~~ the '''~~SCORPIO~~''' source code:<div class="box">> ~~git clone~~ https://github.com/~~E3SM-Project~~/~~scorpio<br~~ /~~>> cd scorpio<br~~ /~~>> git checkout tags~~/~~scorpio~~-v1.1.~~6<br />~~</div>~~If your NetCDF is built with DAP support, you will need to edit '''cmake/TryNetCDF_DAP.c''', changing line 8 from~~ <div class="box">~~#if NC_HAS_DAP==1~~</~~div~~> to <~~div class="box"~~>~~#if NC_HAS_DAP==1 \|\| NC_HAS_DAP2==1 \|\| NC_HAS_DAP4==1~~</div>		*First download the '''PIO''' source code as a gzipped tarball or via ''git clone'':<div class="box">> wget https://github.com/NCAR/ParallelIO/releases/download/pio2_5_4/pio-2.5.4.tar.gz</div>or<div class="box">> git clone https://github.com/NCAR/ParallelIO<br />> cd ParallelIO<br />> git checkout tags/pio2_5_4</div>

	Configure '''~~SCORPIO~~''':<div class="box">> cd ..<br />> mkdir ~~build_scorpio~~<br />> cd ~~build_scorpio~~<br />> CC=mpicc CXX=mpicxx FC=mpifort cmake \<br /> -DNetCDF_PATH=/path/to/netcdf \<br /> -DPnetCDF_PATH=/path/to/pnetcdf \<br /> -DMPIEXEC=/path/submission/scipt \<br /> -DMPIEXEC_MAX_NUMPROCS=8 -DMPIEXEC_NUMPROC_FLAG=-n \<br /> -DPIO_FILESYSTEM_HINTS=gpfs -DPIO_ENABLE_TIMING=OFF -DPIO_ENABLE_TESTS=ON \<br /> -DCMAKE_INSTALL_PREFIX=/~~scorpio~~/install/path ../~~scorpio~~</div>{{note}} Note that ''-DMPIEXEC'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br /> echo "Usage: mpirun -n # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>		Configure '''PIO''':<div class="box">> cd ..<br />> mkdir build_pio<br />> cd build_pio<br />> CC=mpicc CXX=mpicxx FC=mpifort cmake \<br /> -DNetCDF_PATH=/path/to/netcdf \<br /> -DPnetCDF_PATH=/path/to/pnetcdf \<br /> -DMPIEXEC=/path/submission/scipt \<br /> -DMPIEXEC_MAX_NUMPROCS=8 -DMPIEXEC_NUMPROC_FLAG=-n \<br /> -DPIO_FILESYSTEM_HINTS=gpfs -DPIO_ENABLE_TIMING=OFF -DPIO_ENABLE_TESTS=ON \<br /> -DCMAKE_INSTALL_PREFIX=/pio/install/path ../ParallelIO</div>{{note}} Note that ''-DMPIEXEC'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br /> echo "Usage: mpirun -n # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>

	*Compile, test, and install:<div class="box">> make -j 6<br />> make -j 6 tests<br />> ctests<br />> make install</div>		*Compile, test, and install:<div class="box">> make -j 6<br />> make -j 6 tests<br />> ctests<br />> make install</div>

Robertson at 19:38, 6 January 2025

2025-01-06T19:38:46Z

← Older revision		Revision as of 19:38, 6 January 2025
Line 7:		Line 7:

	To obtain '''ARPACK''', execute the following:		To obtain '''ARPACK''', execute the following:
	<div class="box">> git clone https://github.com/myroms/roms_libs.git<br />> cd ~~roms_libraries~~/ARPACK</div>		<div class="box">> git clone https://github.com/myroms/roms_libs.git<br />> cd roms_libs/ARPACK</div>
	To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.		To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.

Arango: /* ARPACK (ARnoldi PACKage) */

2024-12-28T02:03:43Z

ARPACK (ARnoldi PACKage)

← Older revision		Revision as of 02:03, 28 December 2024
Line 7:		Line 7:

	To obtain '''ARPACK''', execute the following:		To obtain '''ARPACK''', execute the following:
	<div class="box">> git clone https://github.com/myroms/~~roms_libraries~~<br />> cd roms_libraries/ARPACK</div>		<div class="box">> git clone https://github.com/myroms/roms_libs.git<br />> cd roms_libraries/ARPACK</div>
	To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.		To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.

Robertson at 20:46, 27 December 2024

2024-12-27T20:46:18Z

← Older revision		Revision as of 20:46, 27 December 2024
Line 7:		Line 7:

	To obtain '''ARPACK''', execute the following:		To obtain '''ARPACK''', execute the following:
	<div class="box">> ~~svn checkout~~ https://~~www~~.myroms~~.org/svn/src~~/~~libs/ARPACK~~<br />> cd ARPACK</div>		<div class="box">> git clone https://github.com/myroms/roms_libraries<br />> cd roms_libraries/ARPACK</div>
	To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.		To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.

Line 18:		Line 18:
	ROMS supports version 8.0 or higher since the '''NUOPC''' layer includes the native CoupleSets for '''ESM''' applications with nested grids. Various types of connections are possible, like '''nest-to-nest''', '''fine-to-coarse''', or '''coarse-to-fine'''. The user decides which nested grids to connect.		ROMS supports version 8.0 or higher since the '''NUOPC''' layer includes the native CoupleSets for '''ESM''' applications with nested grids. Various types of connections are possible, like '''nest-to-nest''', '''fine-to-coarse''', or '''coarse-to-fine'''. The user decides which nested grids to connect.

	*~~Downloaded~~ from github:<div class="box">> git clone https://github.com/esmf-org/esmf.git esmf<br />> cd esmf<br />> git checkout tags/ESMF_8_1_0</div>		*Download from github:<div class="box">> git clone https://github.com/esmf-org/esmf.git esmf<br />> cd esmf<br />> git checkout tags/ESMF_8_1_0</div>

	Configure with environment variables:<div class="box">> export ESMF_DIR=/projects/dmcs_1/src/esmf<br />> export ESMF_COMM=mvapich2<br />> export ESMF_COMPILER=intel<br />> export ESMF_BOPT=O<br />> export ESMF_OPTLEVEL=3<br />> export ESMF_ABI=64<br />> export ESMF_INSTALL_PREFIX=/ESMF/install/dir ESMF_NETCDF="/path/to/netcdf/bin/nc-config"<br />> export ESMF_NETCDF_LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lcurl -lhdf5_hl -lhdf5 -lm -lz -lcurl"<br />> export ESMF_NETCDF_LIBPATH="/path/to/netcdf/lib /path/to/hdf5/lib"<br />> export ESMF_SHARED_LIB_BUILD=OFF<br />> export ESMF_MPIRUN=/path/to/cluster/submission/script</div>{{note}} Note that ''ESMF_MPIRUN'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-np''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -np N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-np" ]<br />then<br /> echo "Usage: mpirun -np # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>		Configure with environment variables:<div class="box">> export ESMF_DIR=/projects/dmcs_1/src/esmf<br />> export ESMF_COMM=mvapich2<br />> export ESMF_COMPILER=intel<br />> export ESMF_BOPT=O<br />> export ESMF_OPTLEVEL=3<br />> export ESMF_ABI=64<br />> export ESMF_INSTALL_PREFIX=/ESMF/install/dir ESMF_NETCDF="/path/to/netcdf/bin/nc-config"<br />> export ESMF_NETCDF_LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lcurl -lhdf5_hl -lhdf5 -lm -lz -lcurl"<br />> export ESMF_NETCDF_LIBPATH="/path/to/netcdf/lib /path/to/hdf5/lib"<br />> export ESMF_SHARED_LIB_BUILD=OFF<br />> export ESMF_MPIRUN=/path/to/cluster/submission/script</div>{{note}} Note that ''ESMF_MPIRUN'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-np''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -np N prog<br /># runs N copies of executable "prog" in parallel. The script must hide all<br /># of the system specific details, such as going through a queueing system<br /># and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br /># stdout and stderr.<br /><br />if [ "$1" != "-np" ]<br />then<br /> echo "Usage: mpirun -np # prog"<br /> exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>

Robertson at 20:02, 27 July 2021

2021-07-27T20:02:56Z

← Older revision		Revision as of 20:02, 27 July 2021
Line 8:		Line 8:
	To obtain '''ARPACK''', execute the following:		To obtain '''ARPACK''', execute the following:
	<div class="box">> svn checkout https://www.myroms.org/svn/src/libs/ARPACK<br />> cd ARPACK</div>		<div class="box">> svn checkout https://www.myroms.org/svn/src/libs/ARPACK<br />> cd ARPACK</div>
	To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.		To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">> make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.

Robertson at 20:00, 27 July 2021

2021-07-27T20:00:15Z

@@ Line 23: / Line 23: @@
 *Build, test, and install:<div class="box">> make -j 10<br />> make all_tests<br />> make install</div>
 ==NetCDF (Network Common Data Form)==
@@ Line 61: / Line 71: @@
 * Build, test, and install '''NetCDF-C''':<div class="box">> make -j 5<br />> make check<br />> make install</div>
 ==PIO (Parallel I/O)==

Robertson at 19:58, 27 July 2021

2021-07-27T19:58:41Z

← Older revision		Revision as of 19:58, 27 July 2021
Line 6:		Line 6:
	Serial and parallel legacy libraries applied to solve large eigenvalue problems using either the Implicitly Restarted Arnoldi Method ('''IRAM''') for sparse matrices or the Lanczos algorithm for symmetric matrices. It includes a subset of the '''BLAS''' and '''LAPACK''' libraries. Some of its functions are used in 4D-Var and the adjoint-based stability analysis propagators.		Serial and parallel legacy libraries applied to solve large eigenvalue problems using either the Implicitly Restarted Arnoldi Method ('''IRAM''') for sparse matrices or the Lanczos algorithm for symmetric matrices. It includes a subset of the '''BLAS''' and '''LAPACK''' libraries. Some of its functions are used in 4D-Var and the adjoint-based stability analysis propagators.

	~~It is distributed in the ROMS~~ '''~~Lib/~~ARPACK''' ~~subdirectory~~. To compile, customize '''~~Lib/ARPACK/~~ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''my_build_paths''' script ~~described~~ in [~~[roms~~.in]].		To obtain '''ARPACK''', execute the following:
			<div class="box">> svn checkout https://www.myroms.org/svn/src/libs/ARPACK<br />> cd ARPACK</div>
			To compile, customize '''ARmake.inc''' for the desired compiler and its flags for the serial ('''FC''' and '''FFLAGS''') and parallel ('''PFC''' and '''PFFLAGS''') versions of the library. Once configured type:<div class="box">make lib plib</div> to build '''libarpack.a''' and '''libparpack.a'''. Once the libraries are built you can move them anywhere you like to make it easier to tell ROMS where to find them through environmental variables or with the '''[[my_build_paths]]''' script.


			{{note}} '''Note:''' Prior to SVN revision 1080 (July 23, 2021), '''ARPACK''' was distributed in the ROMS '''Lib/ARPACK''' subdirectory. For more information about why this was changed see TRAC ticket [https://www.myroms.org/projects/src/ticket/891 #891]

	==ESMF (Earth System Modeling Framework)==		==ESMF (Earth System Modeling Framework)==
Line 62:		Line 67:

	The MCT library is used to couple ROMS atmosphere models (like WRF) and wave models (like SWAN and WW3). For example, SWAN sends to ROMS arrays of wave height, wavelength, average wave periods at the surface and near the bottom, wave propagation direction, near-bottom orbital velocity, and wave-energy dissipation rate. ROMS provides SWAN arrays of water depth, sea-surface elevation, and current velocity. Data exchange between SWAN and ROMS occurs at user-defined synchronization intervals. The frequency of data exchange depends on the application. If the exchanged fields fluctuate rapidly, more frequent synchronization is required. However, data exchange increases run time, so experience is required to determine the optimum synchronization interval for each application.		The MCT library is used to couple ROMS atmosphere models (like WRF) and wave models (like SWAN and WW3). For example, SWAN sends to ROMS arrays of wave height, wavelength, average wave periods at the surface and near the bottom, wave propagation direction, near-bottom orbital velocity, and wave-energy dissipation rate. ROMS provides SWAN arrays of water depth, sea-surface elevation, and current velocity. Data exchange between SWAN and ROMS occurs at user-defined synchronization intervals. The frequency of data exchange depends on the application. If the exchanged fields fluctuate rapidly, more frequent synchronization is required. However, data exchange increases run time, so experience is required to determine the optimum synchronization interval for each application.

			MCT is available via github:

			<div class="box">> git clone https://github.com/MCSclimate/MCT<br />> cd MCT<br />> ./configure<br />> make</div>

	==PIO (Parallel I/O)==		==PIO (Parallel I/O)==

Robertson at 21:22, 27 May 2021

2021-05-27T21:22:58Z

@@ Line 23: / Line 23: @@
 Open-source, machine-independent libraries for storing self-describing scientific data into portable, scalable, appendable, and sharable computer files. It provides '''API''' in several computer languages to create files, to write data into a file, to read data from the file, and access information about the dataset dimensions, variables, and attributes.
-ROMS uses '''NetCDF''' for all its input and output data management. Its '''NetCDF3''' or '''NetCDF4''' type files can be processed using the standard library developed and distributed [http://doi.org/10.5065/D6H70CW6 UCAR/Unidata], the Parallel-IO ('''PIO''') developed at ANL/NCAR ([https://doi.org/10.1177/1094342011428143 Dennis ''et al''., 2012]; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE Hartnett and Edwards, 2021]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library available in [https://e3sm.org/scorpio-parallel-io-library E3SM]. We have migrated to the Fortran-90 '''API''' interface,  which consists of a library and one or two module files. See the [[makefile]] and [[build_Script| build script]] for how to tell ROMS where these are located. You will have to use a copy of the library and module files that were compiled using the same compiler you are using to compile ROMS - this is especially true for the module files.
+ROMS uses '''NetCDF''' for all its input and output data management. Its '''NetCDF3''' or '''NetCDF4''' type files can be processed using the standard library developed and distributed [http://doi.org/10.5065/D6H70CW6 UCAR/Unidata], the Parallel-IO ('''PIO''') developed at ANL/NCAR ([https://doi.org/10.1177/1094342011428143 Dennis ''et al''., 2012]; [https://www.researchgate.net/publication/348169990_THE_PARALLELIO_PIO_CFORTRAN_LIBRARIES_FOR_SCALABLE_HPC_PERFORMANCE Hartnett and Edwards, 2021]), or the Software for Cashing Output and Reads for Parallel I/O ('''SCORPIO''') library available in [https://e3sm.org/scorpio-parallel-io-library E3SM]. We have migrated to the Fortran-90 '''API''' interface,  which consists of a library and one or two module files. See the [[makefile]] and [[build_roms| build script]] for how to tell ROMS where these are located. You will have to use a copy of the library and module files that were compiled using the same compiler you are using to compile ROMS - this is especially true for the module files.
 ==MCT (Model Coupling Toolkit)==
@@ Line 51: / Line 83: @@
 *If you downloaded the gzipped tarball, you should run ''make clean'' because some Fortran .mod files are included and need to be removed:<div class="box">> make clean</div>
-*Configure '''PIO''':<div class="box">> configure CC=mpicc CXX=mpicxx FC=mpifort CFLAGS=-O3 FFLAGS=-O3 FCFLAGS=-O3 \<br />    CPPFLAGS="-I/path/to/netcdf/include -I/path/to/hdf5/include -I/path/to/pnetcdf/include" \<br />    LDFLAGS="-L/path/to/netcdf/lib -L/path/to/hdf5/lib -L/path/to/pnetcdf/lib" \<br />    LIBS="-lhdf5_hl -lhdf5 -lm -lcurl -lz" \<br />    --with-mpiexec=/path/to/cluster/submission/script \<br />    --prefix=/pio/install/path --enable-fortran --disable-timing \<br />    --disable-silent-rules --disable-shared</div>{{note}} Note that ''--with-mpiexec'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br />#    runs N copies of executable "prog" in parallel. The script must hide all<br />#    of the system specific details, such as going through a queueing system<br />#    and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br />#    stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br />        echo "Usage: mpirun -n # prog"<br />        exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>
+*Configure '''PIO''':<div class="box">> ./configure CC=mpicc CXX=mpicxx FC=mpifort CFLAGS=-O3 FFLAGS=-O3 FCFLAGS=-O3 \<br />      CPPFLAGS="-I/path/to/netcdf/include -I/path/to/hdf5/include -I/path/to/pnetcdf/include" \<br />      LDFLAGS="-L/path/to/netcdf/lib -L/path/to/hdf5/lib -L/path/to/pnetcdf/lib" \<br />      LIBS="-lhdf5_hl -lhdf5 -lm -lcurl -lz" \<br />      --with-mpiexec=/path/to/cluster/submission/script \<br />      --prefix=/pio/install/path --enable-fortran --disable-timing \<br />      --disable-silent-rules --disable-shared</div>{{note}} Note that ''--with-mpiexec'' is only needed for running test on clusters that use scheduling systems like SLURM. Your submission script should take '''-n''', '''N''' (number of processes requested), and '''programName''' (in that order) as arguments to construct the appropriate command to run using the scheduling system on your cluster. A SLURM example is provided below:<div class="box"># 1) <this_script> -n N prog<br />#    runs N copies of executable "prog" in parallel. The script must hide all<br />#    of the system specific details, such as going through a queueing system<br />#    and/or calling a system specific mpirun script with modified arguments.<br />#<br /># 2) The output of the application must arrive at the calling shell via<br />#    stdout and stderr.<br /><br />if [ "$1" != "-n" ]<br />then<br />        echo "Usage: mpirun -n # prog"<br />        exit 1<br />fi<br /><br />num_procs=$2<br />shift 2<br />prog=$*<br /><br />echo "srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog"<br />srun --mpi=pmi2 --partition=my_partition --nodes=1 --ntasks=${num_procs} --cpus-per-task=1 --time=00:10:00 --exclusive --export=ALL $prog</div>
 *Build, test, and install '''PIO''':<div class="box">> make -j 5<br />> make check<br />> make install</div>{{warning}} '''Warning:''' If you are using the Intel compilers, you will probably need to remove "'''test_darray_async_many'''" from line 50 of '''test/cunit/run_tests.sh''' for ''make check'' to complete without errors. Details [https://github.com/NCAR/ParallelIO/issues/1867 here].

Arango: /* MCT (Model Coupling Toolkit) */

2021-05-24T01:37:36Z

MCT (Model Coupling Toolkit)

← Older revision		Revision as of 01:37, 24 May 2021
Line 27:		Line 27:
	==MCT (Model Coupling Toolkit)==		==MCT (Model Coupling Toolkit)==

	~~We used the~~ Model ~~Coupling Toolkit~~ (~~MCT;~~ http://www-unix.mcs.anl.gov/mct/; Larson, et al., 2004~~; Jacob et al., 2005) to couple ROMS with SWAN (Warner et al., in prep~~)~~, and ROMS to WRF. MCT is an open-source software library, distributed as a set of Fortran90 modules for constructing a coupled model system from individual component models~~. Each component model has its own grid and runs on its own set of processors. The MCT provides protocols for decomposition and allocation of model grids among different processors, efficient transfer of data fields between the different models, and interpolation algorithms for the data fields that are transferred. SWAN sends to ROMS arrays of wave height, wavelength, average wave periods at the surface and near the bottom, wave propagation direction, near-bottom orbital velocity, and wave-energy dissipation rate. ROMS provides to SWAN arrays of water depth, sea-surface elevation, and current velocity. Data exchange between SWAN and ROMS occurs at user-defined synchronization intervals. The frequency of data exchange depends on the application. If the exchanged fields fluctuate rapidly, more frequent synchronization is required. However, data exchange increases run time, so experience is required to determine the optimum synchronization interval for each application.		Open-source library distributed as a set of Fortran-90 modules for constructing a coupled model system from Earth System Model (ESM) components ([http://www-unix.mcs.anl.gov/mct Larson et al., 2004]). Each component model has its own grid and runs on its own set of processors. The MCT library provides protocols for decomposition and allocation of model grids among different processors, efficient transfer of data fields between the different models, and interpolation algorithms for the data fields that are transferred.

			The MCT library is used to couple ROMS atmosphere models (like WRF) and wave models (like SWAN and WW3). For example, SWAN sends to ROMS arrays of wave height, wavelength, average wave periods at the surface and near the bottom, wave propagation direction, near-bottom orbital velocity, and wave-energy dissipation rate. ROMS provides SWAN arrays of water depth, sea-surface elevation, and current velocity. Data exchange between SWAN and ROMS occurs at user-defined synchronization intervals. The frequency of data exchange depends on the application. If the exchanged fields fluctuate rapidly, more frequent synchronization is required. However, data exchange increases run time, so experience is required to determine the optimum synchronization interval for each application.

	==PIO (Parallel I/O)==		==PIO (Parallel I/O)==