roms.in - Revision history

Robertson at 18:35, 5 May 2025

2025-05-05T18:35:40Z

Arango: /* Parallel I/O (PIO and SCORPIO) Parameters */

2021-06-08T02:26:50Z

Parallel I/O (PIO and SCORPIO) Parameters

← Older revision		Revision as of 02:26, 8 June 2021
Line 199:		Line 199:
	==Parallel I/O (PIO and SCORPIO) Parameters==		==Parallel I/O (PIO and SCORPIO) Parameters==
	<section begin=PIO />		<section begin=PIO />
	* Choose the input and output NetCDF library to use. For example, the user could choose to use the PIO library for writing but still use the standard library for reading. <div class="box">! [1] Standard NetCDF-3 or NetCDF-4 library<br />! [2] Parallel-IO from PIO or SCORPIO library (MPI, MPI-IO applications)<br /><br /> [[Variables#INP_LIB\|INP_LIB]] = 2<br /> [[Variables#OUT_LIB\|OUT_LIB]] = 2</div>		* Choose the input and output NetCDF library to use. For example, the user could choose to use the PIO library for writing but still use the standard library for reading. <span class="twilightBlue">To use this Parallel I/O strategy, the '''PIO''' or '''SCORPIO''' library must be linked to ROMS at compile time and the [[PIO_LIB]] CPP option needs to be activated. It is only available in distributed-memory applications since it uses MPI-IO.</span> <div class="box">! [1] Standard NetCDF-3 or NetCDF-4 library<br />! [2] Parallel-IO from PIO or SCORPIO library (MPI, MPI-IO applications)<br /><br /> [[Variables#INP_LIB\|INP_LIB]] = 2<br /> [[Variables#OUT_LIB\|OUT_LIB]] = 2</div>

	* PIO and SCORPIO offer several methods for reading/writing NetCDF files. SCORPIO also offers ADIOS but that is not implemented in ROMS. Depending on the build of the PIO or SCORPIO libraries, not all the I/O types are available. If the NetCDF library does not support parallel I/O, methods 3 and 4 are not available. Currently, NetCDF4/HDF5 data compression is possible with method 3 during serial write.<div class="box">! [0] parallel read and parallel write of PnetCDF (CDF-5 type files, not recommended because of post-processing)<br />! [1] parallel read and parallel write of NetCDF3 (64-bit offset)<br />! [2] serial read and serial write of NetCDF3 (64-bit offset)<br />! [3] parallel read and serial write of NetCDF4/HDF5<br />! [4] parallel read and parallel write of NETCDF4/HDF5<br /><br /> [[Variables#PIO_METHOD\|PIO_METHOD]] = 2</div>		* PIO and SCORPIO offer several methods for reading/writing NetCDF files. SCORPIO also offers ADIOS but that is not implemented in ROMS. Depending on the build of the PIO or SCORPIO libraries, not all the I/O types are available. If the NetCDF library does not support parallel I/O, methods 3 and 4 are not available. Currently, NetCDF4/HDF5 data compression is possible with method 3 during serial write.<div class="box">! [0] parallel read and parallel write of PnetCDF (CDF-5 type files, not recommended because of post-processing)<br />! [1] parallel read and parallel write of NetCDF3 (64-bit offset)<br />! [2] serial read and serial write of NetCDF3 (64-bit offset)<br />! [3] parallel read and serial write of NetCDF4/HDF5<br />! [4] parallel read and parallel write of NETCDF4/HDF5<br /><br /> [[Variables#PIO_METHOD\|PIO_METHOD]] = 2</div>

Arango: /* Parallel I/O (PIO and SCORPIO) Parameters */

2021-05-24T00:51:20Z

Parallel I/O (PIO and SCORPIO) Parameters

← Older revision		Revision as of 00:51, 24 May 2021
Line 211:		Line 211:
	* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>		* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>

	* Parallel-IO (PIO or SCORPIO) rearranger flow-control direction flag for MPI communications between computational and I/O processes. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>		* Parallel-IO (PIO or SCORPIO) rearranger flow-control direction flag for MPI communications between computational and I/O processes. The flow algorithm controls the rate and volume of messages sent to any destination MPI process. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>

	* Parallel-IO (PIO or SCORPIO) rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>		* Parallel-IO (PIO or SCORPIO) rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>

Robertson at 18:45, 10 May 2021

2021-05-10T18:45:59Z

← Older revision		Revision as of 18:45, 10 May 2021
Line 205:		Line 205:
	* Parallel-IO tasks control parameters. Typically, it is advantageous and highly recommended to define the I/O decomposition in smaller number of processes for efficiency and to avoid MPI communications bottlenecks.<div class="box"> [[Variables#PIO_IOTASKS\|PIO_IOTASKS]] = 1 ! number of I/O processes to define<br /> [[Variables#PIO_STRIDE\|PIO_STRIDE]] = 1 ! stride in the MPI-rank between I/O processes<br /> [[Variables#PIO_BASE\|PIO_BASE]] = 0 ! offset for the first I/O process<br /> [[Variables#PIO_AGGREG\|PIO_AGGREG]] = 1 ! number of MPI-aggregators to use</div>		* Parallel-IO tasks control parameters. Typically, it is advantageous and highly recommended to define the I/O decomposition in smaller number of processes for efficiency and to avoid MPI communications bottlenecks.<div class="box"> [[Variables#PIO_IOTASKS\|PIO_IOTASKS]] = 1 ! number of I/O processes to define<br /> [[Variables#PIO_STRIDE\|PIO_STRIDE]] = 1 ! stride in the MPI-rank between I/O processes<br /> [[Variables#PIO_BASE\|PIO_BASE]] = 0 ! offset for the first I/O process<br /> [[Variables#PIO_AGGREG\|PIO_AGGREG]] = 1 ! number of MPI-aggregators to use</div>

	* Parallel-IO (PIO or SCORPIO) rearranger methods for moving data between computational and I/O processes. It provides the ability to rearrange data between computational and parallel I/O decompositions. Usually the Box rearrangement is more efficient. <div class="box">! [1] Box rearrangement<br />! [2] Subset rearrangement<br /><br /> [[Variables#PIO_REARR\|PIO_REARR]] = 1</div>		* Parallel-IO (PIO or SCORPIO) rearranger methods for moving data between computational and I/O processes. It provides the ability to rearrange data between computational and parallel I/O decompositions. Usually the Box rearrangement is more efficient.<div class="box">! [1] Box rearrangement<br />! [2] Subset rearrangement<br /><br /> [[Variables#PIO_REARR\|PIO_REARR]] = 1</div>
			**In the '''box''' method, data is rearranged from computational to I/O processes in a continuous manner to the data ordering in the file. Since the ordering of data between computational and I/O partitions may be different, the rearrangement will require all-to-all MPI communications. Also, notice that each computing tile may transfer data to one or more I/O processes.
			**In the '''subset''' method, each I/O process is associated with a subset of computing processes. The computing tile sends its data to a unique I/O process. The data on I/O processes may be more fragmented to the ordering on disk, which may increase the communications to the storage medium. However, the rearrangement scales better since all-to-all MPI communications are not required.

	* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>		* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>

Robertson at 18:17, 10 May 2021

2021-05-10T18:17:07Z

@@ Line 196: / Line 196: @@
 * This parameter could also be used to adjust constants in analytical functions at run time.
 <section end=user />
 ==NetCDF-4/HDF5 Compression Parameters==

Robertson at 16:06, 8 January 2021

2021-01-08T16:06:59Z

← Older revision		Revision as of 16:06, 8 January 2021
Line 1:		Line 1:
	<div class="title">Standard Input Script - <span class="red">roms.in</span></div>		<div class="title">Standard Input Script - <span class="red">roms.in</span></div>

	{{note}} '''Notice:''' In SVN revision 933 (January 26, 2019), all "<span class="red">~~roms_~~\.in</span>" files were renamed to <span class="red">roms_.in</span> and all <span class="red">ocean</span> ROMS executables were renamed to <span class="red">roms</span> in order to facilitate and clarify model coupling efforts. More information can be found in the ROMS repository [https://www.myroms.org/projects/src/ticket/794 Trac ticket #794].		{{note}} '''Notice:''' In SVN revision 933 (January 26, 2019), all "<span class="red">ocean_.in</span>" files were renamed to <span class="red">roms_.in</span> and all <span class="red">ocean</span> ROMS executables were renamed to <span class="red">roms</span> in order to facilitate and clarify model coupling efforts. More information can be found in the ROMS repository [https://www.myroms.org/projects/src/ticket/794 Trac ticket #794].

Robertson: Text replacement - "ocean_(.*)\.in" to "roms_$1\.in"

2020-07-21T18:48:57Z

Text replacement - "ocean_(.*)\.in" to "roms_$1\.in"

← Older revision		Revision as of 18:48, 21 July 2020
Line 1:		Line 1:
	<div class="title">Standard Input Script - <span class="red">roms.in</span></div>		<div class="title">Standard Input Script - <span class="red">roms.in</span></div>

	{{note}} '''Notice:''' In SVN revision 933 (January 26, 2019), all "<span class="red">~~ocean_~~.in</span>" files were renamed to <span class="red">roms_.in</span> and all <span class="red">ocean</span> ROMS executables were renamed to <span class="red">roms</span> in order to facilitate and clarify model coupling efforts. More information can be found in the ROMS repository [https://www.myroms.org/projects/src/ticket/794 Trac ticket #794].		{{note}} '''Notice:''' In SVN revision 933 (January 26, 2019), all "<span class="red">roms_\.in</span>" files were renamed to <span class="red">roms_.in</span> and all <span class="red">ocean</span> ROMS executables were renamed to <span class="red">roms</span> in order to facilitate and clarify model coupling efforts. More information can be found in the ROMS repository [https://www.myroms.org/projects/src/ticket/794 Trac ticket #794].

Robertson at 15:06, 16 January 2020

2020-01-16T15:06:40Z

← Older revision		Revision as of 15:06, 16 January 2020
Line 45:		Line 45:
	If using either HSIMT ([[Bibliography#WuH_2010a\|Wu and Zhu, 2010]]) or MPDATA ([[Bibliography#SmolarkiewiczPK_1998a\|Smolarkiewicz and Margolin, 1998]]) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.		If using either HSIMT ([[Bibliography#WuH_2010a\|Wu and Zhu, 2010]]) or MPDATA ([[Bibliography#SmolarkiewiczPK_1998a\|Smolarkiewicz and Margolin, 1998]]) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.

	*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>		*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, [1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

	*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer for adjoint-based algorithms can have different horizontal schemes, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box">[[Variables#ad_Hadvection\|ad_Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br />[[Variables#ad_Vadvection\|ad_Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>		*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer for adjoint-based algorithms can have different horizontal schemes, [1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box">[[Variables#ad_Hadvection\|ad_Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br />[[Variables#ad_Vadvection\|ad_Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

Robertson at 20:06, 14 January 2020

2020-01-14T20:06:00Z

← Older revision		Revision as of 20:06, 14 January 2020
Line 43:		Line 43:
	The user has the option of specifying the full Keyword or the first two letters, regardless if using uppercase or lowercase.		The user has the option of specifying the full Keyword or the first two letters, regardless if using uppercase or lowercase.

	If using either HSIMT (Wu and Zhu, 2010) or MPDATA (Margolin ~~and Smolarkiewicz~~, 1998) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.		If using either HSIMT ([[Bibliography#WuH_2010a\|Wu and Zhu, 2010]]) or MPDATA ([[Bibliography#SmolarkiewiczPK_1998a\|Smolarkiewicz and Margolin, 1998]]) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.

	*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>		*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

Robertson at 20:13, 10 January 2020

2020-01-10T20:13:15Z

← Older revision		Revision as of 20:13, 10 January 2020
Line 31:		Line 31:

	* Domain decomposition parameters for serial, distributed-memory or shared-memory configurations used to determine tile horizontal range indices ([[Istr]],[[Iend]]) and ([[Jstr]],[[Jend]]), [1:[[Ngrids]]] values are expected.<div class="box"> [[NtileI]] == 1 ! I-direction partition<br /> [[NtileJ]] == 1 ! J-direction partition</div>		* Domain decomposition parameters for serial, distributed-memory or shared-memory configurations used to determine tile horizontal range indices ([[Istr]],[[Iend]]) and ([[Jstr]],[[Jend]]), [1:[[Ngrids]]] values are expected.<div class="box"> [[NtileI]] == 1 ! I-direction partition<br /> [[NtileJ]] == 1 ! J-direction partition</div>

			==Tracer Advection Schemes==

			Set horizontal and vertical advection schemes for active and inert tracers. A different advection scheme is allowed for each tracer. For example, a positive-definite (monotonic) algorithm can be activated for salinity and inert tracers, while a different one is set for temperature.

			It is more advantageous to set the horizontal and vertical advection schemes for each tracer with switches instead of a single CPP flag for all of them. Positive-definite and monotonic algorithms (i.e., MPDATA and HSIMT) are appropriate and useful for positive fields like salinity, inert, biological, and sediment tracers. However, since the temperature has a dynamic range with negative and positive values in the ocean, other advection schemes are more appropriate.

			Currently, the following tracer advection schemes are available and are activated using the associated Keyword:
			:<div class="box">Keyword Advection Algorithm<br /><br />A4 4th-order Akima (horizontal/vertical)<br />C2 2nd-order centered differences (horizontal/vertical)<br />C4 4th-order centered differences (horizontal/vertical)<br />HSIMT 3th-order HSIMT with TVD limiter (horizontal/vertical)<br />MPDATA recursive flux corrected MPDATA (horizontal/vertical)<br />SPLINES parabolic splines reconstruction (only vertical)<br />SU3 split third-order upstream (horizontal/vertical)<br />U3 3rd-order upstresm-bias (only horizontal)</div>

			The user has the option of specifying the full Keyword or the first two letters, regardless if using uppercase or lowercase.

			If using either HSIMT (Wu and Zhu, 2010) or MPDATA (Margolin and Smolarkiewicz, 1998) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.

			*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

			*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer for adjoint-based algorithms can have different horizontal schemes, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box">[[Variables#ad_Hadvection\|ad_Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br />[[Variables#ad_Vadvection\|ad_Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>


	==Lateral Open Boundary Conditions Parameters==		==Lateral Open Boundary Conditions Parameters==

← Older revision		Revision as of 00:51, 24 May 2021
Line 211:		Line 211:
	* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>		* Parallel-IO (PIO or SCORPIO) rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>

	* Parallel-IO (PIO or SCORPIO) rearranger flow-control direction flag for MPI communications between computational and I/O processes. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>		* Parallel-IO (PIO or SCORPIO) rearranger flow-control direction flag for MPI communications between computational and I/O processes. The flow algorithm controls the rate and volume of messages sent to any destination MPI process. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>

	* Parallel-IO (PIO or SCORPIO) rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>		* Parallel-IO (PIO or SCORPIO) rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>

← Older revision		Revision as of 15:06, 16 January 2020
Line 45:		Line 45:
	If using either HSIMT ([[Bibliography#WuH_2010a\|Wu and Zhu, 2010]]) or MPDATA ([[Bibliography#SmolarkiewiczPK_1998a\|Smolarkiewicz and Margolin, 1998]]) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.		If using either HSIMT ([[Bibliography#WuH_2010a\|Wu and Zhu, 2010]]) or MPDATA ([[Bibliography#SmolarkiewiczPK_1998a\|Smolarkiewicz and Margolin, 1998]]) options, the user needs to set the same scheme for both horizontal and vertical advection to preserve monotonicity.

	*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>		*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer, [1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box"> [[Variables#Hadvection\|Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br /> [[Variables#Vadvection\|Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

	*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer for adjoint-based algorithms can have different horizontal schemes, 0[1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box">[[Variables#ad_Hadvection\|ad_Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br />[[Variables#ad_Vadvection\|ad_Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>		*Horizontal and vertical advection for each active (temperature and salinity) and inert tracer for adjoint-based algorithms can have different horizontal schemes, [1:[[Variables#NAT\|NAT]]+[[Variables#NPT\|NPT]],[[Variables#Ngrids\|Ngrids]]] values are expected.<div class="box">[[Variables#ad_Hadvection\|ad_Hadvection]] == U3 \ ! temperature<br /> U3 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)<br /><br />[[Variables#ad_Vadvection\|ad_Vadvection]] == C4 \ ! temperature<br /> C4 \ ! salinity<br /> HSIMT \ ! dye_01, inert(1)<br /> HSIMT ! dy2_02, inert(2)</div>

← Older revision		Revision as of 18:35, 5 May 2025
Line 197:		Line 197:
	<section end=user />		<section end=user />

	==Parallel I/O (PIO ~~and SCORPIO~~) Parameters==		==Parallel I/O (PIO) Parameters==
	<section begin=PIO />		<section begin=PIO />
	* Choose the input and output NetCDF library to use. For example, the user could choose to use the PIO library for writing but still use the standard library for reading. <span class="twilightBlue">To use this Parallel I/O strategy, the '''PIO~~''' or '''SCORPIO~~''' library must be linked to ROMS at compile time and the [[PIO_LIB]] CPP option needs to be activated. It is only available in distributed-memory applications since it uses MPI-IO.</span> <div class="box">! [1] Standard NetCDF-3 or NetCDF-4 library<br />! [2] Parallel-IO from PIO ~~or SCORPIO~~ library (MPI, MPI-IO applications)<br /><br /> [[Variables#INP_LIB\|INP_LIB]] = 2<br /> [[Variables#OUT_LIB\|OUT_LIB]] = 2</div>		* Choose the input and output NetCDF library to use. For example, the user could choose to use the PIO library for writing but still use the standard library for reading. <span class="twilightBlue">To use this Parallel I/O strategy, the '''PIO''' library must be linked to ROMS at compile time and the [[PIO_LIB]] CPP option needs to be activated. It is only available in distributed-memory applications since it uses MPI-IO.</span> <div class="box">! [1] Standard NetCDF-3 or NetCDF-4 library<br />! [2] Parallel-IO from the PIO library (MPI, MPI-IO applications)<br /><br /> [[Variables#INP_LIB\|INP_LIB]] = 2<br /> [[Variables#OUT_LIB\|OUT_LIB]] = 2</div>

	* PIO ~~and SCORPIO offer~~ several methods for reading/writing NetCDF files~~. SCORPIO also offers ADIOS but that is not implemented in ROMS~~. Depending on the build of the PIO ~~or SCORPIO~~ libraries, not all the I/O types are available. If the NetCDF library does not support parallel I/O, methods 3 and 4 are not available. Currently, NetCDF4/HDF5 data compression is possible with method 3 during serial write.<div class="box">! [0] parallel read and parallel write of PnetCDF (CDF-5 type files, not recommended because of post-processing)<br />! [1] parallel read and parallel write of NetCDF3 (64-bit offset)<br />! [2] serial read and serial write of NetCDF3 (64-bit offset)<br />! [3] parallel read and serial write of NetCDF4/HDF5<br />! [4] parallel read and parallel write of NETCDF4/HDF5<br /><br /> [[Variables#PIO_METHOD\|PIO_METHOD]] = 2</div>		* PIO offers several methods for reading/writing NetCDF files. Depending on the build of the PIO libraries, not all the I/O types are available. If the NetCDF library does not support parallel I/O, methods 3 and 4 are not available. Currently, NetCDF4/HDF5 data compression is possible with method 3 during serial write.<div class="box">! [0] parallel read and parallel write of PnetCDF (CDF-5 type files, not recommended because of post-processing)<br />! [1] parallel read and parallel write of NetCDF3 (64-bit offset)<br />! [2] serial read and serial write of NetCDF3 (64-bit offset)<br />! [3] parallel read and serial write of NetCDF4/HDF5<br />! [4] parallel read and parallel write of NETCDF4/HDF5<br /><br /> [[Variables#PIO_METHOD\|PIO_METHOD]] = 2</div>

	* Parallel-IO tasks control parameters. Typically, it is advantageous and highly recommended to define the I/O decomposition in smaller number of processes for efficiency and to avoid MPI communications bottlenecks.<div class="box"> [[Variables#PIO_IOTASKS\|PIO_IOTASKS]] = 1 ! number of I/O processes to define<br /> [[Variables#PIO_STRIDE\|PIO_STRIDE]] = 1 ! stride in the MPI-rank between I/O processes<br /> [[Variables#PIO_BASE\|PIO_BASE]] = 0 ! offset for the first I/O process<br /> [[Variables#PIO_AGGREG\|PIO_AGGREG]] = 1 ! number of MPI-aggregators to use</div>		* Parallel-IO tasks control parameters. Typically, it is advantageous and highly recommended to define the I/O decomposition in smaller number of processes for efficiency and to avoid MPI communications bottlenecks.<div class="box"> [[Variables#PIO_IOTASKS\|PIO_IOTASKS]] = 1 ! number of I/O processes to define<br /> [[Variables#PIO_STRIDE\|PIO_STRIDE]] = 1 ! stride in the MPI-rank between I/O processes<br /> [[Variables#PIO_BASE\|PIO_BASE]] = 0 ! offset for the first I/O process<br /> [[Variables#PIO_AGGREG\|PIO_AGGREG]] = 1 ! number of MPI-aggregators to use</div>

	* Parallel-IO ~~(PIO or SCORPIO)~~ rearranger methods for moving data between computational and I/O processes. It provides the ability to rearrange data between computational and parallel I/O decompositions. Usually the Box rearrangement is more efficient.<div class="box">! [1] Box rearrangement<br />! [2] Subset rearrangement<br /><br /> [[Variables#PIO_REARR\|PIO_REARR]] = 1</div>		* Parallel-IO rearranger methods for moving data between computational and I/O processes. It provides the ability to rearrange data between computational and parallel I/O decompositions. Usually the Box rearrangement is more efficient.<div class="box">! [1] Box rearrangement<br />! [2] Subset rearrangement<br /><br /> [[Variables#PIO_REARR\|PIO_REARR]] = 1</div>
	**In the '''box''' method, data is rearranged from computational to I/O processes in a continuous manner to the data ordering in the file. Since the ordering of data between computational and I/O partitions may be different, the rearrangement will require all-to-all MPI communications. Also, notice that each computing tile may transfer data to one or more I/O processes.		**In the '''box''' method, data is rearranged from computational to I/O processes in a continuous manner to the data ordering in the file. Since the ordering of data between computational and I/O partitions may be different, the rearrangement will require all-to-all MPI communications. Also, notice that each computing tile may transfer data to one or more I/O processes.
	**In the '''subset''' method, each I/O process is associated with a subset of computing processes. The computing tile sends its data to a unique I/O process. The data on I/O processes may be more fragmented to the ordering on disk, which may increase the communications to the storage medium. However, the rearrangement scales better since all-to-all MPI communications are not required.		**In the '''subset''' method, each I/O process is associated with a subset of computing processes. The computing tile sends its data to a unique I/O process. The data on I/O processes may be more fragmented to the ordering on disk, which may increase the communications to the storage medium. However, the rearrangement scales better since all-to-all MPI communications are not required.

	* Parallel-IO ~~(PIO or SCORPIO)~~ rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>		* Parallel-IO rearranger flag for MPI communication between computational and I/O processes. In some systems, the Point-to-Point communications is more efficient. <div class="box">! [0] Point-to-Point communications<br />! [1] Collective communications<br /><br />[[Variables#PIO_REARRCOM\|PIO_REARRCOM]] = 0</div>

	* Parallel-IO ~~(PIO or SCORPIO)~~ rearranger flow-control direction flag for MPI communications between computational and I/O processes. The flow algorithm controls the rate and volume of messages sent to any destination MPI process. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>		* Parallel-IO rearranger flow-control direction flag for MPI communications between computational and I/O processes. The flow algorithm controls the rate and volume of messages sent to any destination MPI process. Optimally, the MPI communications should be designed to send a modest number of messages evenly distributed across a number of processes. An excessive number of messages to a single MPI process can exhaust the buffer space which can affect efficiency or failure due to the slowdown in the retransmitting of dropped messages. It only sends messages (Isend) when the receiver is ready and has sufficient resources.<div class="box">! [0] Enable computational to I/O processes, and vice versa<br />! [1] Enable computational to I/O processes only<br />! [2] Enable I/O to computational processes only<br />! [3] Disable flow control<br /><br />[[Variables#PIO_REARRDIR\|PIO_REARRDIR]] = 0</div>

	* Parallel-IO ~~(PIO or SCORPIO)~~ rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>		* Parallel-IO rearranger options for MPI communications from computational to I/O processes (C2I). <div class="box"> [[Variables#PIO_C2I_HS\|PIO_C2I_HS]] = T ! Enable C2I handshake (T/F)<br />[[Variables#PIO_C2I_Send\|PIO_C2I_Send]] = F ! Enable C2I Isends (T/F)<br /> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = 64 ! Maximum pending C2I requests</div>

	* Parallel-IO ~~(PIO or SCORPIO)~~ rearranger options for MPI communications from I/O to computational processes (I2C). <div class="box"> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = F ! Enable I2C handshake (T/F)<br />[[Variables#PIO_I2C_Send\|PIO_I2C_Send]] = T ! Enable I2C Isends (T/F)<br />[[Variables#PIO_I2C_Preq\|PIO_I2C_Preq]] = 65 ! Maximum pending I2C requests</div><section end=PIO />		* Parallel-IO rearranger options for MPI communications from I/O to computational processes (I2C). <div class="box"> [[Variables#PIO_I2C_HS\|PIO_I2C_HS]] = F ! Enable I2C handshake (T/F)<br />[[Variables#PIO_I2C_Send\|PIO_I2C_Send]] = T ! Enable I2C Isends (T/F)<br />[[Variables#PIO_I2C_Preq\|PIO_I2C_Preq]] = 65 ! Maximum pending I2C requests</div><section end=PIO />

	==NetCDF-4/HDF5 Compression Parameters==		==NetCDF-4/HDF5 Compression Parameters==