Parallelization - Revision history

Robertson at 16:35, 5 August 2015

2015-08-05T16:35:12Z

← Older revision		Revision as of 16:35, 5 August 2015
Line 4:		Line 4:

	===ROMS tiling===		===ROMS tiling===
	~~<wikitex>~~ROMS supports serial, OpenMP, and MPI computations, with the user choosing between them at compile time. The serial code can also take advantage of multiple small tiles which can be sized to fit in cache. All are accomplished through domain decomposition in the horizontal. All of the horizontal operations are explicit with a relatively small footprint, so the tiling is a logical choice. Some goals in the parallel design of ROMS were:		ROMS supports serial, OpenMP, and MPI computations, with the user choosing between them at compile time. The serial code can also take advantage of multiple small tiles which can be sized to fit in cache. All are accomplished through domain decomposition in the horizontal. All of the horizontal operations are explicit with a relatively small footprint, so the tiling is a logical choice. Some goals in the parallel design of ROMS were:
	* Minimize code changes.		* Minimize code changes.
	* Don't hard-code the number of processes.		* Don't hard-code the number of processes.
Line 19:		Line 19:
	[[Image:Whole_grid.png\|center\|frame\|<center>Whole grid</center>]]		[[Image:Whole_grid.png\|center\|frame\|<center>Whole grid</center>]]
	<div style="clear:both;"></div>		<div style="clear:both;"></div>
	The computations are done over the cells inside the darker line; the cells are numbered 1 to '''Lm''' in the $\xi$-direction and 1 to '''Mm''' in the $\eta$-direction. Those looking ahead to running in parallel would be wise to include factors of two in their choice of '''Lm''' and '''Mm'''. ROMS will run in parallel with any values of '''Lm''' and '''Mm''', but the computations might not be load-balanced.		The computations are done over the cells inside the darker line; the cells are numbered 1 to '''Lm''' in the <math>\xi</math>-direction and 1 to '''Mm''' in the <math>\eta</math>-direction. Those looking ahead to running in parallel would be wise to include factors of two in their choice of '''Lm''' and '''Mm'''. ROMS will run in parallel with any values of '''Lm''' and '''Mm''', but the computations might not be load-balanced.
	~~</wikitex>~~
	===ROMS internal numbers===		===ROMS internal numbers===
	[[Image:Tile3.png\|center\|frame\|<center>Tiled grid with some internal ROMS parameters.</center>]]		[[Image:Tile3.png\|center\|frame\|<center>Tiled grid with some internal ROMS parameters.</center>]]

Kate: /* Code syntax */ added a line or so about NtileX

2009-11-17T00:06:09Z

Code syntax: added a line or so about NtileX

← Older revision		Revision as of 00:06, 17 November 2009
Line 54:		Line 54:
	What isn't obvious from this is that the argument '''TILE''' means different things depending on if we're using OpenMP or MPI:		What isn't obvious from this is that the argument '''TILE''' means different things depending on if we're using OpenMP or MPI:
	<div class="box"> #ifdef DISTRIBUTE<br> # define TILE MyRank<br> #else<br> # define TILE tile<br> #endif</div>		<div class="box"> #ifdef DISTRIBUTE<br> # define TILE MyRank<br> #else<br> # define TILE tile<br> #endif</div>
			Likewise, '''NtileX''' also depends on whether or not we're using MPI:
			<div class="box"> #ifdef DISTRIBUTE<br> NtileX(1:Ngrids)=1<br> #else<br> NtileX(1:Ngrids)=NtileI(1:Ngrids)<br> #endif</div>
	In other words, for MPI, '''TILE''' becomes the process number and the loop is only executed once.		In other words, for MPI, '''TILE''' becomes the process number and the loop is only executed once.

Robertson at 17:18, 16 November 2009

2009-11-16T17:18:49Z

Show changes

Kate: /* Input/output */ Ok, done?

2009-11-12T21:04:11Z

Input/output: Ok, done?

@@ Line 215: / Line 215: @@
           END IF
 </code></div>
 ===Other figures===

Kate: /* Input/output */

2009-11-12T21:02:09Z

Input/output

← Older revision		Revision as of 21:02, 12 November 2009
Line 176:		Line 176:
	An example of a routine which reads from disk is [[get_grid]], called from [[initial]]. Each MPI process calls [[get_grid]]:		An example of a routine which reads from disk is [[get_grid]], called from [[initial]]. Each MPI process calls [[get_grid]]:
	<div class="box"><code>		<div class="box"><code>
	CALL get_grid (ng, iNLM)		CALL get_grid (ng, iNLM)
	# ifdef DISTRIBUTE		# ifdef DISTRIBUTE
	CALL mp_bcasti (ng, iNLM, exit_flag)		CALL mp_bcasti (ng, iNLM, exit_flag)
	# endif		# endif
	if (exit_flag.ne.NoError) RETURN		if (exit_flag.ne.NoError) RETURN
	</code></div>		</code></div>
	If any one of the processes has trouble, it will enter into the '''exit_flag''' which is then shared by all.		If any one of the processes has trouble, it will enter into the '''exit_flag''' which is then shared by all.
Line 201:		Line 201:
	Within '''nf_fread2d''', we get to a call to the NetCDF library from just the master process:		Within '''nf_fread2d''', we get to a call to the NetCDF library from just the master process:
	<div class="box"><code>		<div class="box"><code>
	IF (InpThread) THEN		IF (InpThread) THEN
	status=nf90_get_var(ncid, ncvarid, wrk, start, total)		status=nf90_get_var(ncid, ncvarid, wrk, start, total)
	:		:
	END IF		END IF
	# ifdef DISTRIBUTE		# ifdef DISTRIBUTE
	CALL mp_bcasti (ng, model, status)		CALL mp_bcasti (ng, model, status)
	# endif		# endif
	IF (status.ne.nf90_noerr) THEN		IF (status.ne.nf90_noerr) THEN
	exit_flag=2		exit_flag=2
	ioerror=status		ioerror=status
	nf_fread2d=status		nf_fread2d=status
	RETURN		RETURN
	END IF		END IF
	</code></div>		</code></div>


	===Other figures===		===Other figures===

Kate at 21:00, 12 November 2009

2009-11-12T21:00:37Z

← Older revision		Revision as of 21:00, 12 November 2009
Line 3:		Line 3:
	[[Image:parallel_tile_map.png\|center\|frame\|<center>Parallel Tile Partitions</center>]]		[[Image:parallel_tile_map.png\|center\|frame\|<center>Parallel Tile Partitions</center>]]
	<div style="clear:both;"></div>		<div style="clear:both;"></div>

			===ROMS tiling===
	ROMS supports serial, OpenMP, and MPI computations, with the user choosing between them at compile time. The serial code can also take advantage of multiple small tiles which can be sized to fit in cache. All are accomplished through domain decomposition in the horizontal. All of the horizontal operations are explicit with a relatively small footprint, so the tiling is a logical choice. Some goals in the parallel design of ROMS were:		ROMS supports serial, OpenMP, and MPI computations, with the user choosing between them at compile time. The serial code can also take advantage of multiple small tiles which can be sized to fit in cache. All are accomplished through domain decomposition in the horizontal. All of the horizontal operations are explicit with a relatively small footprint, so the tiling is a logical choice. Some goals in the parallel design of ROMS were:
	* Minimize code changes.		* Minimize code changes.

Kate: version before lunch, a little more coming.

2009-11-12T20:59:33Z

version before lunch, a little more coming.

Show changes

Kate: copying stuff from the manual, phase I

2009-11-12T19:57:00Z

copying stuff from the manual, phase I

@@ Line 1: / Line 1: @@
 <div class="title">Parallelization</div>
+<wikitex>
 [[Image:parallel_tile_map.png|center|frame|<center>Parallel Tile Partitions</center>]]
 <div style="clear:both;"></div>
 [[Image:tile.png|center|frame|<center>Parallel Tile</center>]]
 <div style="clear:both;"></div>

Robertson at 18:06, 20 March 2008

2008-03-20T18:06:04Z

← Older revision		Revision as of 18:06, 20 March 2008
Line 9:		Line 9:


	[[Image:~~adjoint_communications~~.png\|center\|frame\|<center>ROMS ~~Adjoint~~ Communications</center>]]		[[Image:communications.png\|center\|frame\|<center>ROMS Nonlinear East-West Communications</center>]]
	<div style="clear:both;"></div>		<div style="clear:both;"></div>


	[[Image:adjoint_communications.png\|center\|frame\|<center>ROMS Adjoint Communications</center>]]		[[Image:communications_NS.png\|center\|frame\|<center>ROMS Nonlinear North-South Communications</center>]]
			<div style="clear:both;"></div>


			[[Image:adjoint_communications.png\|center\|frame\|<center>ROMS Adjoint East-West Communications</center>]]
			<div style="clear:both;"></div>


			[[Image:adjoint_communications_NS.png\|center\|frame\|<center>ROMS Adjoint North-South Communications</center>]]
	<div style="clear:both;"></div>		<div style="clear:both;"></div>

Robertson at 18:41, 19 March 2008

2008-03-19T18:41:49Z

← Older revision		Revision as of 18:41, 19 March 2008
Line 1:		Line 1:
	<div class="title">Parallelization</div>		<div class="title">Parallelization</div>

	[[Image:~~communications~~.png\|center\|frame\|<center>ROMS ~~Forward~~ Communications</center>]]		[[Image:parallel_tile_map.png\|center\|frame\|<center>Parallel Tile Partitions</center>]]
			<div style="clear:both;"></div>


			[[Image:tile.png\|center\|frame\|<center>Parallel Tile</center>]]
			<div style="clear:both;"></div>


			[[Image:adjoint_communications.png\|center\|frame\|<center>ROMS Adjoint Communications</center>]]
	<div style="clear:both;"></div>		<div style="clear:both;"></div>