Difference between revisions of "4DVar Tutorial Introduction"

From WikiROMS
Jump to navigationJump to search
(44 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<div class="title">4-Dimensional Variational (4D-Var) Data Assimilation Tutorial</div>__NOTOC__
<div class="title">4-Dimensional Variational (4D-Var) Data Assimilation Tutorial</div>__NOTOC__


{{warning}}This page is under construction{{warning}}
 
This tutorial shows several examples of ROMS 4D-Var data assimilation algorithms in the California Current System (CCS), 1/3 degree resolution ([[Options#WC13|WC13]]).




Line 9: Line 10:
<div style="clear: both"></div>
<div style="clear: both"></div>


==Download the code==
All of the additional scripts necessary to run the 4D-Var tests are available from the ROMS Subversion (SVN) repository. The 4D-Var test cases are for the California Current System, 1/3 degree resolution, ('''WC13''').


<div class="box"><nowiki>> svn checkout --username joeroms https://www.myroms.org/svn/src/test my_test</nowiki></div>
<section begin=setup />
==Model Set-up==
The [[Options#WC13|WC13]] model domain is shown in Fig. 1 and has open boundaries along the northern, western, and southern edges of the model domain.
 
[[Image:hraw_coarse_transect.png|500px|thumb|center|<center>'''Fig. 1:''' Model Bathymetry with 37&deg;N Transect and Target Area</center>]]
<div style="clear: both"></div>
 
In the tutorial, you will perform a 4D-Var data assimilation cycle that spans the period 3-6 January, 2004. The 4D-Var control vector ''&delta;'''''z''' is comprised of increments to the initial conditions, ''&delta;'''''x'''(''t<sub>0</sub>''), surface forcing, &delta;'''f'''(''t''), and open boundary conditions, &delta;'''b'''(''t''). The ''prior'' initial conditions, '''x<sub>b</sub>'''(''t<sub>0</sub>''), are taken from the sequence of 4D-Var experiments described by [[Bibliography#MooreAM_2011b|Moore ''et al.'' (2011b)]] in which data were assimilated every 7 days during the period July 2002- December 2004. The ''prior'' surface forcing, '''f<sub>b</sub>'''(''t''), takes the form of surface wind stress, heat flux, and a freshwater flux computed using the ROMS bulk flux formulation, and using near surface air data from COAMPS ([[Bibliography#DoyleJD_2009|Doyle ''et al.'', 2009]]). Clamped open boundary conditions are imposed on (''u'',''v'') and tracers, and the ''prior'' boundary conditions, '''b<sub>b</sub>'''(''t''), are taken from the global ECCO product ([[Bibliography#WunschC_2007|Wunsch and Heimbach, 2007]]). The free-surface height and vertically integrated velocity components are subject to the usual Chapman and Flather radiation conditions at the open boundaries. The ''prior'' surface forcing and open boundary conditions are provided daily and linearly interpolated in time. Similarly, the increments ''&delta;'''''f'''(''t'') and ''&delta;'''''b'''(''t'') are also computed daily and linearly interpolated in time.
 
The observations assimilated into the model are satellite SST, satellite SSH in the form of a gridded product from Aviso, and hydrographic observations of temperature and salinity collected from Argo floats and during the GLOBEC/LTOP and CalCOFI cruises off the coast of Oregon and southern California, respectively. The observation locations are illustrated in Fig. 2.
 
{|align="center"
|+style="caption-side:bottom;"|'''Figure 2:''' WC13 Observations
|-
|[[Image:aviso_ssh_2019.png|thumb|400px|<center>'''a)''' Aviso SSH</center>]]
|[[Image:blended_sst_2019.png|thumb|400px|<center>'''b)''' Blended SST</center>]]
|-
|[[Image:in_situ_T_2019.png|thumb|400px|<center>'''c)''' In Situ Temperature</center>]]
|[[Image:in_situ_S_2019.png|thumb|400px|<center>'''d)''' In Situ Salinity</center>]]
|-
|}<section end=setup />
 
==Download Lectures and Exercises==
 
The workshop Agenda can be found [https://www.myroms.org/Workshops/4DVAR2019/Agenda.pdf here]. Links to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop] lectures are provided below. These lectures are meant to supplement this 4D-Var tutorial.
 
*'''Lecture 1:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_1_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">4D-Var: Some Basics</span>
*'''Lecture 2:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_2_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">The Mechanics of 4D-Var</span>
*'''Lecture 3:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_3_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Dual 4D-Var</span>
*'''Lecture 4:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_4_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observing System Simulation Experiments (OSSEs)</span>
*'''Lecture 5:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_5_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observation Impact &amp; Observation Sensitivity</span>
*'''Lecture 6:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_6_2019.pdf PDF]</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The MARACOOS analysis-forecast system</span>
*'''Lecture 7:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_7_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Array Modes</span>
*'''Lecture 8:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_8_2019.pdf PDF]-->PDF</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The West Coast Ocean Forecasting System (WCOFS)</span>
*'''Lecture 9:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Lectures/Lecture_9_2019.pdf PDF]-->PDF</span> &nbsp; &nbsp; &nbsp; <span class="twilightBlue">The CeNCOOS and PacIOOS analysis-forecast systems</span>
 
Below are the tutorials pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:


==Directory Structure==
*'''Tutorial 01:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_01_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Explanation of cpp options, ocean.in, s4dvar.in (I4DVAR)</span>
*'''Tutorial 02:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_02_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Multiple outer loops</span>
*'''Tutorial 03:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_03_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercises 1 and 2</span>
*'''Tutorial 04:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_04_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Calculation of prior error standard deviations</span>
*'''Tutorial 05:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_05_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Explanation of cpp options, ocean.in, s4dvar.in (4D-PSAS)</span>
*'''Tutorial 06:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_06_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Semi-variograms</span>
*'''Tutorial 07:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_07_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">ERDDAP data server and management tools</span>
*'''Tutorial 08:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_08_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercises 3 & 4</span>
*'''Tutorial 09:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_09_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Computing normalization coefficients for covariance models</span>
*'''Tutorial 10:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_10_2019.pdf PDF_Arango] | [http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_10a_2019.pdf PDF_Wilkin]</span> &nbsp; <span class="twilightBlue">Building your observation files</span>
*'''Tutorial 11:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_11_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Observation Impact & Observation Sensitivity</span>
*'''Tutorial 12:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_12_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercise 5</span>
*'''Tutorial 13:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_13_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Using ERDDAP to view observation impact information</span>
*'''Tutorial 14:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_14_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Discussion of exercise 6 & 7</span>
*'''Tutorial 15:''' &nbsp; <span style="font-size: 75%"><!--[http://www.myroms.org/Workshops/4DVAR2019/Tutorials/Tutorial_15_2019.pdf PDF]--></span> &nbsp; <span class="twilightBlue">Putting it all together</span>
 
Below are the exercises pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:
 
*'''Exercise 1:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_1_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Incremental, Strong Constraint 4D-Var</span>
*'''Exercise 2:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_2_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">I4D-Var with Multiple Outer-loops</span>
*'''Exercise 3:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_3_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Dual Formulation 4D-Var - 4D-PSAS</span>
*'''Exercise 4:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_4_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Weak Constraint Dual Formulation 4D-Var</span>
*'''Exercise 5:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_5_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Analysis Cycle Observation Impacts</span>
*'''Exercise 6:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_6_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Analysis Cycle Observation Sensitivity</span>
*'''Exercise 7:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_7_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Reduced-Rank Array Modes</span>
*'''Exercise 8:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_8_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Forecast Cycle Observation Impacts</span>
*'''Exercise 9:''' &nbsp; <span style="font-size: 75%">[http://www.myroms.org/Workshops/4DVAR2019/Exercises/EXERCISE_9_2019.pdf PDF]</span> &nbsp; <span class="twilightBlue">Forecast Cycle Observation Sensitivities</span>
 
Below are the homeworks pertinent to the [https://www.myroms.org/index.php?page=4DVAR_2019_info 2019 ROMS 4D-Var workshop]:
 
*'''Homework 1:''' &nbsp; &nbsp; <span class="twilightBlue">Building the standard deviation file for user model configuration</span>
*'''Homework 2:''' &nbsp; &nbsp; <span class="twilightBlue">Building the file of normalization coefficients for the prior error covariance matrix for user model configuration</span>
*'''Homework 3:''' &nbsp; &nbsp; <span class="twilightBlue">Build observation file for user model configuration</span>
 
==Download Tutorial Files==
All of the data files and scripts necessary to run the 4D-Var tests discussed in this tutorial are available from the ROMS Subversion (SVN) repository. To download, execute the following command (replacing <span class="red">joe_roms</span> with your ROMS username):
<div class="box"><span class="red"><nowiki>> svn checkout --username joe_roms https://www.myroms.org/svn/src/test my_test</nowiki></span></div>
or
<div class="box"><span class="red"><nowiki>> svn checkout https://www.myroms.org/svn/src/test my_test</nowiki></span></div>
if your local username in your machine matches your ROMS username.
 
==Tutorial Directory Structure==
When the checkout is complete, you will have the following directory structure under your <span class="twilightBlue">my_test</span> directory.
When the checkout is complete, you will have the following directory structure under your <span class="twilightBlue">my_test</span> directory.


   <span class="twilightBlue">
   <span class="twilightBlue">/WC13                               Main California Current System 4D-Var applications
/WC13                           Main California Current System application
       /ARRAY_MODES                   Stabilized representer matrix array modes and clipping
       /ARRAY_MODES             Stabilized representer matrix array modes and clipping
       /Data                         Input data directory
       /Data                     Input data directory
       /Functionals                   Analytical expressions header files
       /Functionals             Analytical expression header file
       /I4DVAR                       Primal form of incremental, strong constraint 4D-Var, I4D-Var
       /I4DVAR                   Primal form of incremental, strong constraint 4D-Var, I4D-Var
       /Normalization                 4D-Var error covariance normalization coefficients
      /I4DVAR_impact            I4D-Var observation impact
       /plotting                     4D-Var plotting scripts (Matlab and ROMS plotting package)
       /Normalization           4D-Var error covariance normalization coefficients
       /PSAS                         Dual form of 4D-Var, Physical-space Statistical Analysis System, 4D-PSAS
       /plotting                 4D-Var plotting scripts (Matlab and ROMS plotting package)
       /PSAS_analysis_impact          4D-PSAS analysis observation impact
       /PSAS                     Dual form of 4D-Var, Physical-space Statistical Analysis System, 4D-PSAS
       /PSAS_analysis_sensitivity    4D-PSAS analysis observation sensitivity (adjoint of 4D-PSAS)
       /PSAS_impact              4D-PSAS observation impact
       /PSAS_forecast_impact          4D-PSAS forecast observation impact
       /PSAS_sensitivity        4D-PSAS observation sensitivity (adjoint of 4D-PSAS)
       /PSAS_forecast_sensitivity    4D-PSAS forecast observation sensitivity (adjoint of 4D-PSAS)
       /R4DVAR                  Dual form of 4D-Var, indirect representer method, R4D-Var
      /R4DVAR_impact            R4D-Var observation impact
       /R4DVAR_sensitivity      R4D-Var observation sensitivity (adjoint of R4D-Var)
   </span>
   </span>
Mostly all the directories have a <span class="twilightBlue">Readme</span> file with detailed instructions for configuring, compiling, running, and plotting the results.
==References==
The technical description of the algorithms and application used in this tutorial are described in Moore ''et al.'' ([[Bibliography#MooreAM_2011a|2011a]], [[Bibliography#MooreAM_2011b|b]], [[Bibliography#MooreAM_2011c|c]]).

Revision as of 13:30, 2 August 2019

4-Dimensional Variational (4D-Var) Data Assimilation Tutorial


This tutorial shows several examples of ROMS 4D-Var data assimilation algorithms in the California Current System (CCS), 1/3 degree resolution (WC13).



Model Set-up

The WC13 model domain is shown in Fig. 1 and has open boundaries along the northern, western, and southern edges of the model domain.

Fig. 1: Model Bathymetry with 37°N Transect and Target Area

In the tutorial, you will perform a 4D-Var data assimilation cycle that spans the period 3-6 January, 2004. The 4D-Var control vector δz is comprised of increments to the initial conditions, δx(t0), surface forcing, δf(t), and open boundary conditions, δb(t). The prior initial conditions, xb(t0), are taken from the sequence of 4D-Var experiments described by Moore et al. (2011b) in which data were assimilated every 7 days during the period July 2002- December 2004. The prior surface forcing, fb(t), takes the form of surface wind stress, heat flux, and a freshwater flux computed using the ROMS bulk flux formulation, and using near surface air data from COAMPS (Doyle et al., 2009). Clamped open boundary conditions are imposed on (u,v) and tracers, and the prior boundary conditions, bb(t), are taken from the global ECCO product (Wunsch and Heimbach, 2007). The free-surface height and vertically integrated velocity components are subject to the usual Chapman and Flather radiation conditions at the open boundaries. The prior surface forcing and open boundary conditions are provided daily and linearly interpolated in time. Similarly, the increments δf(t) and δb(t) are also computed daily and linearly interpolated in time.

The observations assimilated into the model are satellite SST, satellite SSH in the form of a gridded product from Aviso, and hydrographic observations of temperature and salinity collected from Argo floats and during the GLOBEC/LTOP and CalCOFI cruises off the coast of Oregon and southern California, respectively. The observation locations are illustrated in Fig. 2.

Figure 2: WC13 Observations
a) Aviso SSH
b) Blended SST
c) In Situ Temperature
d) In Situ Salinity

Download Lectures and Exercises

The workshop Agenda can be found here. Links to the 2019 ROMS 4D-Var workshop lectures are provided below. These lectures are meant to supplement this 4D-Var tutorial.

  • Lecture 1:   PDF   4D-Var: Some Basics
  • Lecture 2:   PDF   The Mechanics of 4D-Var
  • Lecture 3:   PDF   Dual 4D-Var
  • Lecture 4:   PDF   Observing System Simulation Experiments (OSSEs)
  • Lecture 5:   PDF   Observation Impact & Observation Sensitivity
  • Lecture 6:   PDF       The MARACOOS analysis-forecast system
  • Lecture 7:   PDF   Array Modes
  • Lecture 8:   PDF       The West Coast Ocean Forecasting System (WCOFS)
  • Lecture 9:   PDF       The CeNCOOS and PacIOOS analysis-forecast systems

Below are the tutorials pertinent to the 2019 ROMS 4D-Var workshop:

  • Tutorial 01:   PDF   Explanation of cpp options, ocean.in, s4dvar.in (I4DVAR)
  • Tutorial 02:   PDF   Multiple outer loops
  • Tutorial 03:     Discussion of exercises 1 and 2
  • Tutorial 04:   PDF   Calculation of prior error standard deviations
  • Tutorial 05:   PDF   Explanation of cpp options, ocean.in, s4dvar.in (4D-PSAS)
  • Tutorial 06:   PDF   Semi-variograms
  • Tutorial 07:   PDF   ERDDAP data server and management tools
  • Tutorial 08:     Discussion of exercises 3 & 4
  • Tutorial 09:   PDF   Computing normalization coefficients for covariance models
  • Tutorial 10:   PDF_Arango | PDF_Wilkin   Building your observation files
  • Tutorial 11:   PDF   Observation Impact & Observation Sensitivity
  • Tutorial 12:     Discussion of exercise 5
  • Tutorial 13:   PDF   Using ERDDAP to view observation impact information
  • Tutorial 14:     Discussion of exercise 6 & 7
  • Tutorial 15:     Putting it all together

Below are the exercises pertinent to the 2019 ROMS 4D-Var workshop:

  • Exercise 1:   PDF   Incremental, Strong Constraint 4D-Var
  • Exercise 2:   PDF   I4D-Var with Multiple Outer-loops
  • Exercise 3:   PDF   Dual Formulation 4D-Var - 4D-PSAS
  • Exercise 4:   PDF   Weak Constraint Dual Formulation 4D-Var
  • Exercise 5:   PDF   Analysis Cycle Observation Impacts
  • Exercise 6:   PDF   Analysis Cycle Observation Sensitivity
  • Exercise 7:   PDF   Reduced-Rank Array Modes
  • Exercise 8:   PDF   Forecast Cycle Observation Impacts
  • Exercise 9:   PDF   Forecast Cycle Observation Sensitivities

Below are the homeworks pertinent to the 2019 ROMS 4D-Var workshop:

  • Homework 1:     Building the standard deviation file for user model configuration
  • Homework 2:     Building the file of normalization coefficients for the prior error covariance matrix for user model configuration
  • Homework 3:     Build observation file for user model configuration

Download Tutorial Files

All of the data files and scripts necessary to run the 4D-Var tests discussed in this tutorial are available from the ROMS Subversion (SVN) repository. To download, execute the following command (replacing joe_roms with your ROMS username):

> svn checkout --username joe_roms https://www.myroms.org/svn/src/test my_test

or

> svn checkout https://www.myroms.org/svn/src/test my_test

if your local username in your machine matches your ROMS username.

Tutorial Directory Structure

When the checkout is complete, you will have the following directory structure under your my_test directory.

 /WC13                               Main California Current System 4D-Var applications
      /ARRAY_MODES                   Stabilized representer matrix array modes and clipping
      /Data                          Input data directory
      /Functionals                   Analytical expressions header files
      /I4DVAR                        Primal form of incremental, strong constraint 4D-Var, I4D-Var
      /Normalization                 4D-Var error covariance normalization coefficients
      /plotting                      4D-Var plotting scripts (Matlab and ROMS plotting package)
      /PSAS                          Dual form of 4D-Var, Physical-space Statistical Analysis System, 4D-PSAS
      /PSAS_analysis_impact          4D-PSAS analysis observation impact
      /PSAS_analysis_sensitivity     4D-PSAS analysis observation sensitivity (adjoint of 4D-PSAS)
      /PSAS_forecast_impact          4D-PSAS forecast observation impact
      /PSAS_forecast_sensitivity     4D-PSAS forecast observation sensitivity (adjoint of 4D-PSAS)
 

Mostly all the directories have a Readme file with detailed instructions for configuring, compiling, running, and plotting the results.

References

The technical description of the algorithms and application used in this tutorial are described in Moore et al. (2011a, b, c).