Will there be JEDI/UFO functions replacing matlab scripts for observation preprocessing?

[stef]

Hi,

I was wondering if you could comment on to which extent the observation
preprocessing currently implemented in Matlab will be re-implemented in
the JEDI framework? Specifically, I'm looking at "Quality Control in UFO"

https://jointcenterforsatellitedataassi ... index.html

and tried to find something appropriate for making e.g. super-obserservations.
They have several thinning and a "SuperOb" filter, but if I understand correctly,
these would have to be modified to somehow take account of the ROMS grid.
Or can this already be done with JEDI?

Or will you keep the Matlab functions described in Tutorial 10:

http://www.myroms.org/Workshops/4DVAR20 ... 0_2019.pdf

Thanks for your help!

[arango]

JEDI uses IODA-type NetCDF files, which are enhanced NetCDF-4 files with Group partitions. A group is like a subdirectory in the NetCDF file for organization. I did write the roms2ioda.m Matlab script, which I updated today. For more information, please check myroms/roms_matlab#5. Currently, I am working on restructuring the ROMS observation operators (myroms/roms#62), which will be available soon. The plan is to have IODA-type files that can be used in both ROMS native 4D-Var drivers and the ROMS-JEDI family of various data assimilation algorithms.

I haven't used the available IODA/UFO filters in JEDI. They are used mainly in atmospheric model applications. We will need area-averaged and /or time-averaged H(x) in ocean models. These filters are not available yet in JEDI, but they will be available in the native ROMS 4D-Var drivers soon, when I am done with the restructuring that uses IODA-type NetCDF-4 files instead.

[stef]

Wow, exciting, thanks for the update!

One reason I asked is that I saw the ioda converters for marine data:

https://github.com/JCSDA-internal/ioda- ... src/marine

I think some of these can generate ioda obs files from the file formats delivered by
the data providers (argo, copernicus, satellite centers etc.). It seems promising,
but I don't know how comprehensive it will be.
For example I think argo profiles are not available, just the gridded climatology.

Those may be useful regardless of whether the preprocessing (QC, binning/superobs etc.)
is done in UFO directly or Matlab/Python. I was just very curious about whether you have
a fixed roadmap/opinion regarding this.

[arango]

I know the IODA Python scripts, but I have never used them. I am adding a few variables to the MetaData group to exclusive use by native ROMS 4D-Var and ROMS-JEDI. You will need an additional script to append such unavailable variables in the standard IODA file. That shouldn't be a problem because NetCDF files are flexible, and we can add any variables, attributes, etc., without affecting their functionality and usage.

[stef]

I am adding a few variables to the MetaData group to exclusive use by native ROMS 4D-Var and ROMS-JEDI.

Good to know, thank you!