Ocean Modeling Discussion

ROMS/TOMS

Search for:
It is currently Sat Jul 20, 2019 4:32 pm




Post new topic Reply to topic  [ 3 posts ] 

All times are UTC

Author Message
PostPosted: Thu Aug 30, 2018 2:51 am 
Offline

Joined: Mon Jun 20, 2005 3:46 pm
Posts: 95
Location: South Australian Research and Development Institute
Is there such a thing as a matlab function that lists the filenames in a thredds / opendap catalog? (That is, an equivalent to "ls" for opendap?)

Typically the filenames contain a date string, eg http:\\thredds\...\...\xxx20091123T173000Zxxx.nc.

Normally I would loop through hour by hour, construct a filename by concatenation, read and concatenate the data on each loop. It struck me that this is probably an antediluvian approach; looping may be necessary but constructing a new filename on each loop may not be, if I could "ls" the filenames in the catalog. But a search through the matlab native netcdf tools, snctools, and elsewhere, failed to reveal how.


Top
 Profile  
Reply with quote  
PostPosted: Thu Aug 30, 2018 2:27 pm 
Offline
User avatar

Joined: Mon Apr 28, 2003 5:44 pm
Posts: 491
Location: Rutgers University
John L.,

There is no standard way that people set up their opendap catalogs that would make this straightforward.

If you are in control of the THREDDS service, then my suggestion would be to set up a time aggregation. Then you can request a data subset from a single stable URL by searching a range of dates/times.

If you are not in control of the service, you could configure your own THREDDS to build the aggregation if the remote file list matches a predictable regular expression.

There are some services where the filelist is maintained in a database also accessible by http. NASA's PO-DAAC is like that, and I have a Matlab function called podaac_get_dapurl.m that returns a list of OPeNDAP data URLs for a date range for a given PO-DAAC dataset "Short Name". e.g. for VIIRS SST https://podaac.jpl.nasa.gov/dataset/VII ... -L3U-v2.41 ...

>> shortname = 'VIIRS_NPP-OSPO-L3U-v2.41';
>> flist = podaac_get_dapurl(today,shortname);

John W.

_________________
John Wilkin: DMCS Rutgers University
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu


Top
 Profile  
Reply with quote  
PostPosted: Fri Aug 31, 2018 2:18 am 
Offline

Joined: Mon Jun 20, 2005 3:46 pm
Posts: 95
Location: South Australian Research and Development Institute
John W,

Thank you very much for the reply and I imagine I am not the only one who will find it useful.

I am not in control of the THREDDS service, but I must admit that the idea of configuring my own thredds sounds like acquiring yet another complex skill that will be redundant by the next time I need to use it. Since posting, I ran across thredds_dump.m and thredds_info.m (in snctools by John Evans) that looked promising (and possibly general across different opendaps), but before I could get around to downloading the netcdf Java library they require, someone else sent me a script which picks the filenames out of the scrambled mess output from the matlab function urlread.m, which reads the catalog and sends it to a scrambled mess (of ascii characters). As he said, it ain't pretty, but it worked, and it solved my immediate problem (though I am not sure it is any more general or simpler than my old method, described in my first post).

I was wondering if maybe John Evans would weigh in on this.

Thanks again.

John L.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group