creat boundary files from Hycom data

Discussion about analysis, visualization, and collaboration tools and techniques

Moderators: arango, robertson

Post Reply
Message
Author
Vera
Posts: 15
Joined: Wed Apr 26, 2017 5:23 pm
Location: Sun Yat-sen University

creat boundary files from Hycom data

#1 Post by Vera » Thu May 18, 2017 8:45 am

I will appreciate it if anyone can give me some advice.
Before creating boundary files(by pycnal), I used the make_remap_weights_file.py, and the hycom grid was made by the get_hycom_GLBa0.08_Palau_grid.py. I just changed the domain.
when I run "python3 make_remap_weights_file.py", the error message is:
Traceback (most recent call last):
File "make_remap_weights_file.py", line 7, in <module>
srcgrd = pycnal_toolbox.Grid_HYCOM.get_nc_Grid_HYCOM('./HYCOM_GLBa0.08_PALAU_grid.nc')
File "/usr/local/pycnal/lib/python3.5/site-packages/pycnal_toolbox/Grid_HYCOM/get_nc_Grid_HYCOM.py", line 47, in get_nc_Grid_HYCOM
h[j,i] = depth_bnds[bottom[j,i]+1]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
I think there must be something wrong with the hycom grid, any suggestions?
Thanks in advance.
Vera

User avatar
kate
Posts: 3700
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: creat boundary files from Hycom data

#2 Post by kate » Thu May 18, 2017 5:14 pm

I think there must be something wrong with the hycom grid, any suggestions?
No, the problem is not with the HYCOM grid. The problem is that Python is getting picky as it matures. This:

Code: Select all

h[j,i] = depth_bnds[bottom[j,i]+1]
needs to be:

Code: Select all

h[j,i] = depth_bnds[int(bottom[j,i])+1]
I need to collect my current code and push the updates.

Vera
Posts: 15
Joined: Wed Apr 26, 2017 5:23 pm
Location: Sun Yat-sen University

Re: creat boundary files from Hycom data

#3 Post by Vera » Fri May 19, 2017 2:06 am

That's very nice of you. Thank you, Kate!
I'm sorry that I didn't know python much.

Vera
Posts: 15
Joined: Wed Apr 26, 2017 5:23 pm
Location: Sun Yat-sen University

Re: creat boundary files from Hycom data

#4 Post by Vera » Tue May 23, 2017 1:02 pm

I met anyother strange problem again, please help me.
After I run make_bry_file.py, it will list all the hycom data I get,and give the following message:

Code: Select all

Load geographical grid from file
b'/home/le/PyCNAL-master/examples/Nanhai/data/HYCOM_GLBa0.08_salt_2015_001.nc'

Creating boundary file /home/le/PyCNAL-master/examples/Nanhai/bdry/HYCOM_GLBa0.08_salt_2015_001_ssh_bdry_Nanhai.nc
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "make_bdry_file.py", line 26, in do_file
    zeta = remap_bdry(file, 'ssh', src_grd, dst_grd, dst_dir=dst_dir)
  File "/home/le/PyCNAL-master/examples/Nanhai/remap_bdry.py", line 40, in remap_bdry
    src_var = cdf.variables[src_varname]
KeyError: 'ssh'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "make_bdry_file.py", line 83, in <module>
    results = p.map(partial_do_file, lst_file)
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
KeyError: 'ssh'
The first thing I'm comfused is, why the file is not a grid file after the sentence "Load geographical grid from file"? should I change this and how to change? I have no clue.
The second, which file is the target file of the function "do_file" in make_bry_file.py? I can't understand python well, sorry.
The last, how to solve my problem.
Thank you very much

User avatar
kate
Posts: 3700
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: creat boundary files from Hycom data

#5 Post by kate » Tue May 23, 2017 4:54 pm

Sorry, I've been playing with multiprocessing to use more than one core at a time. These are much, much easier to debug without that. Some of the examples have both the straightforward version of things as well as the multiprocessing version. See if you can find one of those to work from.

Vera
Posts: 15
Joined: Wed Apr 26, 2017 5:23 pm
Location: Sun Yat-sen University

Re: creat boundary files from Hycom data

#6 Post by Vera » Wed May 24, 2017 4:03 am

Thank you for your reply, Kate!
I set processes = 1, but it is still the same problem.
I find the straight version in Yellow_sea example, and imitate it(still using Hycom data).The error messeage is:
Load geographical grid from file
b'/home/le/PyCNAL-master/examples/Nanhai/data/HYCOM_GLBa0.08_salt_2015_001.nc'

Creating boundary file /home/le/PyCNAL-master/examples/Nanhai/bdry/HYCOM_GLBa0.08_salt_2015_001_ssh_bdry_Nanhai.nc
Traceback (most recent call last):
File "make_bry.py", line 46, in <module>
zeta = remap_bdry(file, 'ssh', src_grd, dst_grd, dst_dir=dst_dir)
File "/home/le/PyCNAL-master/examples/Nanhai/remap_bdry.py", line 40, in remap_bdry
src_var = cdf.variables[src_varname]
KeyError: 'ssh'
So I think the problem maybe isn't relative to Multi-process。

User avatar
kate
Posts: 3700
Joined: Wed Jul 02, 2003 5:29 pm
Location: IMS/UAF, USA

Re: creat boundary files from Hycom data

#7 Post by kate » Wed May 24, 2017 4:53 am

It's looking for 'ssh' in the HYCOM file. What variables are in your HYCOM file? Do you know how to read the traceback? Both agree that line 40 of remap_bdry.py is in trouble. So you look at that line and figure out if it should be succeeding, or why it isn't.

Vera
Posts: 15
Joined: Wed Apr 26, 2017 5:23 pm
Location: Sun Yat-sen University

Re: creat boundary files from Hycom data

#8 Post by Vera » Wed May 24, 2017 9:33 am

I have solved it, thank you.

Post Reply