Large Files on Linux Systems

Facts, news, and guidance about ROMS software

Moderators: arango, robertson

Post Reply
Message
Author
rroberts
Posts: 3
Joined: Wed Jun 04, 2003 8:31 pm
Location: UNSW@ADFA

Large Files on Linux Systems

#1 Post by rroberts » Mon Jan 23, 2006 7:33 pm

What do I need to do to get the capability for large Netcdf files on a Linux system?

I had modified my earlier version of Roms to write binary files. Now I am switching to a newer version and want to use Netcdf files for output. Unfortunately the program bombs when the file reaches the 2 GB size.

How do I implement the LFS capability?

Thanks in advance.

lefevre

#2 Post by lefevre » Mon Jan 30, 2006 9:11 am

Hi,

You may to install the netcdf library with large file support : version >= 3.6
With ROMS UCLA version, that work fine !

jerome

User avatar
shchepet
Posts: 185
Joined: Fri Nov 14, 2003 4:57 pm

netCDF files larger that 2GBytes

#3 Post by shchepet » Mon Feb 06, 2006 7:16 pm

At first, I consider it a bad habit, because it can be avoided by arranging your output
into sequence of files, rather than single big file. Secondly, it restricts portability: you
must have an operating system which has Large File Support and you can no longer
open it from a 32-bit Matlab, and any other piece of software which uses earlier
versions of netCDF.

If you still desire to have netCDF files larger than 2Gbytes, here is what you have to do:

1. Make sure that you have the latest version of netcdf library, I am using

3.6.1 --- beta3

do not use earlier release of 3.6, as it is buggy specifically in large offset part.
Please let me know, it you are aware of a newer version.

2. When you create netcdf file, make sure that you enable 64BIT offset in netCDF:
it would not do it by default. To do so, find all occasions of NF_CREATE and
replace NF_CLOBBER with NF_64BIT_OFFSET, so it should look like

CALL NF_CREATE (filename, NF_64BIT_OFFSET, ncid)

More information about this matter can be found in

http://www.unidata.ucar.edu/software/ne ... 005fCREATE

rroberts
Posts: 3
Joined: Wed Jun 04, 2003 8:31 pm
Location: UNSW@ADFA

#4 Post by rroberts » Mon Feb 06, 2006 7:35 pm

Thanks.

I have it working.

Post Reply