ROMS git

Tutorial: Installing ROMS via git

I would like to explain how and why one would install ROMS via git-svn.

What is git?

Git is a distributed version control system available from [1]. That site also contains links to documentation and other resources, including the fabulous Pro Git book. I have also written about it here and on the ROMS blog. As git is a distributed system, any copy you make of a repository is complete unto itself, with history and possibly branches. If you download ROMS via git-svn, you now have an environment in which you can save your own changes, create new branches, and keep a history of what you have tried.

As a ROMS developer, I have write access to a branch at the Rutgers svn server, but that's not true of most users. As ROMS users, you still have opportunity to create and modify ROMS files that you might want to manage with some sort of versioning software. This is not easy with svn, in which each "sandbox" can only point to one repository. You've got to have one pointing to the myroms.org site in order to get updates. Then the easiest (but unsatisfactory) way to back up your changes is via tarball.

Prerequisites

You will need:

• svn
• git
• some Perl scripts which come with svn, but aren't always installed
• Perl

If everything is there and you are in a git sandbox, you can have this sort of exchange:

% git svn --version
git-svn version 1.7.1 (svn 1.6.5)

If everything is there and you are not in a git sandbox, you get this instead:

% git svn --version
fatal: Not a git repository (or any of the parent directories): .git
Already at toplevel, but .git not found
at /Users/kate/libexec/git-core/git-svn line 276

If the Perl scripts are not there, you get something more like:

% git svn --version
Can't locate SVN/Core.pm in @INC (@INC contains: /u1/uaf/kate/...

You might have the best luck with package managers rather than installing from source code.

Procedure

For the Hong Kong training, Dale asked me to set up the CIRCLE test problem, in three different flavors. I did so, but the initial conditions require the use of a C language Bessel function, changing the link rules for ROMS. This is the perfect example of why you would want to be able to create alternate branches. What I am going to lead you through here is setting up three branches:

• master, a copy of the myroms.org trunk
• circle, a branch with the special features of the CIRCLE problem
• my_stuff (or whatever name you like), a place to put your own changes

Setup

% git config --global user.name “me”
% git config --global user.email “me@work”
% git config --global color.ui “auto”

For the first two, substitute "me" with your real name in quotes and "me@work" with your real email address. The third is entirely optional, but provides more colorful output from git. For all, leave out the "--global" argument to have it apply to the current directory only. As it is, it will put information into a .gitconfig file in your home directory.

Downloading ROMS

We want to download a specific revision of ROMS rather than the entire history. As I write this, the current revision (visible at [2]) is 529. A recent prior version is 526, from just after the annual changing of the Copyright notice in all files. I'd like to fetch something just before the current version so you can see the update and merge procedures. Grabbing rev 526:

% git svn –r 526 clone --username name https://www.myroms.org/svn/src/trunk [MyDir]

Here, "--username name" is not needed if your ROMS userid is the same as your userid on the local computer, "name" being your ROMS name. If you leave off the MyDir argument, it will create a directory called "trunk".

Now you can change into the directory called MyDir or trunk and see all the ROMS directories. There will also be a hidden directory called ".git". One file there that's useful to look at is .git/config, containing information about other repositories it knows about. Since we haven't told it about other remote sites yet, it should look something like:

[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
[svn-remote "svn"]
url = https://www.myroms.org/svn/src/trunk
fetch = :refs/remotes/git-svn

Ignoring Files

Let's create a ".gitignore" file:

% cat > .gitignore
Build
ocean?
core*

We can add this to the list of files to be tracked by git:

% git add .gitignore
% git commit -m "adding .gitignore"

Without the "-m string" option, it will open up a text editor, waiting for a description for this commit.

Creating Branches

As we said above, we would like to have three branches at the end, one containing some code for a CIRCLE test problem. Let's start off with that:

% git branch circle
% git checkout circle
% git branch

This last command should just show that we are now in the circle branch:

* circle
master

Now to fetch the circle code from [3]. From the trunk directory, we can apply these changes to the circle branch:

% gunzip Circle.diffs.gz
% patch -p1 < Circle.diffs

We can see what we have with "git status":

% git status
# On branch circle
# Changed but not updated:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: Master/Module.mk
# modified: ROMS/Modules/mod_scalars.F
# modified: ROMS/Utility/checkdefs.F
# modified: ROMS/Utility/def_info.F
# modified: ROMS/Utility/wrt_info.F
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# Apps/
# makefile.circle
no changes added to commit (use "git add" and/or "git commit -a")

A "git commit -a" will add the updates to all the files that git is already tracking. For this case, we want it to track a couple of new things:

% git add Apps makefile.circle

Now we can commit the whole works:

% git commit -a -m "Circle problem"

When in this branch, one can run any of the three Circle cases:

• CIRLCE_BOX - a square grid with masking
• CIRCLE_ROUND - requires an external circular grid
• CIRCLE_POLAR - a donut-shaped domain

If you use build.bash, edit the line near the bottom invoking "make" and change it to "make -f makefile.circle".

Fetching Updates

Remember that we hadn't grabbed the absolute latest, greatest ROMS. We want to go back to the master branch to bring it in, but first, it's always good practice to make sure there are no stray files lying around:

% git status
# On branch circle
nothing to commit (working directory clean)

Now we can go back to the master branch and fetch the updates:

% git checkout master
% git svn rebase

A rebase is a reordering of the commits, in this case putting our changes after the history that's coming from the svn repository:

First, rewinding head to replay your work on top of it...
Applying: adding .gitignore

In this case, I want to keep the master branch as a clean copy of the ROMS trunk code (except for that .gitignore).

Merging and a Conflict

We've updated the master branch, but not the circle branch. Let's go take care of that:

% git checkout circle

I can simply show you the merge:

% git merge master

or I can show you a little conflict just for fun. Let's change ROMS/Version to look like:

ROMS/TOMS Framework: January 19, 2011
===================

Copyright (c) 2002-2011 The ROMS/TOMS Group
Licensed under a MIT/X style license
See License_ROMS.txt

svn: $HeadURL$
svn: $LastChangedBy$
svn: $LastChangedRevision$
svn: $LastChangedDate$
svn: $Id$

Notice that the svn stuff like "$HeadURL$" has not been expanded. Some of these tags are read by ROMS, so we can fill in useful information there, for at least the URL and author:

svn: $https://www.myroms.org/svn/src/trunk/$
svn: $arango$

Heck if I'm going to keep the revision number up to date, but go for it if you're so inclined.

Now we're ready for a fun merge:

% git merge master
error: Your local changes to 'ROMS/Version' would be overwritten by merge. Aborting.
Please, commit your changes or stash them before you can merge.

Oops, maybe not quite. Let's do that commit, then try again:

% git commit -a -m "having fun"
% git merge master

We got a conflict message and we can also see the list of troublesome files with:

% git status

Opening this file we see:

<<<<<<< HEAD
ROMS/TOMS Framework: January 19, 2011
=======
ROMS/TOMS Framework: January 18, 2011
>>>>>>> master

Merge conflicts always have the trouble shown between the "<<<<<<<" string and the ">>>>>>>" string, with the "=======" string dividing the two versions. Let's just clean up the mess, leaving Hernan's version and then tell git the problem has been resolved:

% git add ROMS/Version

We can check "git status" again to see that the red file has turned green and we are ready to commit:

% git commit

Conflicts are usually pretty painless, though I know I'm in for some fun merging Hernan's spatially variable bottom drag with my own version of the same feature. However, with git-svn, each change comes in independently and it's easier to isolate the changes. After updating the master, it is possible to literally apply one patch at a time with "git cherry-pick".

Another Branch

Above we said we want a place for our own changes as well as a place for the CIRCLE problem. We want to make the new branch to be a copy of the master branch. The command "git branch name" copies the current branch, so let's go back to master first:

% git checkout master
% git branch my_stuff
% git checkout my_stuff
% git branch

One purpose of this new branch could be for running the Tide_bay application. We have the code over in the circle branch, but it can't be compiled there due to the Bessel function weirdness. Let's fetch it over to this new branch:

% git checkout circle Apps/Tide_bay

We can see the new stuff with:

% git status

Clearly, we want to commit this here:

% git commit -m "Tide_bay setup"

Last Tips

• Remember to check that "git status" is clean before changing branches.
• Though a git archive can know about multiple other git repositories, it can only know about one svn site and it has to be set up during the "git svn clone" operation.
• Side by side directories can each point to different svn sites and be a "git remote" of the other (as discussed here [4]).

Cherry-picking Notes

I was asked for more on "git cherry-pick". Here's an example in two directories, one pointing to an Hernan repo, one pointing to my svn branch. Over in the arango directory:

% git svn rebase

This fetches the changes since last time, however long I've let it go. Ideally, I'd do a "git log" first to see where things stood before. Now I'll do it after:

% git log
commit 5d4b1e87f84c76e424567cf672a30b7b1623d315
Author: arango <arango@f091316a-d328-0410-a40a-876eff57d070>
Date: Tue Jan 18 16:38:32 2011 +0000

src:ticket:483

git-svn-id: https://www.myroms.org/svn/omlab/branches/arango@1361 f091316a-d

commit 91a7542ec4e99ca44b340600af3c63519c8f1d35
Author: arango <arango@f091316a-d328-0410-a40a-876eff57d070>
Date: Fri Jan 14 21:33:39 2011 +0000

src:ticket:482

git-svn-id: https://www.myroms.org/svn/omlab/branches/arango@1358 f091316a-d

commit 2445e4e62f73f54115d2d16c69d6be093b56b0ea
Author: arango <arango@f091316a-d328-0410-a40a-876eff57d070>
Date: Thu Jan 6 19:40:42 2011 +0000

src:ticket:481

and so on... In this case I can tell by dates which commits are new to me, since I last merged sometime between the Dec 2 and the Dec 30 commits. That brings the arango directory up to date.

I now go to my svn_src directory. I already told it that the arango directory is a remote, but I need to update its view of it:

% git remote update

This pulls over information from the arango repo into the local repo. The remotes I'm tracking can be viewed:

% git branch -a
* kate_svn
master
remotes/arango/master
remotes/cygnus/kate_svn
remotes/cygnus/master

Now I can pull the changes over one at a time and deal with any conflicts as they arise:

% git cherry-pick 4a46a1098e96a5

with the "4a4..." being the start of the SHA1 number corresponding to the commit I'm fetching over. Some come cleanly, some do not:

Automatic cherry-pick failed. After resolving the conflicts,
mark the corrected paths with 'git add <paths>' or 'git rm <paths>'
and commit the result with:

git commit -c 4a46a1098e96a5

I can see which files to clean up with "git status":

# Unmerged paths:
# (use "git reset HEAD <file>..." to unstage)
# (use "git add/rm <file>..." as appropriate to mark resolution)#
# both modified: ROMS/Nonlinear/main3d.F
# both modified: ROMS/Nonlinear/set_avg.F
# both modified: ROMS/Utility/wrt_avg.F
# both modified: ROMS/Utility/wrt_his.F

Just as in the merge conflict, I need to go into each file, then tell git that all is well with that file:

% git add ROMS/Nonlinear/main3d.F

That file will now show up in green instead of red in the "git status" report. Once all conflicted files are clear, it is safe to "git commit" and accept the merge message it brings up. If the cherry-pick is clean, the commit happens automatically and "git status" shows the all clear.