Talk:Git

Git

ROMS source code is distributed using both Git and Subversion(SVN). The Git repository includes the full history of changes to the ROMS source code. Details on using Subversion can be found here. There are command line and GUI Git clients available for nearly every operating system and a list of popular clients can be found here. This page will help you get started with downloading ROMS with Git.

Git Overview

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. All the ROMS/TOMS files are stored in a Git repository on www.myroms.org with access controlled by requiring authentication with the same ROMS Username/Password combination assigned to registered users of the ROMS Forum.

This Git repository is the official version of the code which only the developers are allowed to change. Users should download the ROMS code to their local machines using a git client. Don't attempt to use a regular web browser to browse or download files from the Git repository - there are much better tools for interacting with the code repository. See the notes below under the heading View the Repository.

We strongly recommend users always check out the current master version since this has the most recent updates and bug fixes. The tags are kept largely as a historical record of stable releases at the conclusion of major code upgrades.

If you are making changes of your own, keep them in a separate branch, leaving the master branch to track changes from the source. Git makes it so much easier to manage your own modifications than svn for those of us without write permission on the repository.

Below is a general description of how Git works. Please look at the Pro Git book for more detailed information. We have not yet tried any GUI clients but may add brief how-tos for the most popular GUIs at a later date.

Saving ROMS Login Credentials

To avoid having to type your ROMS password many times while executing the steps in the Downloading ROMS section below, we suggest you setup a git credential manager.

NOTE: If you have already setup a credential manager for other git repositories you should be able to add an entry for myroms following the documentation for the credential manager you are using.

There are a few different options for storing your ROMS login credentials in order to avoid having to type your username and password for every command. See here for more details. The simplest way to save your credentials is to use the built-in git-credential-store option. This option is very similar to the default way older Subversion clients store credentials so keep in mind that both your username and password will be stored without encryption in plain text. To use this method, execute the following:

   > git config --global credential.helper 'store --file ~/.my-credentials'

This will create a plain text file that only the file owner (you) has permission to access.

Saving Login Credentials Securely on Mac OS

If you are working on Mac OS there is a simple option for securely storing you credentials using the OS X Keychain. Simply execute:

   > git config --global credential.helper osxkeychain

A window may pop up asking for your system's password to allow the credentials to be stored in your Keychain.

Saving Login Credentials Securely on Linux

Saving your credentials securely on Linux is a bit more complex and requires extra utilities (GPG) and configuration. This processes is adapted from here and requires Git version 1.8.3 or higher and GNU Privacy Guard (GnuPG or GPG). If you already use GPG and have a trusted key already you can skip the first step.

• Create a GPG key:
  > gpg --gen-key
  gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
  
  Please select what kind of key you want:
  (1) RSA and RSA (default)
  (2) DSA and Elgamal
  (3) DSA (sign only)
  (4) RSA (sign only)
  Your selection? 1
  RSA keys may be between 1024 and 4096 bits long.
  What keysize do you want? (2048) 4096
  Requested keysize is 4096 bits
  Please specify how long the key should be valid.
  0 = key does not expire
  <n> = key expires in n days
  <n>w = key expires in n weeks
  <n>m = key expires in n months
  <n>y = key expires in n years
  Key is valid for? (0) 0
  Key does not expire at all
  Is this correct? (y/N) y
  
  GnuPG needs to construct a user ID to identify your key.
  
  Real name: Joe Roms
  Email address: joe_roms@host.com
  Comment: GPG Key
  You selected this USER-ID:
  "Joe Roms <joe_roms@host.com>"
  
  Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
  
  You need a Passphrase to protect your secret key.
  
  # You will be asked to type your passphrase twice
  
  We need to generate a lot of random bytes. It is a good idea to perform
  some other action (type on the keyboard, move the mouse, utilize the
  disks) during the prime generation; this gives the random number
  generator a better chance to gain enough entropy.
  gpg: key 33EA0E26 marked as ultimately trusted
  public and secret key created and signed.
  
  gpg: checking the trustdb
  gpg: 3 marginal(s) needed, 1 complete(s) needed, classic trust model
  gpg: depth: 0 valid: 2 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 2u
  pub 4096R/33EA0E26 2019-06-05
  Key fingerprint = 2CA2 904C A7DE CAEF 2266 70F6 1C74 375B 33EA 0E26
  uid Joe Roms <joe_roms@host.com>
  sub 4096R/5A0EDB59 2019-06-05

• Download the git-credential-netrc helper script from github and install it in a directory within your PATH. If there a multiple Git users on your system, you might want to have a system administrator place the script in /usr/bin or /usr/local/bin so all users will have access to it:
  > curl -o ~/bin/git-credential-netrc https://raw.githubusercontent.com/git/git/master/contrib/credential/netrc/git-credential-netrc.perl
  and give permissions to execute this script:
  > chmod 764 ~/bin/git-credential-netrc

• Make a .netrc file in your home directory in clear text using your favorite text editor:
  > cd ~
  > vi .netrc
  Enter the following text (replacing joe_roms and secret with your ROMS username and password) and save the file:
  machine www.myroms.org
  login joe_roms
  password secret
  protocol https
  You can add other blocks like the one above for other servers as well.

• Encrypt that file using the email address you entered earlier while generating your GPG key:
  > gpg -e -r joe_roms@host.com .netrc
  This will create a file named .netrc.gpg. You can now safely delete the .netrc file (the unecrypted one).

• Use the newly encrypted file for all git operations that require credentials:
  > git config --global credential.helper "netrc -f ~/.netrc.gpg -v"
  or for just a specific respoitory:
  > cd /path/to/repository
  > git config --local credential.helper "netrc -f ~/.netrc.gpg -v"
  The first option will save the configuration to ~/.gitconfig while the second option will save to .git/config for the current git repository.

From now on, any Git command using an HTTPS URL which requires authentication will decrypt that .netrc.gpg file and use the login/password associated to the server you are contacting. The first time, GPG will ask you for the passphrase of your GPG key, to decrypt the file. The other times, the gpg-agent launched automatically by the first GPG call will provide that passphrase for you.

Note: If your system's gpg program has a different name you will need to change the "git config" line accordingly. For example, on an UBUNTU machine the command would look something like this:

   > git config --global credential.helper "netrc -g gpg2 -f ~/.netrc.gpg -v"

Saving Login Credentials Securely on Windows

We will add this once tested but you can follow the steps here if you would like to try it yourself.

Downloading ROMS

WARNING: It is strongly suggested that you clone the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like dos2unix or recode. Even with these utilities you may still have problems compiling ROMS.

NOTE: To avoid having to type your ROMS password many times while executing the steps below, we suggest you first set up a git credential manager as described in the Saving ROMS Login Credentials section above. If you have already setup a credential manager for other git repositories you should be able to add an entry for myroms following the documentation for the credential manager you are using.

In order download source code from a git repository, git client software must be installed on your local machine. Most Linux distributions come with git, so shell commands may be used without installing additional software. The general form of git commands is:

   git action <repository>

To check-out the files from the ROMS repository master (latest version), enter (notice https instead of http):

   > git clone https://www.myroms.org/git/src MyDir

where MyDir is the destination directory on your local computer. It will be created if not found. If your want to avoid typing your username for all future code updates, change the command to:

   > git clone https://joe_roms@www.myroms.org/git/src MyDir

You only clone once because Git will keep track of the source, destination and a bunch of other information. For more detail on command line use and syntax, see the Pro Git book.

In order to synchronize the Git and Subversion repository revision information, we have created a Git filter that will insert the git hash of the last time a file was modified, similar to the Subversion $Id$ keyword. In order for the filter to work, it has to be explicitly enabled by the user. After the git clone command from above completes change into MyDir and execute the following:

   > git config filter.id.smudge ".git_filters/id.smudge %f"
   > git config filter.id.clean ".git_filters/id.clean %f"

In order to get the Git Id hashes to fill in, you will need to force the smudge filter to run on the source code you have already downloaded. This takes several minutes but only needs to be done once. For bash shells execute:

   > rm .git/index
   > git checkout HEAD -- "$(git rev-parse --show-toplevel)"

For csh/tcsh shells execute:

   > rm .git/index
   > git checkout HEAD -- "`git rev-parse --show-toplevel`"

If you look at the top of any ROMS source file downloaded from the git repository, you will see both the Git commit hash for the last time that file was modified and the corresponding SVN revision properties. For example, the first few lines of ROMS/Modules/mod_ncparam.F look like this:

#include "cppdefs.h"
MODULE mod_ncparam
!
!git $Id: b42658ab652647fca27d50af4afa15c3401fe544 $
!svn $Id: mod_ncparam.F 968 2019-06-17 16:02:48Z arango $
!================================================== Hernan G. Arango ===
! Copyright (c) 2002-2019 The ROMS/TOMS Group  !
! Licensed under a MIT/X style license  !
! See License_ROMS.txt  !
!=======================================================================

You can then use the 40 character (hexidecimal) Git hash to show more information about that code update:

> git show --name-only b42658ab652647fca27d50af4afa15c3401fe544

While you are getting your configuration in order, you also want to tell git who you are:

  > git config --global user.name "your name"
  > git config --global user.email "your@email"

This is important if you plan to ever commit your own modifications.

Several GUI front-ends to Git exist, allowing the user to have visual prompts to help them manage their files and credentials.

Updates

Now and again, you might feel the urge to get up to speed with the latest changes that have been made to the ROMS repository. When that happens, simply go to the directory that was "MyDir" above and type:

   > git pull

Git will remember where you checked out from before and see if a newer revision exists. If so, it will download and apply all the relevant changes.

Using GUI Git Clients

WARNING: It is strongly suggested that you checkout the ROMS source code using the same operating system you wish to compile and run ROMS on. If you download the code on a Windows machine and wish to run it on a non-Windows machine you will need convert the line endings with a utility like dos2unix or recode. Even with these utilities you may still have problems compiling ROMS.

Once we have some experience with some GUI Git clients, we will add to this section.

View the Repository

The easiest way to view a repository is to simply look at it with Trac in a web browser. Enter the URL:

 https://www.myroms.org/projects/git-src/login

and log in with your ROMS credentials. Once logged in, click the Browse Source tab near the top of the page. This is also a good way to see individual files in the repository without actually downloading them.

Managing Your Own Modifications

This assumes that you have a fresh clone of the myroms repository on the master branch. You want to keep master as a pure copy of the source version and keep your own changes in say the arctic branch. Start by creating a branch and switching to it:

> git branch arctic
> git checkout arctic

Now you can make whatever modifications you like (and test them out). To see what changed, you can use git status and git diff. To save your changes, do a:

> git commit -a

Though if you add new files you will have to git add them first.

Getting the Updates

It is easy to fetch and merge the updates. Start by making sure your directory has been cleanly checked in with git status. Then you can update your master branch:

> git checkout master
> git pull

Then bring the changes into your arctic branch:

> git checkout arctic
> git merge master

This will bring in everything that changed since your last git pull, so you might find it easier to keep on top of things by doing this often, not putting it off for years. You can also bring in changes one at a time with git cherry-pick. Again, check the Pro Git book for much more information about all of these operations.

Note that this will save your arctic branch locally, under the .git directory. You can back this up as you would any other important files you have. The gitish way to back it up is to the cloud, say on github. In the best of all possible open source projects, the master would be on github too, and you'd simply fork from it there. If you want to try this out, fork from Mark Hadfield's copy of the myroms source at [1].