Getting Jupyter and git to work together

This is an abbreviated version of this blog post

Locate your global .gitconfig file. This will be in your home directory. Executing "git config --list --show-origin" should help you locate it as this command lists all your git configuration options and where they are set. You may need to set your file browser to show hidden files. If you edit the global .gitconfig file, then the results of git config --list --show-origin should change accordingly.

Download jq and make a note of where you saved it. In my case I saved it as "c:\tools\jq-win64.exe".

Create a file called .gitattributes_global in the same directory as your .gitconfig file. It should have the following entry.

*.ipynb filter=nbstrip_full

Edit your global .gitconfig file by adding the following entries. Note that you will need to make appropriate edits to the precise location of jq and your gitattributes_global file (which is slightly fiddly on Windows, see below).

[core]
attributesfile = LOCATION_OF_GITATTRIBUTES_GLOBAL

[filter "nbstrip_full"]
clean = "LOCATION_OF_JQ --indent 1 \
        '(.cells[] | select(has(\"outputs\")) | .outputs) = []  \
        | (.cells[] | select(has(\"execution_count\")) | .execution_count) = null  \
        | .metadata = {\"language_info\": {\"name\": \"python\", \"pygments_lexer\": \"ipython3\"}} \
        | .cells[].metadata = {} \
        '"
smudge = cat		
required = true

Platform specifics

Unix Instructions

On a Unix system entering the location of the file .gitattributes_global is straightforward, just enter the full path on your file system to the file. Similarly one can specify the location of JQ.

Windows Instructions

On a normal Windows system you will need to escape the backslash characters. The following worked for me.

[user]
	email = johnarmstrong5@googlemail.com
	name = John Armstrong

[core]
attributesfile = C:\\Users\\johna\\.gitattributes_global

[filter "nbstrip_full"]
clean = "C:\\\\tools\\\\jq-win64 --indent 1 \
        '(.cells[] | select(has(\"outputs\")) | .outputs) = []  \
        | (.cells[] | select(has(\"execution_count\")) | .execution_count) = null  \
        | .metadata = {\"language_info\": {\"name\": \"python\", \"pygments_lexer\": \"ipython3\"}} \
        | .cells[].metadata = {} \
        '"
smudge = cat		
required = true

Cygwin Instructions

One further alternative is to use a Cygwin system on Windows to run git so for me the location c:\tools\jq-win64.ext would need to be entered as /cygdrive/c/tools/jq-win64.exe.

Testing

It isn't easy to tell if this is set up correctly, so we introduce a deliberate error that should make it clear if there is a problem.

Edit an ipynb file. I happened to choose one called python/optimalconsumption.ipynb for my testing. Then use git add as shown below to add the file to your project. This should not produce an error.

git add python/optimalconsumption.ipynb

git reset HEAD python/optimalconsumption.ipynb

Change the .gitconfig file so the location of the jq tool is clearly incorrect. git add should now produce an error when you execute

git add python/optimalconsumption.ipynb

Correct the .gitconfig file back so the location of jq is correct. A git add command such as the following should now work.

git add python/optimalconsumption.ipynb

git reset HEAD python/optimalconsumption.ipynb
git checkout python/optimalconsumption.ipynb