How do I properly git stash/pop in pre-commit hooks to get a clean working tree for tests?

Question

I'm trying to do a pre-commit hook with a bare run of unit tests and I want to make sure my working directory is clean. Compiling takes a long time so I want to take advantage of reusing compiled binaries whenever possible. My script follows examples I've seen online:

# Stash changes
git stash -q --keep-index

# Run tests
...

# Restore changes
git stash pop -q

This causes problems though. Here's the repro:

1. Add // Step 1 to a.java
2. git add .
3. Add // Step 2 to a.java
4. git commit
   1. git stash -q --keep-index # Stash changes
   2. Run tests
   3. git stash pop -q # Restore changes

At this point I hit the problem. The git stash pop -q apparently has a conflict and in a.java I have

// Step 1
<<<<<<< Updated upstream
=======
// Step 2
>>>>>>> Stashed changes

Is there a way to get this to pop cleanly?

Answer 1

There is—but let's get there in a slightly roundabout fashion. (Also, see warning below: there's a bug in the stash code which I thought was very rare, but apparently more people are running into. New warning, added in Dec 2021: git stash has been rewritten in C and has a whole new crop of bugs. I used to suggest mildly that git stash be avoided; now I urge everyone to avoid it if at all possible.)

git stash push (the default action for git stash; note that this was spelled git stash save in 2015, when I wrote the first version of this answer) makes a commit that has at least two parents (see this answer to a more basic question about stashes). The stash commit is the work-tree state, and the second parent commit stash^2 is the index-state at the time of the stash.

After the stash is made (and assuming no -p option), the script—git stash is a shell script—uses git reset --hard to clean out the changes.

When you use --keep-index, the script does not change the saved stash in any way. Instead, after the git reset --hard operation, the script uses an extra git read-tree --reset -u to wipe out the work-directory changes, replacing them with the "index" part of the stash.

In other words, it's almost like doing:

git reset --hard stash^2

except that git reset would also move the branch—not at all what you want, hence the read-tree method instead.

This is where your code comes back in. You now # Run tests on the contents of the index commit.

Assuming all goes well, I presume you want to get the index back into the state it had when you did the git stash, and get the work-tree back into its state as well.

With git stash apply or git stash pop, the way to do that is to use --index (not --keep-index, that's just for stash-creation time, to tell the stash script "whack on the work directory").

Just using --index will still fail though, because --keep-index re-applied the index changes to the work directory. So you must first get rid of all of those changes ... and to do that, you simply need to (re)run git reset --hard, just like the stash script itself did earlier. (Probably you also want -q.)

So, this gives as the last # Restore changes step:

# Restore changes
git reset --hard -q
git stash pop --index -q

(I'd separate them out as:

git stash apply --index -q && git stash drop -q

myself, just for clarity, but the pop will do the same thing).

---

As noted in a comment below, the final git stash pop --index -q complains a bit (or, worse, restores an old stash) if the initial git stash push step finds no changes to save. You should therefore protect the "restore" step with a test to see if the "save" step actually stashed anything.

The initial git stash --keep-index -q simply exits quietly (with status 0) when it does nothing, so we need to handle two cases: no stash exists either before or after the save; and, some stash existed before the save, and the save did nothing so the old existing stash is still the top of the stash stack.

I think the simplest method is to use git rev-parse to find out what refs/stash names, if anything. So we should have the script read something more like this:

#! /bin/sh
# script to run tests on what is to be committed

# First, stash index and work dir, keeping only the
# to-be-committed changes in the working directory.
old_stash=$(git rev-parse -q --verify refs/stash)
git stash push -q --keep-index
new_stash=$(git rev-parse -q --verify refs/stash)

# If there were no changes (e.g., `--amend` or `--allow-empty`)
# then nothing was stashed, and we should skip everything,
# including the tests themselves.  (Presumably the tests passed
# on the previous commit, so there is no need to re-run them.)
if [ "$old_stash" = "$new_stash" ]; then
    echo "pre-commit script: no changes to test"
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

# Run tests
status=...

# Restore changes
git reset --hard -q && git stash apply --index -q && git stash drop -q

# Exit with status from test-run: nonzero prevents commit
exit $status

---

warning: small bug in git stash

(Note: I believe this bug was fixed in the conversion to C. Instead, there are numerous other bugs now. They will no doubt eventually be fixed, but depending on which version of Git you are using, git stash may have various bugs of varying seriousness.)

There's a minor bug in the way git stash writes its "stash bag". The index-state stash is correct, but suppose you do something like this:

cp foo.txt /tmp/save                    # save original version
sed -i '' -e '1s/^/inserted/' foo.txt   # insert a change
git add foo.txt                         # record it in the index
cp /tmp/save foo.txt                    # then undo the change

When you run git stash push after this, the index-commit (refs/stash^2) has the inserted text in foo.txt. The work-tree commit (refs/stash) should have the version of foo.txt without the extra inserted stuff. If you look at it, though, you'll see it has the wrong (index-modified) version.

The script above uses --keep-index to get the working tree set up as the index was, which is all perfectly fine and does the right thing for running the tests. After running the tests, it uses git reset --hard to go back to the HEAD commit state (which is still perfectly fine) ... and then it uses git stash apply --index to restore the index (which works) and the work directory.

This is where it goes wrong. The index is (correctly) restored from the stash index commit, but the work-directory is restored from the stash work-directory commit. This work-directory commit has the version of foo.txt that's in the index. In other words, that last step—cp /tmp/save foo.txt—that undid the change, has been un-un-done!

(The bug in the stash script occurs because the script compares the work-tree state against the HEAD commit in order to compute the set of files to record in the special temporary index before making the special work-dir commit part of the stash-bag. Since foo.txt is unchanged with respect to HEAD, it fails to git add it to the special temporary index. The special work-tree commit is then made with the index-commit's version of foo.txt. The fix is very simple but no one has put it into official git [yet?].

Not that I want to encourage people to modify their versions of git, but here's the fix.)

Answer 2

It seems that most answers here, at this time, are at least 5 years old. git stash has been re-written in C, there were some new bugs, and I don't really know how reliable it is.

I've used this a few times and it seems to be working - I wrote it from scratch, just using the docs as a guide. Has not been battle tested.

# We stash and un-stash changes ourselves.
#  - If any pre-commit/lint-staged checks fail, any auto-fixes will be lost.

# Create stash
#   index is the "staging area", so --keep-index means that anything you have already staged will be un-touched.
# NOTE: we always create a stash - possibly even a totally empty one.
git stash --keep-index --include-untracked --message="pre-commit auto-stash"
uncoloredStashedStat=$(git stash show --include-untracked stash@{0})
[[ $uncoloredStashedStat ]] && {
  echo "Stashed:"
  git diff --stat --staged stash@{0}
}

lintStagedStatus='failed'

yarn lint-staged --no-stash --concurrent $pre_commit_concurrency --shell "/bin/bash"  && {
  lintStagedStatus='passed'
}

outputSuppressed=$(git add --intent-to-add "**/*.snap")
diff=$(git diff)
[[ $diff ]] && {
  echo "staging modifications from pre-commit scripts:"
  git diff
  git add .
}

# Pop stash
#   We always create a stash - so we will always pop it.
#   Popped stash should generally not cause merge conflicts,
#   if your editor is formatting+autofixing code on save.
[[ $uncoloredStashedStat ]] && echo "restoring stash..."
git stash pop

if test "$lintStagedStatus" != 'passed'; then
  exit 1;
fi

Answer 3

Thanks to the answer of @torek I was able to put together a script that also deals with untracked files. (Note: I don't want to use git stash -u due to a unwanted behaviour of git stash -u)

The mentioned git stash bug remains unchanged and I am not yet sure, whether this method might run into issues when a .gitignore is among the changed files. (same applies to @torek's answer)

#! /bin/sh
# script to run tests on what is to be committed
# Based on http://stackoverflow.com/a/20480591/1606867

# Remember old stash
old_stash=$(git rev-parse -q --verify refs/stash)

# First, stash index and work dir, keeping only the
# to-be-committed changes in the working directory.
git stash save -q --keep-index
changes_stash=$(git rev-parse -q --verify refs/stash)
if [ "$old_stash" = "$changes_stash" ]
then
    echo "pre-commit script: no changes to test"
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

#now let's stash the staged changes
git stash save -q
staged_stash=$(git rev-parse -q --verify refs/stash)
if [ "$changes_stash" = "$staged_stash" ]
then
    echo "pre-commit script: no staged changes to test"
    # re-apply changes_stash
    git reset --hard -q && git stash pop --index -q
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

# Add all untracked files and stash those as well
# We don't want to use -u due to
# http://blog.icefusion.co.uk/git-stash-can-delete-ignored-files-git-stash-u/
git add .
git stash save -q
untracked_stash=$(git rev-parse -q --verify refs/stash)

#Re-apply the staged changes
if [ "$staged_stash" = "$untracked_stash" ]
then
    git reset --hard -q && git stash apply --index -q stash@{0}
else
    git reset --hard -q && git stash apply --index -q stash@{1}
fi

# Run tests
status=...

# Restore changes

# Restore untracked if any
if [ "$staged_stash" != "$untracked_stash" ]
then
    git reset --hard -q && git stash pop --index -q
    git reset HEAD -- . -q
fi

# Restore staged changes
git reset --hard -q && git stash pop --index -q

# Restore unstaged changes
git reset --hard -q && git stash pop --index -q

# Exit with status from test-run: nonzero prevents commit
exit $status

Answer 4

based off torek's answer I came up with a method to ensure the right behavior of stashing changes without using git rev-parse, instead I used git stash create and git stash store (although using git stash store is not strictly necessary) Note due to the environment I am working in my script is written in php instead of bash

#!/php/php
<?php
$files = array();
$stash = array();
exec('git stash create -q', $stash);
$do_stash = !(empty($stash) || empty($stash[0]));
if($do_stash) {
    exec('git stash store '.$stash[0]); //store the stash (does not tree state like git stash save does)
    exec('git stash show -p | git apply --reverse'); //remove working tree changes
    exec('git diff --cached | git apply'); //re-add indexed (ready to commit) changes to working tree
}
//exec('git stash save -q --keep-index', $stash);
exec('git diff-index --cached --name-only HEAD', $files );

// dont redirect stderr to stdin, we will get the errors twice, redirect it to dev/null
if ( PHP_OS == 'WINNT' )
  $redirect = ' 2> NUL';
else
  $redirect = ' 2> /dev/null';
$exitcode = 0;

foreach( $files as $file ) {

  if ( !preg_match('/\.php$/i', $file ) )
    continue;

  exec('php -l ' . escapeshellarg( $file ) . $redirect, $output, $return );
  if ( !$return ) // php -l gives a 0 error code if everything went well
    continue;

  $exitcode = 1; // abort the commit
  array_shift( $output ); // first line is always blank
  array_pop( $output ); // the last line is always "Errors parsing httpdocs/test.php"

  echo implode("\n", $output ), "\n"; // an extra newline to make it look good
}
if($do_stash) {
    exec('git reset --hard -q');
    exec('git stash apply --index -q');
    exec('git stash drop -q');
}
exit( $exitcode );

?>

php script adapted from here http://blog.dotsamazing.com/2010/04/ask-git-to-check-if-your-codes-are-error-free/