How to Remove a Large File from Commit History in Git?
If you develop a game in Unity and save assets like images or audio files on Github without using Git FS, your repo’s size might become bigger and bigger when you make any changes to Github. Given the fact that Git History will record all the changes of codes, images, audio files, or even binary files.
This could make the building time of games become longer and delay your work process because it’s will take too much time in CI/CD Pipeline while downloading assets from Github and even building the games.
How to clean up those large files in Git Commit History?
In this post, we will provide a guide and practical example to resolve this problem.
Step 1: Check the Size of Git History
du -hs .git/objects
Step 2: Create a script to find larges files
Create git_find_big.sh script
#set -x# Shows you the largest objects in your repo's pack file.
# Written for osx.
# @see http://stubbisms.wordpress.com/2009/07/10/git-script-to-show-largest-pack-objects-and-trim-your-waist-line/
# @author Antony Stubbs# set the internal field spereator to line break, so that we can iterate easily over the verify-pack output
IFS=$'\n';# list all objects including their size, sort by size, take top 10
objects=`git verify-pack -v .git/objects/pack/pack-*.idx | grep -v chain | sort -k3nr | head`echo "All sizes are in kB's. The pack column is the size of the object, compressed, inside the pack file."output="size,pack,SHA,location"
for y in $objects
# extract the size in bytes
size=$((`echo $y | cut -f 5 -d ' '`/1024))
# extract the compressed size in bytes
compressedSize=$((`echo $y | cut -f 6 -d ' '`/1024))
# extract the SHA
sha=`echo $y | cut -f 1 -d ' '`
# find the objects location in the repository tree
other=`git rev-list --all --objects | grep $sha`
#lineBreak=`echo -e "\n"`
doneecho -e $output | column -t -s ', '
Step 3: Find Top 10 Large files
Step 4. Install git-filter-repo
pip3 install git-filter-repo
Step 5: Remove large files from the .git history
You can get the Top 10 large file names from Step 3.
Then run below to delete them in .git history.
git filter-repo --force --path <large file path> --invert-paths
This command will rewrite your .git history and delete it.
Step 6: Clean Unused Fils
git reflog expire --expire=now --allgit gc --prune=now
Step 7. Add Git Remote URL and Push Back the Result
# Add Your Github URL
git remote add origin <Your Github URL># Push the result to all the branches
git push --all --force# Push the result to origin repo
git push -u origin --all
git push -u origin --tags -f
This tutorial provides a solution to optimize your repo’s size on Github. Remember you should use Git FS to save a large size of contents.
If not, don’t forget to regularly check whether there are some large files in Git History or not.
Making your game codes as small as possible not only can boost work efficiency, but also provide a high-quality game on time and on budget. Because it’s easy to maintain codes and add new features with fewer efforts.