How to automatically keep git repositories up to date

In this blog post, we will walk through and build a solution step-by-step that automatically keeps your git repositories up to date. This solution we will be creating will work on all platforms.

Whether you are at work or off the clock, it's likely that you have folders of projects under source control. My personal PC has 89 different repositories - here's a snippet of them!

I've always had this desire to write something to keep all of my git projects up to date. At my job, before working on a new feature, I pull the latest code for a given project before I create the feature branch. I do these same steps every day; I should take the opportunity to write a script to save myself time.

It is 2022 and we expect our solution to run on all operating systems; we will be creating our script with Powershell. (Windows Powershell was Windows-only, with Powershell, we are able to write scripts cross platform, thanks to .NET Core). Before we start writing code, we should define what we are creating, for it is far too easy to build the wrong solution or never finish due to scope creep. Our script is going to satisfy these requirements:

1. The script will scan sub-folders that are git repositories (projects).
2. The script will pull the latest changes from the main/master branch.
3. The script will not undo any pending changes to files within a project.
4. The script will leave the project in its current branch after processing is complete.
5. The script will be runnable from any directory.

With our requirements defined, lets write this script!

The first step is to make sure Powershell is installed. For Windows users, make sure your execution policy is at least set to RemoteSigned (execution policy has no effect on MacOS/Linux). Having your execution policy set improperly will prevent you from running powershell scripts. You can run the below command to set your execution policy.

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Powershell files end in .ps, so let's create one in the same directory as your git project folders and open this new Powershell file in your favorite text editor.

The script will scan sub-folders that are git repositories

Our first logical step for our script is retrieving all of the folders in our current directory. The cmdlet to use in Powershell for this feature is Get-ChildItem. We can pass in a -Directory parameter which returns directories for a given path.

$Folders = Get-ChildItem -Path (Get-Location) -Directory

We could theoretically have any number of git repositories, so in order to keep track of them we might think to use an array, but arrays in Powershell are immutable. We will use a List instead.

$GitFolders = New-Object System.Collections.Generic.List[string]

Now that we have all of the folders to inspect, let's check each folder if it contains a ".git" sub-folder.

foreach ($Folder in $Folders) {

    # Check if we have a .git folder in the directory
    $FilePath = $Folder | Select-Object -ExpandProperty FullName    
    $HasGit = Get-ChildItem -Path $FilePath -Directory -Hidden -Filter .git

    # Save off folders that are managed by git
    if ($NULL -ne $HasGit) {
        $GitFolders.Add($FilePath)        
    }
}

All of our code so far looks like this.

# Get folders in our current directory
$Folders = Get-ChildItem -Path (Get-Location) -Directory

# Hold folders we can process
$GitFolders = New-Object System.Collections.Generic.List[string]
foreach ($Folder in $Folders) {

    # Check if we have a .git folder in the directory
    $FilePath = $Folder | Select-Object -ExpandProperty FullName    
    $HasGit = Get-ChildItem -Path $FilePath -Directory -Hidden -Filter .git

    # Save off folders that are managed by git
    if ($NULL -ne $HasGit) {
        $GitFolders.Add($FilePath)        
    }
}

The script will pull the latest from the main/master branch

If we make a few assumptions, namely that all of our git repositories are already in our main/master branch and that our git repositories have no uncommitted changes, then pulling the latest code down is quite easy!

# Pulls the latest code from git
foreach ($Folder in $GitFolders) {

    Set-Location -LiteralPath $Folder
    git pull
}

If everything "just works" for you, you have the state of your git repositories to thank. The code we've written isn't full-proof - in fact, the code will not work if you have any uncommitted changes when git pull is run. If you have uncommitted changes, you will see this in your console/terminal: error: Your local changes to the following files would be overwritten by merge:.

In order for us to avoid this error, we need to make sure our repository has no working changes when pulling changes.

The script will not undo any pending changes to files within a project

If we cannot undo pending changes, but the presence of pending changes prevents us from updating our git repository, how can we proceed? We will make use of git stash to save our changes away, update the git repository, and then re-apply our changes to the git repository.

# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
    Set-Location -LiteralPath $Folder    

    # Save any pending changes
    git stash
    
    git pull

    # Revert pending changes
    git stash pop
}

We should be mostly successful with this code, although you may run into an issue when git stash pop is called where the stashed files cannot be re-applied due to a merge conflict. Luckily, git saves this stash in your repository so you can manually resolve the merge conflict. We'll have to take this concession for the purposes of our script since there is no way to automatically resolve these merge issues.

There are a number of enhancements we can make here; it is likely that your repositories may have no changes to stash, so we can avoid calling git stash pop if not necessary. We can also short-circuit our code if our local repository is already up to date compared to our remote repository.

# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
    Set-Location -LiteralPath $Folder

    # Save any pending changes
    $GitStash = git stash
    
    $GitPull = git pull

    # Short-circuit if we have the latest changes
    if ($GitPull -eq "Already up to date.") {
        continue
    }

    # Revert pending changes
    if ($GitStash -ne "No local changes to save") {
        git stash pop
    }    
}

The script will leave the project in its current branch after processing is complete

In addition to accomplishing the goal of leaving the repository in the branch it is currently in, we will also be tackling the task of retrieving the latest changes from our main/master branch. A little bit of git ( git branch and git checkout) will give us the functionality we desire.

# Saves our current and main/master branch of the repository
$CurrentGitBranch = git branch --show-current
$GitMainBranch = git branch | Where-Object {$_.EndsWith("main") -or $_.EndsWith("master")}
           
# Switch to our main branch
git checkout $GitMainBranch

# git stash, pull, then stash pop   
    
# Switch back to our existing branch
git checkout $CurrentGitBranch

I'd like to call out a few details; --show-current, which gives us a nice string value of the current branch we are in. The second detail I'd like to call out is how we retrieve the main/master branch of our git repository, let's look at this a bit deeper.

When we normally call git branch, we get output like this.

PS C:\Users\zachary\source\repos\secure-electron-template> git branch
  Slapbox/master
  broken-menu-translation
  license-keys
* master
  sandbox-dec-2021
  sandbox-enabled
  sandbox-v2
  v10

In this example, we need to pull out the value "master". We can make use of the Where-Object cmdlet; notice that we filter with EndsWith, for if we used StartsWith we'd have to add complexity to deal with spaces and possibly "*". If we run the code listed above, we will see that our code isn't pulling in the results that we want.

PS C:\Users\zachary\source\repos\secure-electron-template> git branch | Where-Object {$_.EndsWith("main") -or $_.EndsWith("master")}
  Slapbox/master
* master

I don't want to pull the "Slapbox/master" branch, so I need to exclude this branch. Let's look at an updated command to exclude this branch.

$GitMainBranch = git branch | Where-Object {-Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master"))}

Putting everything together, as well as some additional short-circuiting and small enhancements, we have this as our result.

$ExecutingDirectory = (Get-Location).Path

# Pulls the latest code from git
foreach ($Folder in $GitFolders) {

    # Reset our execution directory
    Set-Location -LiteralPath $ExecutingDirectory
    
    Write-Host "Processing $($Folder)"
    Set-Location -LiteralPath $Folder

    # Saves our current and main/master branch of the repository
    $CurrentGitBranch = git branch --show-current
    $GitMainBranch = git branch | Where-Object {-Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master"))}    
    
    # If we cannot find the main branch, skip over this repository
    if ($NULL -eq $GitMainBranch) {
        continue
    }
    
    # Optionally remove the '*' or ' ' characters to get the true branch name
    $GitMainBranch = $GitMainBranch -Replace '[* ]',''
    $NeedsToGitCheckout = $CurrentGitBranch -ne $GitMainBranch
    
    if ($NeedsToGitCheckout -eq $TRUE) {
        
        # Switch to our main branch
        git checkout $GitMainBranch
    }

    # Save any pending changes
    $GitStash = git stash
    
    $GitPull = git pull

    # Short-circuit if we have the latest changes
    if ($GitPull -eq "Already up to date.") {
        if ($NeedsToGitCheckout -eq $TRUE) {
            git checkout $CurrentGitBranch
        }

        continue
    }

    # Revert pending changes
    if ($GitStash -ne "No local changes to save") {
        git stash pop
    }

    if ($NeedsToGitCheckout -eq $TRUE) {
        
        # Switch back to our existing branch
        git checkout $CurrentGitBranch
    }
}

# Reset our execution directory
Set-Location -LiteralPath $ExecutingDirectory

The script will be runnable from any directory

In order that we can call our script from any directory, we should make a script module. In order to create a script module, we need to encapsulate our script in a function. A function in Powershell has this basic syntax.

function Get-Version {
    // code
}

We will run Get-Verb | Sort-Object Verb in order to return a list of approved Powershell verbs and choose the one that most-best describes what our function does. We will choose Update-GitRepositories as our function name.

function Update-GitRepositories {
    // existing code
}

The next step is to put our function in a directory where module loading is active in. Module loading is active in any of the directories from $ENV:PSModulePath. We can see all available directories easily by running $ENV:PSModulePath -split ";".

PS C:\Users\zachary> $ENV:PSModulePath -split ";"
C:\Users\zachary\Documents\PowerShell\Modules
C:\Program Files\PowerShell\Modules
c:\program files\powershell\7\Modules

C:\Users\zachary\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\PowerShell
C:\Program Files\WindowsPowerShell\Modules
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\PowerShell\Modules\

I recommend we use C:\Program Files\WindowsPowerShell\Modules as the install location, as this will be available to all users for a Windows machine. For script module auto-loading to work, you will need to name the parent folder, and the script module file (.psm1 extension) with the same name. You will need to create this file with administrator permissions; below you can see the file on my computer.

Now, you can run Update-GitRepositories at any location on your machine!

The entire UpdateGitRepositories.psm1 script contains:

function Update-GitRepositories {

    <#
    .SYNOPSIS
        Updates all .git repositories at a given path by pulling the latest changes
        from the main/master branch.
    
    .DESCRIPTION
        Update-GitRepositories searches all directories at the given path (the
        default path is the path where this command is executed) if the directory
        contains a git repository (.git folder). If a .git folder is present in
        the directory, this command will stash any git changes, switch to the main/
        master branch, pull the latest changes from git, and revert back to the
        original git branch the repository was in before moving on to the next
        repository.    

    .PARAMETER Path
        The path at which to execute this function at. By default, this value is
        the current directory.

    .EXAMPLE
        Update-GitRepositories

    .EXAMPLE
        Update-GitRepositories -Path "C:\Users\Me\repos"

    .INPUTS
        String
    
    #>

    [CmdletBinding(SupportsShouldProcess)]
    param(
        [ValidateNotNullOrEmpty()]
        [string]$Path = (Get-Location)
    )

    BEGIN {}

    PROCESS {

        # Pulls all folders at the given path that contain a .git folder
        $GitFolders = @(Get-ChildItem -Path (Get-Location) -Directory | Where-Object {Get-ChildItem -Path ($_ | Select-Object -ExpandProperty FullName) -Directory -Hidden -Filter .git})

        $ExecutingDirectory = (Get-Location).Path

        # Pulls the latest code from git
        foreach ($Folder in $GitFolders) {
        
            # Reset our execution directory
            Set-Location -LiteralPath $ExecutingDirectory

            Write-Verbose -Message "Processing $($Folder)"
            Set-Location -LiteralPath $Folder

            # Saves our current and main/master branch of the repository
            $CurrentGitBranch = git branch --show-current
            $GitMainBranch = git branch | Where-Object { -Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master")) }    

            # If we cannot find the main branch, skip over this repository
            if ($NULL -eq $GitMainBranch) {

                Write-Verbose -Message "Failed to find the main branch, skipping this git repository"
                continue
            }

            # Optionally remove the '*' or ' ' characters to get the true branch name
            $GitMainBranch = $GitMainBranch -Replace '[* ]', ''
            $NeedsToGitCheckout = $CurrentGitBranch -ne $GitMainBranch

            if ($NeedsToGitCheckout -eq $TRUE) {

                # Switch to our main branch
                git checkout $GitMainBranch
            }

            # Save any pending changes
            $GitStash = git stash

            $GitPull = git pull

            # Short-circuit if we have the latest changes
            if ($GitPull -eq "Already up to date.") {        
                if ($NeedsToGitCheckout -eq $TRUE) {
                    git checkout $CurrentGitBranch
                }

                continue
            }

            # Revert pending changes
            if ($GitStash -ne "No local changes to save") {
                git stash pop
            }

            if ($NeedsToGitCheckout -eq $TRUE) {
    
                # Switch back to our existing branch
                git checkout $CurrentGitBranch
            }
        }

        # Reset our execution directory
        Set-Location -LiteralPath $ExecutingDirectory
    }

    END {}    
}

Thank you for reading this blog post, I hope you learned a bit about Powershell and git in the process! Thank you for your continued support - if you have comments/questions about this blog post, or have suggestions about future posts you'd like to see me write, please reach out to me here.