How to automatically keep git repositories up to date
In this blog post, we will walk through and build a solution step-by-step that automatically keeps your git repositories up to date. This solution we will be creating will work on all platforms.
Whether you are at work or off the clock, it's likely that you have folders of projects under source control. My personal PC has 89 different repositories - here's a snippet of them!
I've always had this desire to write something to keep all of my git projects up to date. At my job, before working on a new feature, I pull the latest code for a given project before I create the feature branch. I do these same steps every day; I should take the opportunity to write a script to save myself time.
It is 2022 and we expect our solution to run on all operating systems; we will be creating our script with Powershell. (Windows Powershell was Windows-only, with Powershell, we are able to write scripts cross platform, thanks to .NET Core). Before we start writing code, we should define what we are creating, for it is far too easy to build the wrong solution or never finish due to scope creep. Our script is going to satisfy these requirements:
- The script will scan sub-folders that are git repositories (projects).
- The script will pull the latest changes from the main/master branch.
- The script will not undo any pending changes to files within a project.
- The script will leave the project in its current branch after processing is complete.
- The script will be runnable from any directory.
With our requirements defined, lets write this script!
The first step is to make sure Powershell is installed. For Windows users, make sure your execution policy is at least set to RemoteSigned (execution policy has no effect on MacOS/Linux). Having your execution policy set improperly will prevent you from running powershell scripts. You can run the below command to set your execution policy.
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Powershell files end in .ps
, so let's create one in the same directory as your git project folders and open this new Powershell file in your favorite text editor.
For those of you who want a Powershell refresher - click me!
Powershell is heavily based on cmdlets ("command-lets"). Cmdlets can be thought of as functions that take in some input and generally output a .NET object. Cmdlets take the form Verb-Noun
(ie. Get-Command
). Cmdlets can be chained through pipelines ("|"). Cmdlets can have required or optional parameters. Parameters are preceeded by a -
(ie. Get-Command -Name CustomName
).
Powershell has keywords as well as #comments
and allows for $variables
to be defined.
The script will scan sub-folders that are git repositories
Our first logical step for our script is retrieving all of the folders in our current directory. The cmdlet to use in Powershell for this feature is Get-ChildItem. We can pass in a -Directory
parameter which returns directories for a given path.
$Folders = Get-ChildItem -Path (Get-Location) -Directory
We could theoretically have any number of git repositories, so in order to keep track of them we might think to use an array, but arrays in Powershell are immutable. We will use a List instead.
$GitFolders = New-Object System.Collections.Generic.List[string]
Now that we have all of the folders to inspect, let's check each folder if it contains a ".git" sub-folder.
foreach ($Folder in $Folders) {
# Check if we have a .git folder in the directory
$FilePath = $Folder | Select-Object -ExpandProperty FullName
$HasGit = Get-ChildItem -Path $FilePath -Directory -Hidden -Filter .git
# Save off folders that are managed by git
if ($NULL -ne $HasGit) {
$GitFolders.Add($FilePath)
}
}
All of our code so far looks like this.
# Get folders in our current directory
$Folders = Get-ChildItem -Path (Get-Location) -Directory
# Hold folders we can process
$GitFolders = New-Object System.Collections.Generic.List[string]
foreach ($Folder in $Folders) {
# Check if we have a .git folder in the directory
$FilePath = $Folder | Select-Object -ExpandProperty FullName
$HasGit = Get-ChildItem -Path $FilePath -Directory -Hidden -Filter .git
# Save off folders that are managed by git
if ($NULL -ne $HasGit) {
$GitFolders.Add($FilePath)
}
}
Challenge - can you write our script in one line?
It's possible! Here's what I came up with, can you do better?
$GitFolders = @(Get-ChildItem -Path (Get-Location) -Directory | Where-Object {Get-ChildItem -Path ($_ | Select-Object -ExpandProperty FullName) -Directory -Hidden -Filter .git})
The script will pull the latest from the main/master branch
If we make a few assumptions, namely that all of our git repositories are already in our main/master branch and that our git repositories have no uncommitted changes, then pulling the latest code down is quite easy!
# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
Set-Location -LiteralPath $Folder
git pull
}
If everything "just works" for you, you have the state of your git repositories to thank. The code we've written isn't full-proof - in fact, the code will not work if you have any uncommitted changes when git pull
is run. If you have uncommitted changes, you will see this in your console/terminal: error: Your local changes to the following files would be overwritten by merge:
.
In order for us to avoid this error, we need to make sure our repository has no working changes when pulling changes.
The script will not undo any pending changes to files within a project
If we cannot undo pending changes, but the presence of pending changes prevents us from updating our git repository, how can we proceed? We will make use of git stash
to save our changes away, update the git repository, and then re-apply our changes to the git repository.
# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
Set-Location -LiteralPath $Folder
# Save any pending changes
git stash
git pull
# Revert pending changes
git stash pop
}
We should be mostly successful with this code, although you may run into an issue when git stash pop
is called where the stashed files cannot be re-applied due to a merge conflict. Luckily, git saves this stash in your repository so you can manually resolve the merge conflict. We'll have to take this concession for the purposes of our script since there is no way to automatically resolve these merge issues.
There are a number of enhancements we can make here; it is likely that your repositories may have no changes to stash, so we can avoid calling git stash pop
if not necessary. We can also short-circuit our code if our local repository is already up to date compared to our remote repository.
# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
Set-Location -LiteralPath $Folder
# Save any pending changes
$GitStash = git stash
$GitPull = git pull
# Short-circuit if we have the latest changes
if ($GitPull -eq "Already up to date.") {
continue
}
# Revert pending changes
if ($GitStash -ne "No local changes to save") {
git stash pop
}
}
The script will leave the project in its current branch after processing is complete
In addition to accomplishing the goal of leaving the repository in the branch it is currently in, we will also be tackling the task of retrieving the latest changes from our main/master branch. A little bit of git ( git branch
and git checkout
) will give us the functionality we desire.
# Saves our current and main/master branch of the repository
$CurrentGitBranch = git branch --show-current
$GitMainBranch = git branch | Where-Object {$_.EndsWith("main") -or $_.EndsWith("master")}
# Switch to our main branch
git checkout $GitMainBranch
# git stash, pull, then stash pop
# Switch back to our existing branch
git checkout $CurrentGitBranch
I'd like to call out a few details; --show-current
, which gives us a nice string value of the current branch we are in. The second detail I'd like to call out is how we retrieve the main/master branch of our git repository, let's look at this a bit deeper.
When we normally call git branch
, we get output like this.
PS C:\Users\zachary\source\repos\secure-electron-template> git branch
Slapbox/master
broken-menu-translation
license-keys
* master
sandbox-dec-2021
sandbox-enabled
sandbox-v2
v10
git branch
In this example, we need to pull out the value "master". We can make use of the Where-Object
cmdlet; notice that we filter with EndsWith, for if we used StartsWith we'd have to add complexity to deal with spaces and possibly "*". If we run the code listed above, we will see that our code isn't pulling in the results that we want.
PS C:\Users\zachary\source\repos\secure-electron-template> git branch | Where-Object {$_.EndsWith("main") -or $_.EndsWith("master")}
Slapbox/master
* master
git branch
I don't want to pull the "Slapbox/master" branch, so I need to exclude this branch. Let's look at an updated command to exclude this branch.
$GitMainBranch = git branch | Where-Object {-Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master"))}
Putting everything together, as well as some additional short-circuiting and small enhancements, we have this as our result.
$ExecutingDirectory = (Get-Location).Path
# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
# Reset our execution directory
Set-Location -LiteralPath $ExecutingDirectory
Write-Host "Processing $($Folder)"
Set-Location -LiteralPath $Folder
# Saves our current and main/master branch of the repository
$CurrentGitBranch = git branch --show-current
$GitMainBranch = git branch | Where-Object {-Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master"))}
# If we cannot find the main branch, skip over this repository
if ($NULL -eq $GitMainBranch) {
continue
}
# Optionally remove the '*' or ' ' characters to get the true branch name
$GitMainBranch = $GitMainBranch -Replace '[* ]',''
$NeedsToGitCheckout = $CurrentGitBranch -ne $GitMainBranch
if ($NeedsToGitCheckout -eq $TRUE) {
# Switch to our main branch
git checkout $GitMainBranch
}
# Save any pending changes
$GitStash = git stash
$GitPull = git pull
# Short-circuit if we have the latest changes
if ($GitPull -eq "Already up to date.") {
if ($NeedsToGitCheckout -eq $TRUE) {
git checkout $CurrentGitBranch
}
continue
}
# Revert pending changes
if ($GitStash -ne "No local changes to save") {
git stash pop
}
if ($NeedsToGitCheckout -eq $TRUE) {
# Switch back to our existing branch
git checkout $CurrentGitBranch
}
}
# Reset our execution directory
Set-Location -LiteralPath $ExecutingDirectory
The script will be runnable from any directory
In order that we can call our script from any directory, we should make a script module. In order to create a script module, we need to encapsulate our script in a function. A function in Powershell has this basic syntax.
function Get-Version {
// code
}
We will run Get-Verb | Sort-Object Verb
in order to return a list of approved Powershell verbs and choose the one that most-best describes what our function does. We will choose Update-GitRepositories as our function name.
function Update-GitRepositories {
// existing code
}
The next step is to put our function in a directory where module loading is active in. Module loading is active in any of the directories from $ENV:PSModulePath
. We can see all available directories easily by running $ENV:PSModulePath -split ";"
.
PS C:\Users\zachary> $ENV:PSModulePath -split ";"
C:\Users\zachary\Documents\PowerShell\Modules
C:\Program Files\PowerShell\Modules
c:\program files\powershell\7\Modules
C:\Users\zachary\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\PowerShell
C:\Program Files\WindowsPowerShell\Modules
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\PowerShell\Modules\
I recommend we use C:\Program Files\WindowsPowerShell\Modules as the install location, as this will be available to all users for a Windows machine. For script module auto-loading to work, you will need to name the parent folder, and the script module file (.psm1 extension) with the same name. You will need to create this file with administrator permissions; below you can see the file on my computer.
Now, you can run Update-GitRepositories at any location on your machine!
Install-Module -Name UpdateGitRepositories
.The entire UpdateGitRepositories.psm1 script contains:
function Update-GitRepositories {
<#
.SYNOPSIS
Updates all .git repositories at a given path by pulling the latest changes
from the main/master branch.
.DESCRIPTION
Update-GitRepositories searches all directories at the given path (the
default path is the path where this command is executed) if the directory
contains a git repository (.git folder). If a .git folder is present in
the directory, this command will stash any git changes, switch to the main/
master branch, pull the latest changes from git, and revert back to the
original git branch the repository was in before moving on to the next
repository.
.PARAMETER Path
The path at which to execute this function at. By default, this value is
the current directory.
.EXAMPLE
Update-GitRepositories
.EXAMPLE
Update-GitRepositories -Path "C:\Users\Me\repos"
.INPUTS
String
#>
[CmdletBinding(SupportsShouldProcess)]
param(
[ValidateNotNullOrEmpty()]
[string]$Path = (Get-Location)
)
BEGIN {}
PROCESS {
# Pulls all folders at the given path that contain a .git folder
$GitFolders = @(Get-ChildItem -Path (Get-Location) -Directory | Where-Object {Get-ChildItem -Path ($_ | Select-Object -ExpandProperty FullName) -Directory -Hidden -Filter .git})
$ExecutingDirectory = (Get-Location).Path
# Pulls the latest code from git
foreach ($Folder in $GitFolders) {
# Reset our execution directory
Set-Location -LiteralPath $ExecutingDirectory
Write-Verbose -Message "Processing $($Folder)"
Set-Location -LiteralPath $Folder
# Saves our current and main/master branch of the repository
$CurrentGitBranch = git branch --show-current
$GitMainBranch = git branch | Where-Object { -Not $_.Contains("/") -and ($_.EndsWith("main") -or $_.EndsWith("master")) }
# If we cannot find the main branch, skip over this repository
if ($NULL -eq $GitMainBranch) {
Write-Verbose -Message "Failed to find the main branch, skipping this git repository"
continue
}
# Optionally remove the '*' or ' ' characters to get the true branch name
$GitMainBranch = $GitMainBranch -Replace '[* ]', ''
$NeedsToGitCheckout = $CurrentGitBranch -ne $GitMainBranch
if ($NeedsToGitCheckout -eq $TRUE) {
# Switch to our main branch
git checkout $GitMainBranch
}
# Save any pending changes
$GitStash = git stash
$GitPull = git pull
# Short-circuit if we have the latest changes
if ($GitPull -eq "Already up to date.") {
if ($NeedsToGitCheckout -eq $TRUE) {
git checkout $CurrentGitBranch
}
continue
}
# Revert pending changes
if ($GitStash -ne "No local changes to save") {
git stash pop
}
if ($NeedsToGitCheckout -eq $TRUE) {
# Switch back to our existing branch
git checkout $CurrentGitBranch
}
}
# Reset our execution directory
Set-Location -LiteralPath $ExecutingDirectory
}
END {}
}
Thank you for reading this blog post, I hope you learned a bit about Powershell and git in the process! Thank you for your continued support - if you have comments/questions about this blog post, or have suggestions about future posts you'd like to see me write, please reach out to me here.