Powershell: git source not updated with error message fatal: remote origin already exists when using ErrorActionPreference: STOP

Summary

This is a reopen of #4110 (closed) which was fixed, but I found another way to reproduce it when on a Windows/Powershell runner.

Steps to reproduce

Set the ErrorActionPreference variable to STOP in your .gitlab-ci.yml and run any powershell job.

.gitlab-ci.yml

variables:
  ErrorActionPreference: STOP
stages:
  - build
build_test_zip:
  tags:
    - msbuild
  stage: build

Actual behavior

Running with gitlab-runner 11.10.1 (1f513601)
  on SB401k-MSBuild-Runner FLKALD4H
Using Shell executor...
Running on EC2AMAZ-N0FBDNM...
Reinitialized existing Git repository in C:/GitLab-Runner/builds/FLKALD4H/0/redacted/.git/
Fetching changes...
git.exe : fatal: remote origin already exists.
At C:\Windows\TEMP\build_script637788777\script.ps1:221 char:1
+ & "git" "remote" "add" "origin" "https://gitlab-ci-token:Q4bym4wsAUGA ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (fatal: remote origin already exists.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
ERROR: Job failed: exit status 1

Expected behavior

This should not generate an error.

Used GitLab Runner version

11.10.1 (1f513601) Windows x86