Eric

Azure DevOps Releases: Auto Create New Release After Pipeline Build

I hit a snag with my planned post for this weeks and decided to hope back over to Azure DevOps and show how to create a Release and have that Release be triggered anytime a build Pipeline complete successfully. This post is going to be using the same Azure DevOps Project as all the previous posts in this series which are linked below.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging
Azure DevOps Pipelines: Depends On with Conditionals in YAML
Azure DevOps Pipelines: PowerShell Task

Creating a Release

In Azure DevOps to create a release go to Pipelines and then Releases. Since this is our project’s first Release we have a New pipeline button to click to start the creation process.

The New pipeline button will start the creation process by showing a Select a template dialog. Since our release isn’t really going to do anything yet we are going to click the Empty job option.

Next, we want to add the artifacts from our build Pipeline to this Release. Click Add an artifact to start the process.

The Add an artifact dialog will show which allows us a lot of options on the source of the artifact. For this setup, we are going to use a Source type of Build since our artifacts are the result of an Azure DevOps Build. The next option we need to select is Source which is where we select which Azure DevOps Build Pipeline we want to use artifacts from. In this sample case, we only have a single option. Once the Source is selected a couple more options will show up, but we are taking the default values for those and just clicking the Add button.

Now that we have the basic release setup click the Save button. There will be a prompt for a folder. Either enter a folder name or leave blank to keep the release in the root and then click OK to complete the save.

Auto Create After Pipeline Build

Now that we have our basic Release Pipeline we are going to set up a continuous deployment trigger on the artifact so that any time a new build is completed. In the Artifacts area click the Lighting Bolt on the artifact the trigger should be on, we only have one option in our sample.

A dialog will show different trigger options. We are going to Enable the Continuous deployment trigger. Notice that when the trigger is enabled there is an option to only trigger for specific branches. I have used this option pretty often to only trigger a release when a build is done on master. When done click the X to close the dialog and then Save the release. I also renamed the release to Web App 1 & 2 to make it clearer what the release is doing.

Now pop over to the Pipelines and run a build. After the build is complete go back to the Release and you will see that a new release was triggered, and deployed in this case based on how the rest of the Release was set up.

Wrapping Up

In this post, we created our first Release, which doesn’t actually do anything, and configured it to deploy automatically on the build of our application. I use a similar setup to automatically deploy to a QA environment. This setup can open up a ton of scenarios.

GitHub: Use Actions to Run Multiple Jobs

In the post, we are going to take our sample Workflow that builds two ASP.NET Core web applications and split it so each web application is built individually. This post is using the repo and Workflow built in the following posts if you need to catch up.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application
GitHub: Use Actions to Publish Artifacts

Starting Point and the Plan

Our Workflow currently contains a single job that just happens to build two ASP.NET Core web application based on the fact that the .NET CLI picks up and builds both applications. The following is the YAML for our current Workflow.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish 
    
    - name: Upload WebApp1 Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
    - name: Upload WebApp2 Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp2
        path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

This post is going to take this Workflow and split the build and publish of the two web applications into two jobs. By splitting the Workflow into multiple jobs we open the possibility that the jobs can run in parallel. One reason to do this would be to speed up the total Workflow run time if you have parts of your build that are independent. Another example of why you would need multiple jobs is if the different jobs need different needed to run on different operating systems such as one needing to run Windows and another a Linux.

Creating the Jobs

The following is the Workflow set up along with the job to build the first web application. The first change was of the ID from build to build_web_app1 since each job has to have a unique ID. Most of the rest of the highlighted changes are related to the .NET CLI commands that are now directed at a specific project. Do also note that we changed from a hardcoded path to using expression to get the workspace path which is the ${{ github.workspace }} bit instead of /home/runner/work/Playground/Playground/. See the expression syntax docs for more info.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/

Here is the full file with both jobs defined. As you can see the second job is basically the same thing as the first one with a different ID, name, and project.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
  build_web_app2:

    name: Build WebApp2
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
       
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp2
        path: ${{ github.workspace }}/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After all the edits are done commit the changes to the repo to run the Workflow. From the results of the Workflow run, you will see that it now has two jobs and we still got the artifacts for both applications as we had before.

Wrapping Up

If your applications need it breaking them up into different jobs can be helpful not only with Workflow runtimes but it can also help your ability to reason about what each part of your build process is doing.

GitHub: Use Actions to Publish Artifacts

This post is going to take the GitHub Actions Workflow we set up in the last post and add a couple of steps that will provide us with access to our application’s binaries. If you are new to this series the following post will catch you up if needed.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application

Edit the Workflow

Our first step is to get back to the Workflow we want to edit. At the top of the repo click Actions.

On the left side of the screen select the specific Workflow, .NET Core in this case. Now that the list is filtered to just the Workflow we are interested in select the three dots on the most recent run and then click View workflow file.

On the next screen click the pencil above the Workflow file to edit the Workflow.

Add the end of the file adds a call to the .NET CLI to publish. The following is the full file and the last two lines are the publish step.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish

Click the Start commit button and commit the change to the branch of your choice. I committed the change directly to master which triggered the Workflow to run. From the logs of the Workflow, you can see that the publish step executed successfully.

Publish Build Artifacts

Now that we have the two web applications publishing we need a way to get the file for the applications. To do this we are going to use the Upload Artifacts Action. I used the output of the Publish step above to find the path to the publish folder for each application and then used an Upload Artifacts Action for each application. The following are the two steps added to the bottom of our existing Workflow.

- name: Upload WebApp1 Build Artifact
  uses: actions/[email protected]
  with:
    name: WebApp1
    path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
    
- name: Upload WebApp2 Build Artifact
  uses: actions/[email protected]
  with:
    name: WebApp2
    path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After checking in the changes let the Workflow run. Once complete if you click on the details of the Workflow run you will see it now has Artifacts that can be downloaded.

Wrapping Up

We now have a Workflow that results in files we could actually deploy on top of verifying that the applications builds and the tests pass. I hope this has given you a good jumping-off point to build your own Workflows. We have only scratched the surface of what can be done with GitHub Actions and I’m looking forward to seeing what else we can do.

GitHub: Use Actions to build ASP.NET Core Application

In this week’s post, we are going to use GitHub’s Actions to build one of the applications that we imported from an Azure DevOps Repo. The sample repo we are using can be found here.

Create an Action Workflow

From the repo in GitHub click the Actions option at the top center of the screen.

The Actions page will make suggestions based on the contents of the repo you are working with. In our case, the suggested .NET Core workflow is the one we are interested in. Click the Set up this workflow button.

The next screen that shows will be an editor loaded with the YAML for the .NET Core workflow we selected. For now, we are going to keep the YAML that was defaulted in and click the Start commit button. This workflow may or may not work for our repo at this point we are still exploring and can change as needed after we get a feel for how Actions work.

The next dialog is the commit details. For this initial change, we are going commit directly to master with the default commit message. Click Commit new file to continue.

View Workflow Status

Now that we have a workflow set up click on the Actions tab of the repo again to view the list of workflows and their status. As you can see in this screenshot the commit queued our new workflow to run.

The workflow finished quickly so I didn’t get to see the details while it was running, but if you click on commit title, Create dotnetcore.yml in this example, it will take you to the detail of this workflow run. From this view, you will see the jobs for the workflow listed on the left side of the screen, we only have one job which is the build. When you click on a job you will see the logs from that job. The following screenshot is the sample build job with the details of the build step expanded to show that both WebApp1 and WebApp2 were built.

Wrapping Up

Hopefully, this post will give you a good jumping-off point to create your own GitHub Actions. I was impressed with how easy it was to get started and the wide verity of languages supported especially for a feature set that has been out for less than a year. Check back in next week for more exploration of Actions.

GitHub: Import an Azure DevOps Repo

Over the last couple of months, we have been exploring some of the features of Azure DevOps around Pipelines and Repos. I thought it would be interesting to see how the same type of setup and process might look using GitHub instead of Azure DevOps. I haven’t used GitHub other than for a basic repo before so I’m not sure how much of the Azure DevOps post will carry over, but we are going to find out. In this first post, we are going to import the repo we used in the Azure DevOps series from our Azure DevOps Repo.

Importing a Repo

To start in the project click the New button in the Repositories section of your GitHub dashboard.

On the next page click the Import a repository link.

The first thing the import process wants to know is the URL of our old repository. To get this we need to head over to our Azure DevOps Repo. Once in the Azure DevOps Repo click the Clone button.

When the dialog for clone shows you will see the URL, but before copying the URL hit the Generate Git Credentials button. This will create a username and password we will also need to enter when importing the repo at GitHub.

Here is the dialog after generating credentials.

Use the button next to the URL to copy URL and head back to GitHub and paste it into the URL box.  Next, enter a name for the repo and click Begin import.

The following is the screen you will see next while GitHub works on the import. Since the repo we are importing from needs credentials it will fail after a couple of minutes and ask for a login and password. If you hit refresh it will prompt you immediately without having to wait.

Copy and paste the values we generated in Azure DevOps above into the Login and Password boxes and then click Submit.

The page will update when the process is complete. GitHub will also send you an email so don’t feel the need to keep the page open while the process is running.

GitHub Repo Cleanup

Now that our repo is in GitHub we can clean up some of the items that were specific to Azure DevOps. For the sample project, this would include the files that were used in/or defined our build pipeline. The following are the files that can be deleted.

  • azure-pipelines.yml
  • build.yml

Wrapping up

As you have seen the repo transition from Azure DevOps to GitHub is a simple process. I’m looking forward to exploring how GitHub handles some of the scenarios from the Azure DevOps series. I’m betting that GitHub has support for most of them especially since the introductions of Actions.

Spring Ramblings

We are well into one of the strangest springs, well really years, that I can remember. Read no further if you’re looking for anything technical. As the title says this post is going to be a bit of rambling from me as I come off the series of posts on Azure DevOps.

COVID-19

COVID-19, Coronavirus, SARS-Cov-2, The Rona, or whatever your favorite name for the virus is the cause of this year being so strange. The virus has been devastating. We have been lucky that we don’t know anyone who has died so far. I don’t think anyone in the world has escaped the effects. Even if you or your loved ones haven’t been sick or died I’m sure like us you know someone who has lost their job at a minimum. More this as affected life will be mixed in the rest of the post, but I thought it best to make it clear upfront that the virus is the root of what is to follow. I also wanted to drive home that I realize how much it has changed life for so many people even when the rest of the post may not convey that level of seriousness.

Working from home

I have been lucky that my company basically went fully remote in March so I have been working form home for around two months now. Getting the job part of working remotely has been good. In fact, I can’t remember the last time I had so much time to focus. It is crazy how much time gets consumed with random interactions when you are physically in the office. One of the downsides is I really miss the level of interaction I had with the teams I work with. A lot of my random interactions aren’t personal and deal with answering questions and providing different perspectives on problems and I really enjoy that part of my job. It isn’t that those interactions don’t still happen when really needed, but they do seem to be much more intentional.

If you are working from home make sure you are willing to invest in your setup. While I was effective when I first started working from home my setup wasn’t 100% on par with work. I ended up buying a 4k monitor and a couple of things to switch inputs between my work and home equipment and it has made a huge difference. My home setup is now much better than the one I have at work. Being able to exit work mode and be back in home mode in seconds has also been great for when I finish work and the family has some stuff to do on the computer.

Blogging, reading, podcasts, etc.

Blogging in this new world hasn’t been easy for me. Thankfully I had a good set of content planned out with the Azure DevOps posts. Part of the reason for this post is I’m not sure what topics I am going to move to next. If anyone has any requests let me know!

I have slowed down a ton on reading and listening to podcasts since I’m no longer commuting. I have found a good way to fit in the same level of content when I’m not driving 1.25+ hours a day. While I don’t miss driving or the traffic it was a useful time to learn and a good transition period to and from work mode.

On the positive side not having a commute means that finding time to exercise has been easier, not that having time helps with the motivation to do so. It also means more time with the family which is also great, most days.

Wrapping Up

Hopefully, you enjoyed my spring rabblings. I try and keep this type of post to a minimum and stay focused on technical topics, but at times I need a break and this help. While I don’t say it much I really appreachte you all taking the time to read my blog.

Azure DevOps Pipelines: PowerShell Task

This is going to be a quick post that shows the use of the PowerShell task in a Pipeline. Nothing in the post is really specific to the Azure DevOps Project we have been using over the last few weeks, but just in case you’re totally new to Azure DevOps and/or this series you can use the following posts to get started.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging
Azure DevOps Pipelines: Depends On with Conditionals in YAML

PowerShell Task

The PowerShell task will allow you to do pretty much anything. If there isn’t an existing DevOps task that fits your needs more than likely you can find a way to use the PowerShell task to accomplish what you need within the context of the computer the task is running on and even external computers that depending on your networking and security setup. The following is a sample task I added to a Pipeline that will output all the environment variables to the logs. This is an inline script, but you can also run scripts from files. Also, note that this works on both Windows and Linux agents.

- task: [email protected]
  inputs:
    targetType: 'inline'
    script: 'Get-ChildItem -Path Env:\'

While this script isn’t super useful for a production Pipeline I often use it when setting up a Pipeline to get a good feel for what is available variable wise. Also, keep in mind that depending on the trigger of the run these variables can be different. For example, if a run was triggered by a pull request you will have a number of pull request related variables. The following is the output of this command on my test project that was triggered via a pull request and therefore contains a bunch of SYSTEM_PULLREQUEST_x variables with information about the pull request. The agent was running Linux.

Name                           Value
----                           -----
AGENT_ACCEPTTEEEULA            True
AGENT_BUILDDIRECTORY           /home/vsts/work/1
AGENT_DISABLELOGPLUGIN_TESTFI… true
AGENT_DISABLELOGPLUGIN_TESTRE… true
AGENT_HOMEDIRECTORY            /home/vsts/agents/2.165.2
AGENT_ID                       9
AGENT_JOBNAME                  Build WebApp1
AGENT_JOBSTATUS                Succeeded
AGENT_MACHINENAME              fv-az563
AGENT_NAME                     Hosted Agent
AGENT_OS                       Linux
AGENT_OSARCHITECTURE           X64
AGENT_READONLYVARIABLES        true
AGENT_RETAINDEFAULTENCODING    false
AGENT_ROOTDIRECTORY            /home/vsts/work
AGENT_TEMPDIRECTORY            /home/vsts/work/_temp
AGENT_TOOLSDIRECTORY           /opt/hostedtoolcache
AGENT_VERSION                  2.165.2
AGENT_WORKFOLDER               /home/vsts/work
agent.jobstatus                Succeeded
ANDROID_HOME                   /usr/local/lib/android/sdk
ANDROID_SDK_ROOT               /usr/local/lib/android/sdk
ANT_HOME                       /usr/share/ant
AZURE_EXTENSION_DIR            /opt/az/azcliextensions
AZURE_HTTP_USER_AGENT          VSTS_08ccc6b2-4e5e-4621-8f5b-3fe0de2efa22_build…
BOOST_ROOT_1_69_0              /usr/local/share/boost/1.69.0
BOOST_ROOT_1_72_0              /usr/local/share/boost/1.72.0
BUILD_ARTIFACTSTAGINGDIRECTORY /home/vsts/work/1/a
BUILD_BINARIESDIRECTORY        /home/vsts/work/1/b
BUILD_BUILDID                  73
BUILD_BUILDNUMBER              merge_20200422.1
BUILD_BUILDURI                 vstfs:///Build/Build/73
BUILD_CONTAINERID              3453972
BUILD_DEFINITIONNAME           Playground
BUILD_DEFINITIONVERSION        4
BUILD_QUEUEDBY                 Microsoft.VisualStudio.Services.TFS
BUILD_QUEUEDBYID               00000002-0000-8888-8000-000000000000
BUILD_REASON                   PullRequest
BUILD_REPOSITORY_CLEAN         False
BUILD_REPOSITORY_GIT_SUBMODUL… False
BUILD_REPOSITORY_ID            ff7a6325-1129-42e3-b095-6a39ef6a6bd3
BUILD_REPOSITORY_LOCALPATH     /home/vsts/work/1/s
BUILD_REPOSITORY_NAME          Playground
BUILD_REPOSITORY_PROVIDER      TfsGit
BUILD_REPOSITORY_URI           https://[email protected]/ericlanders…
BUILD_REQUESTEDFOR             Eric Anderson
BUILD_REQUESTEDFOREMAIL        [email protected]
BUILD_REQUESTEDFORID           45247cb1-8f49-4c03-a4c5-b03ac3286c99
BUILD_SOURCEBRANCH             refs/pull/10/merge
BUILD_SOURCEBRANCHNAME         merge
BUILD_SOURCESDIRECTORY         /home/vsts/work/1/s
BUILD_SOURCEVERSION            3e2b77c27f31a4c729a5f195b49d2e108500399d
BUILD_SOURCEVERSIONAUTHOR      Eric Anderson
BUILD_SOURCEVERSIONMESSAGE     Merge pull request 10 from docChanges into mast…
BUILD_STAGINGDIRECTORY         /home/vsts/work/1/a
BUILDCONFIGURATION             Release
BUILDWEBAPP2                   false
CHROME_BIN                     /usr/bin/google-chrome
CHROMEWEBDRIVER                /usr/local/share/chrome_driver
COMMON_TESTRESULTSDIRECTORY    /home/vsts/work/1/TestResults
CONDA                          /usr/share/miniconda
DEBIAN_FRONTEND                noninteractive
DOTNET_SKIP_FIRST_TIME_EXPERI… 1
ENDPOINT_URL_SYSTEMVSSCONNECT… https://dev.azure.com/ericlanderson/
GECKOWEBDRIVER                 /usr/local/share/gecko_driver
GIT_TERMINAL_PROMPT            0
GOROOT                         /usr/local/go1.14
GOROOT_1_11_X64                /usr/local/go1.11
GOROOT_1_12_X64                /usr/local/go1.12
GOROOT_1_13_X64                /usr/local/go1.13
GOROOT_1_14_X64                /usr/local/go1.14
GRADLE_HOME                    /usr/share/gradle
HOME                           /home/vsts
ImageOS                        ubuntu18
ImageVersion                   20200406.2
INPUT_ARGUMENTS                
INVOCATION_ID                  3e6abb812a484ab39fadc9e8721258ee
JAVA_HOME                      /usr/lib/jvm/zulu-8-azure-amd64
JAVA_HOME_11_X64               /usr/lib/jvm/zulu-11-azure-amd64
JAVA_HOME_12_X64               /usr/lib/jvm/zulu-12-azure-amd64
JAVA_HOME_7_X64                /usr/lib/jvm/zulu-7-azure-amd64
JAVA_HOME_8_X64                /usr/lib/jvm/zulu-8-azure-amd64
JOURNAL_STREAM                 9:30085
LANG                           C.UTF-8
LEIN_HOME                      /usr/local/lib/lein
LEIN_JAR                       /usr/local/lib/lein/self-installs/leiningen-2.9…
M2_HOME                        /usr/share/apache-maven-3.6.3
MSDEPLOY_HTTP_USER_AGENT       VSTS_08ccc6b2-4e5e-4621-8f5b-3fe0de2efa22_build…
PATH                           /opt/microsoft/powershell/7:/usr/share/rust/.ca…
PIPELINE_WORKSPACE             /home/vsts/work/1
POWERSHELL_DISTRIBUTION_CHANN… Azure-DevOps-ubuntu18
PSModulePath                   /home/vsts/.local/share/powershell/Modules:/usr…
RUNNER_TOOLSDIRECTORY          /opt/hostedtoolcache
SELENIUM_JAR_PATH              /usr/share/java/selenium-server-standalone.jar
SWIFT_PATH                     /usr/share/swift/usr/bin
SYSTEM                         build
SYSTEM_ARTIFACTSDIRECTORY      /home/vsts/work/1/a
SYSTEM_COLLECTIONID            08ccc6b2-4e5e-4621-8f5b-3fe0de2efa22
SYSTEM_COLLECTIONURI           https://dev.azure.com/ericlanderson/
SYSTEM_CULTURE                 en-US
SYSTEM_DEFAULTWORKINGDIRECTORY /home/vsts/work/1/s
SYSTEM_DEFINITIONID            5
SYSTEM_DEFINITIONNAME          Playground
SYSTEM_ENABLEACCESSTOKEN       SecretVariable
SYSTEM_HOSTTYPE                build
SYSTEM_ISSCHEDULED             False
SYSTEM_JOBATTEMPT              1
SYSTEM_JOBDISPLAYNAME          Build WebApp1
SYSTEM_JOBID                   98395c9e-7365-5c3f-03de-ec42b09a8a98
SYSTEM_JOBIDENTIFIER           WebApp1.__default
SYSTEM_JOBNAME                 __default
SYSTEM_JOBPARALLELISMTAG       Private
SYSTEM_JOBPOSITIONINPHASE      1
SYSTEM_PHASEATTEMPT            1
SYSTEM_PHASEDISPLAYNAME        Build WebApp1
SYSTEM_PHASEID                 a142d6c6-ff80-5cff-8292-5044e2c5b0ef
SYSTEM_PHASENAME               WebApp1
SYSTEM_PIPELINESTARTTIME       2020-04-22 06:11:44-05:00
SYSTEM_PLANID                  726fda14-a3a2-45b1-b745-bef8cf17bdaa
SYSTEM_PULLREQUEST_ISFORK      False
SYSTEM_PULLREQUEST_PULLREQUES… 10
SYSTEM_PULLREQUEST_PULLREQUES… 1
SYSTEM_PULLREQUEST_SOURCEBRAN… refs/heads/docChanges
SYSTEM_PULLREQUEST_SOURCECOMM… ba11cb768bc75ae65ff6b7ac6afb8a2950063f07
SYSTEM_PULLREQUEST_SOURCEREPO… https://[email protected]/ericlanders…
SYSTEM_PULLREQUEST_TARGETBRAN… refs/heads/master
SYSTEM_SERVERTYPE              Hosted
SYSTEM_STAGEATTEMPT            1
SYSTEM_STAGEDISPLAYNAME        __default
SYSTEM_STAGEID                 96ac2280-8cb4-5df5-99de-dd2da759617d
SYSTEM_STAGENAME               __default
SYSTEM_TASKDEFINITIONSURI      https://dev.azure.com/ericlanderson/
SYSTEM_TASKDISPLAYNAME         PowerShell
SYSTEM_TASKINSTANCEID          6417fa85-e8cf-55f9-817e-d698bd79d6f7
SYSTEM_TASKINSTANCENAME        PowerShell
SYSTEM_TEAMFOUNDATIONCOLLECTI… https://dev.azure.com/ericlanderson/
SYSTEM_TEAMFOUNDATIONSERVERURI https://dev.azure.com/ericlanderson/
SYSTEM_TEAMPROJECT             Playground
SYSTEM_TEAMPROJECTID           7550ca2f-9ffe-45b7-abd5-c4e92a4a5f4e
SYSTEM_TIMELINEID              726fda14-a3a2-45b1-b745-bef8cf17bdaa
SYSTEM_TOTALJOBSINPHASE        1
SYSTEM_WORKFOLDER              /home/vsts/work
TASK_DISPLAYNAME               PowerShell
TF_BUILD                       True
USER                           vsts
VCPKG_INSTALLATION_ROOT        /usr/local/share/vcpkg
VSTS_AGENT_PERFLOG             /home/vsts/perflog
VSTS_PROCESS_LOOKUP_ID         vsts_54420f58-c41f-4a43-8ce8-bbbac5023620

I don’t know about you but being able to see what paths the built-in path variables actually map to helps me a lot especially when files need to be moved around.

Wrapping Up

As stated above you can do just about anything with the PowerShell task. I have used it for everything from reading a JSON file to building a VM for QA. If you hadn’t used this task before I hope this post helped you get started and opened your eyes to the huge range of things you can do with the PowerShell task.

Azure DevOps Repos: Bypass Branch Policies

Last week we covered adding branch policies to a branch in an Azure DevOps Repo and this week we are going to deal with what happens when you need to break the policies you set up for some reason.

Setting Bypass Security

There are a couple of ways to set up bypassing depending on how broadly you want to give someone rights to bypass. We are going to start with the narrower option which is allowing bypass for an individual branch (sadly security options are currently available at the folder level like policies are). Starting from the list of branches for your repo mouse over the branch you want to set security for and click the three dots for the menu and select Branch security.

On the dialog that shows find the user, you want to change for security for, Eric Anderson in this example. After selecting a user their specific setting will be loaded to the right. On Bypass policies when completing pull requests change the option to Allow.

The second option for setting bypass security is at the repo level. From your Project settings under Repos select Repositories and then fine the Branches node under the project you want to set the policy for.

From here it is the same as the branch level. Find the user, you want to change for security for and set Bypass policies when completing pull requests to Allow.

Bypassing Policies on a Pull Request

Now that we have our security setup we are going to walk through what the bypass process looks like. Here we have a pull request that is missing approval by a reviewer.

Now let us say we don’t have a reviewer available for some reason and we need to complete this PR without review. Use the dropdown on the Set auto-complete button and click Complete.

When the PR completion dialog show you will notice a section at the top with a red background that lists out the policies that haven’t been met. With your new-found security, you will also have a section for Policy override options. To proceed and bypass the policies check the Override branch policies and enable merge checkbox, enter your reason for overriding, and click the Override and complete button.

Do note that the fact a PR was overridden is visible on the list of completed PRs and the reason will show when mousing over the bypassed indicator as well as in the details of the PR.

Wrapping Up

Branch policies are great and will help you make sure the code that makes it in your branches are high quality and don’t break your builds. Hopefully, you won’t need to bypass your policies often, but now you know-how without having to temporarily remove the policies or getting people used to blindly approving changes.

Azure DevOps Repos: Branch Policies

For the last few weeks I have been doing a series of posts about Azure DevOps Pipelines and I hit a post I wanted to do that didn’t make sense without introducing the Branch Policies feature of Azure Repos. This post is going to assume you already have an Azure DevOps Project with some code in it. If not you can check out my post on Getting Started with Azure DevOps.

Repo Introduction

The repo used here is the same one used in the Pipelines posts linked above and contains two .NET Core 3.1 web applications. The repo also contains three branches (master, releases/1.0, and releases/1.1).

It is worth noting that putting a forward slash in a branch name displays as a folder in the UI as you can see with releases in the screenshot above.

Editing Policies

Mouse over either a specific branch or a folder and it will show the three dots for the menu. Click the dots and then select Branch policies. For this example, we are putting policies on the master branch.

This will bring you to the page that allows you to view and edit the policies on the selected branch or folder.

The descriptions do a good job of explaining what policies do what so I’m not going to bore you with repeating them. The official docs on branch policies also go into a lot more detail. If you are not working along I highly recommend using Require a minimum number of reviews and Check for comment resolution. Build validation I would recommend no matter your team size as it keeps you away from the possibility of having that one magical machine that is the only one your build will work on. The following screenshot is with the first two recommend policies set. The settings shown for the require a minimum number of reviews are based on the fact that my project only has one contributor. After your done make sure and click the Save changes button.

Build Validation Policies

There is a bit more to the build validation policy which is why I’m covering it in a different section. On the Branch policies screen click the Add build policy button.

In the edit build policy screen, the only required change is selecting the Build pipeline to make available when a PR that is targeting the branch that is policy is for. Here we are using the automatic Trigger so any time we push to our remote branch when it has an open pull request it will run the select build pipeline. Policy requirement controls if a successful build is required before the pull request can complete or not. For a since person project Build expiration isn’t a big deal, but if your working with a team it can be helpful. Click Save when you are done.

Back on the branch policies screen, you will see the new requirement listed. You can also add as many build validations as you need.

Wrapping Up

Hopefully this quick little into to branch policies will help your team improve the quality of the code that makes it into your branches. I know to require another person or two to review your code before you can check-in sounds like it will slow you down if you are new to the concept but in reality, it helps catch issues before they make it to QA and production which saves time and money in the long run.

Azure DevOps Pipelines: Depends On with Conditionals in YAML

A few weeks ago we covered Conditionals in YAML to show how to conditionally run tasks and jobs as well as how to make a job dependent on another job. This post is going to cover combing conditional and job dependencies. If you are new to this series you can use the following posts to catch up.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging

Existing Job

As a reminder, our sample pipeline has 4 jobs. The WebApp1 job always runs, the WebApp2 job is run or skipped based on a pipeline variable, the DependentJob depends on WebApp1 and WebApp2, and finally, the TagSources job is dependent on all the previous jobs. We are going to be tweaking the TagSources job in this post. The following is the YAML for the setup of the TagSources job without its tasks.

- job: TagSources
  displayName: 'Tag Sources'
  pool:
    vmImage: 'ubuntu-latest'

  dependsOn:
  - WebApp1
  - WebApp2
  - DependentJob

With this setup WebApp1, WebApp2, and DependentJob all have to report successful or the TagSources job will be skipped. The following screenshot shows a pipeline run with the variable to build WebApp2 set to false.

As you can see the TagSources job was skipped because one of its dependent jobs was skipped.

Dependencies with Conditions

Let us say for our pipeline we want the TagSources job to run as long as all jobs were successful or if WebApp1 was successful and the WebApp2 and DependentJob jobs were skipped. To do this we are going to add a condition element and manually check the results of the dependencies as you can see in the following.

- job: TagSources
  displayName: 'Tag Sources'
  pool:
    vmImage: 'ubuntu-latest'

  dependsOn:
  - WebApp1
  - WebApp2
  - DependentJob
  condition: |
    and
    (
      eq(dependencies.WebApp1.result, 'Succeeded'),
      in(dependencies.WebApp2.result, 'Succeeded', 'Skipped'),
      in(dependencies.DependentJob.result, 'Succeeded', 'Skipped')
    )

And you can see in the results the TagSources job ran even with the two skipped jobs.

Wrapping Up

If there is a simpler way to accomplish what we did above I would love to hear about it. If I remember correctly I found the above in a GitHub issue, but I don’t have the link. I’m not sure how many of you will have Pipelines that will need this, but hopefully, this will save someone some research time.