.NET Core

GitHub: Use Actions to Run Multiple Jobs

In the post, we are going to take our sample Workflow that builds two ASP.NET Core web applications and split it so each web application is built individually. This post is using the repo and Workflow built in the following posts if you need to catch up.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application
GitHub: Use Actions to Publish Artifacts

Starting Point and the Plan

Our Workflow currently contains a single job that just happens to build two ASP.NET Core web application based on the fact that the .NET CLI picks up and builds both applications. The following is the YAML for our current Workflow.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish 
    
    - name: Upload WebApp1 Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
    - name: Upload WebApp2 Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp2
        path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

This post is going to take this Workflow and split the build and publish of the two web applications into two jobs. By splitting the Workflow into multiple jobs we open the possibility that the jobs can run in parallel. One reason to do this would be to speed up the total Workflow run time if you have parts of your build that are independent. Another example of why you would need multiple jobs is if the different jobs need different needed to run on different operating systems such as one needing to run Windows and another a Linux.

Creating the Jobs

The following is the Workflow set up along with the job to build the first web application. The first change was of the ID from build to build_web_app1 since each job has to have a unique ID. Most of the rest of the highlighted changes are related to the .NET CLI commands that are now directed at a specific project. Do also note that we changed from a hardcoded path to using expression to get the workspace path which is the ${{ github.workspace }} bit instead of /home/runner/work/Playground/Playground/. See the expression syntax docs for more info.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/

Here is the full file with both jobs defined. As you can see the second job is basically the same thing as the first one with a different ID, name, and project.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
  build_web_app2:

    name: Build WebApp2
    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
       
    - name: Upload Build Artifact
      uses: actions/[email protected]
      with:
        name: WebApp2
        path: ${{ github.workspace }}/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After all the edits are done commit the changes to the repo to run the Workflow. From the results of the Workflow run, you will see that it now has two jobs and we still got the artifacts for both applications as we had before.

Wrapping Up

If your applications need it breaking them up into different jobs can be helpful not only with Workflow runtimes but it can also help your ability to reason about what each part of your build process is doing.

GitHub: Use Actions to Publish Artifacts

This post is going to take the GitHub Actions Workflow we set up in the last post and add a couple of steps that will provide us with access to our application’s binaries. If you are new to this series the following post will catch you up if needed.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application

Edit the Workflow

Our first step is to get back to the Workflow we want to edit. At the top of the repo click Actions.

On the left side of the screen select the specific Workflow, .NET Core in this case. Now that the list is filtered to just the Workflow we are interested in select the three dots on the most recent run and then click View workflow file.

On the next screen click the pencil above the Workflow file to edit the Workflow.

Add the end of the file adds a call to the .NET CLI to publish. The following is the full file and the last two lines are the publish step.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/[email protected]
      
    - name: Setup .NET Core
      uses: actions/[email protected]
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish

Click the Start commit button and commit the change to the branch of your choice. I committed the change directly to master which triggered the Workflow to run. From the logs of the Workflow, you can see that the publish step executed successfully.

Publish Build Artifacts

Now that we have the two web applications publishing we need a way to get the file for the applications. To do this we are going to use the Upload Artifacts Action. I used the output of the Publish step above to find the path to the publish folder for each application and then used an Upload Artifacts Action for each application. The following are the two steps added to the bottom of our existing Workflow.

- name: Upload WebApp1 Build Artifact
  uses: actions/[email protected]
  with:
    name: WebApp1
    path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
    
- name: Upload WebApp2 Build Artifact
  uses: actions/[email protected]
  with:
    name: WebApp2
    path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After checking in the changes let the Workflow run. Once complete if you click on the details of the Workflow run you will see it now has Artifacts that can be downloaded.

Wrapping Up

We now have a Workflow that results in files we could actually deploy on top of verifying that the applications builds and the tests pass. I hope this has given you a good jumping-off point to build your own Workflows. We have only scratched the surface of what can be done with GitHub Actions and I’m looking forward to seeing what else we can do.

GitHub: Use Actions to build ASP.NET Core Application

In this week’s post, we are going to use GitHub’s Actions to build one of the applications that we imported from an Azure DevOps Repo. The sample repo we are using can be found here.

Create an Action Workflow

From the repo in GitHub click the Actions option at the top center of the screen.

The Actions page will make suggestions based on the contents of the repo you are working with. In our case, the suggested .NET Core workflow is the one we are interested in. Click the Set up this workflow button.

The next screen that shows will be an editor loaded with the YAML for the .NET Core workflow we selected. For now, we are going to keep the YAML that was defaulted in and click the Start commit button. This workflow may or may not work for our repo at this point we are still exploring and can change as needed after we get a feel for how Actions work.

The next dialog is the commit details. For this initial change, we are going commit directly to master with the default commit message. Click Commit new file to continue.

View Workflow Status

Now that we have a workflow set up click on the Actions tab of the repo again to view the list of workflows and their status. As you can see in this screenshot the commit queued our new workflow to run.

The workflow finished quickly so I didn’t get to see the details while it was running, but if you click on commit title, Create dotnetcore.yml in this example, it will take you to the detail of this workflow run. From this view, you will see the jobs for the workflow listed on the left side of the screen, we only have one job which is the build. When you click on a job you will see the logs from that job. The following screenshot is the sample build job with the details of the build step expanded to show that both WebApp1 and WebApp2 were built.

Wrapping Up

Hopefully, this post will give you a good jumping-off point to create your own GitHub Actions. I was impressed with how easy it was to get started and the wide verity of languages supported especially for a feature set that has been out for less than a year. Check back in next week for more exploration of Actions.

Azure DevOps Pipelines: Reusable YAML

In this post, we are going to refactor our sample Azure DevOps Pipeline to move some of the redundant YAML to a new file and replace the redundant parts of our main YAML file. This post is going to build on the Azure DevOps project created in previous posts. If you are just joining this series check out the previous posts to find out how the project has progressed.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML

Starting YAML

The following is the YAML for our current pipeline that builds two different web applications using two different jobs. Looking at the two jobs you will notice that they both have the same steps. The only difference in the steps is which project to build (WebApp1.csproj or WebApp2.csproj) and what to call the published artifact (WebApp1 or WebApp2). When developing applications we would never stand for this level of duplication and the same should apply to our pipelines.

trigger: none

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration)' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: 'WebApp1'
      publishLocation: 'pipeline'

- job: WebApp2
  displayName: 'Build WebApp2'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/WebApp2.csproj'
      arguments: '--configuration $(buildConfiguration)' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/WebApp2.csproj'
      arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: 'WebApp2'
      publishLocation: 'pipeline'

Add a New File

To attack the duplication above we need to take the shared steps from above and move them somewhere they can be reused. We will be walking through the steps using the Azure DevOps web site and committing directly to the master branch, but these same steps could be performed locally or on the web on any branch. First, from the Repos section of the site we need to add a new file by clicking the three dots at the level we want the file added. In this case, we are adding to the root of the repo but the same option is available on any folder.

A dialog will show where you can enter the New file name, we are going to use build.yml in this case. Next, click Create to continue.

Shared YAML

Now that we have a new file we can start building the new YAML that will handle the repeated steps from the original jobs. The first thing we are going to do is define a set of parameters that this set of steps can be called with. We are going to use this to pass what project to build, which build configuration to use, and what name the published artifact. The following is the definition of our parameters.

parameters:
- name: buildConfiguration
  type: string
  default: 'Release'
- name: project
  type: string
  default: ''
- name: artifactName
  type: string
  default: ''

We can then use these parameters in the rest of the file using the ${{ parameterName }} syntax. Note that any pipeline variables are also available using the $(variableName) syntax. The following bit of YAML shows both types in the arguments line.

- task: [email protected]
  displayName: 'Publish Application'
  inputs:
    command: 'publish'
    publishWebProjects: false
    projects: '**/${{ parameters.project }}'
    arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Build.ArtifactStagingDirectory)'

While you can use pipeline variables I recommend passing all the values you need via parameters for the same reason that we try to avoid global variables when doing general programming. I’m using both here to show the usage of each. The following is the full YAML in our new file.

parameters:
- name: buildConfiguration
  type: string
  default: 'Release'
- name: project
  type: string
  default: ''
- name: artifactName
  type: string
  default: ''

steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/${{ parameters.project }}'
      arguments: '--configuration ${{ parameters.buildConfiguration }}' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/${{ parameters.project }}'
      arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: ${{ parameters.artifactName }}
      publishLocation: 'pipeline'

Finally, commit the changes to the new file.

Using Shared YAML

Not that we have the YAML that is the same between our two build jobs we can switch back over to our main YAML file, azure-pipelines.yml in the sample, and remove the steps we are wanting to replace. While the jobs will both have a steps section the only thing we will have left in them is a template call to our other YAML file, build.yml for the sample, that passes the parameters to run the other file with. The following is the resulting YAML file with the call to the shared file in both jobs highlighted.

trigger: none

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - template: build.yml
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp1.csproj
      artifactName: WebApp1

- job: WebApp2
  displayName: 'Build WebApp2'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - template: build.yml
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp2.csproj
      artifactName: WebApp2

Wrapping Up

Being able to remove duplication from your YAML files should help improve the maintainability of your pipelines. I know the samples don’t show it, but the template is just a step and you could have other steps before or after it just like you would with normal tasks.

Azure DevOps Pipelines: Multiple Jobs in YAML

This post is going to show how to run multiple jobs out of a single YAML file from an Azure DevOps Pipeline. This post is going to build on the Azure DevOps project created in previous posts. If you are just joining this series check out the previous posts to find out how the project has progressed.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core

Starting Point and the Plan

As the sample stands now we have a single Pipeline that builds two different ASP.NET Core web applications in a single job using the following YAML.

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: [email protected]
  inputs:
    packageType: 'sdk'
    version: '3.1.x'

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'dotnet build $(buildConfiguration)'
  
- task: [email protected]
  inputs:
    command: 'publish'
    publishWebProjects: true
    arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

- task: [email protected]
  inputs:
    targetPath: '$(Build.ArtifactStagingDirectory)'
    publishLocation: 'pipeline'

This post is going to take this pipeline and split the build and publish of the two web applications and make each application its own job. In Pipelines a job is something that a single agent takes and runs. By splitting into multiple jobs the pipeline can run multiple jobs at the same time if you have enough build agents available. One reason to do this would be to speed up the total Pipeline run if you have parts of your build that are independent. Another example of why you would need jobs is if the different jobs need different agents such as one needing a Windows agent and another a Linux agent.

Creating the Jobs

Having different jobs means we are going to have to move things like what agent pool to use and the steps for the job under a jobs element and then declare a specific job and the details that job needs to run. As you can see in the following example the end goal is the same as the YAML from above (except it is dealing with a specific project), but the details are nested under jobs and defined under a job.

trigger:
- master

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration)' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: 'WebApp1'
      publishLocation: 'pipeline'

Also notice that you can still define variables that can be used across jobs as is done above with the buildConfiguration variable. The following is the full YAML file that builds and publishes the artifacts for both web applications.

trigger:
- master

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration)' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/WebApp1.csproj'
      arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: 'WebApp1'
      publishLocation: 'pipeline'

- job: WebApp2
  displayName: 'Build WebApp2'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: [email protected]
    displayName: 'Use .NET 3.1.x'
    inputs:
      packageType: 'sdk'
      version: '3.1.x'

  - task: [email protected]
    displayName: 'Build'
    inputs:
      command: 'build'
      projects: '**/WebApp2.csproj'
      arguments: '--configuration $(buildConfiguration)' 
  
  - task: [email protected]
    displayName: 'Publish Application'
    inputs:
      command: 'publish'
      publishWebProjects: false
      projects: '**/WebApp2.csproj'
      arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

  - task: [email protected]
    displayName: 'Publish Artifacts'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifact: 'WebApp2'
      publishLocation: 'pipeline'

After all your edits are done commit the changes to your YAML file and then run the pipeline. As you can see from the following screenshot of my sample pipeline run the pipeline has two jobs instead of one that the original YAML resulted in. Also, note that the pipeline results in two published artifacts (one per job in our case) instead of the one with the original.

Wrapping Up

As mentioned above there are a lot of reasons you might want to split up your pipeline into multiple jobs and hopefully, you now have a good idea of how that is done. Make sure and check back in the future for a post on how to take repeated tasks and make them reusable.

Azure DevOps Publish Artifacts for ASP.NET Core

This post is going to build on the Azure DevOps project we created in the last few posts and get the build pipeline to the point you have the application’s binaries. If you are just joining this series check out the previous posts to catch up.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps

Edit the Pipeline

First, we need to get back to the pipeline we were working on. From the Project menu select Pipelines.

This will land you on a page that lists your recently run pipelines. If you don’t see your pipeline list you might have to click the All option near the top of the page. Since we only have one pipeline in this project we can use the ellipsis to open a context menu and click Edit.

Publish the Application

At this point, the YAML for our pipeline looks like the following.

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: [email protected]
  inputs:
    packageType: 'sdk'
    version: '3.1.x'

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'dotnet build $(buildConfiguration)'

The pipeline will currently tell us if the included project builds, but doesn’t provide us with the results of that build. Using the Task panel on the right search for the .NET Core task and then click the resulting task. This is the task you would want to use to invoke any of the .NET CLI commands.

Use the drop-down for Command and select publish. For this sample, the defaults for the rest of the settings will be fine. Finally, click Add to add the task to the YAML file.

The following is the resulting YAML.

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: [email protected]
  inputs:
    packageType: 'sdk'
    version: '3.1.x'

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'dotnet build $(buildConfiguration)'
  
- task: [email protected]
  inputs:
    command: 'publish'
    publishWebProjects: true

Before we move on I want to point out the Settings link above the tasks in the YAML editor. Clicking Settings will load that task into the task panel on the right of the screen where you can make changes and then if you hit the add button it will replace your existing task with a new one with your new options selected. Be careful to not change the selection in the YAML editor as the add button is just replacing the selected text not remembering what task you click settings on. When finished click the Save button and go through the commit process. When that is finished click the Run button to execute the pipeline.

Publish Build Artifacts

The pipeline run should succeed, but we still don’t have any files we can use. Learning what variables are available in the pipeline and how to use them is one of the hardest parts of getting started with Azure Pipelines. For our example, we are trying to get the two zip files created by the publish step above which means our pipeline will need to publish artifacts to make the files available. We are going to tweak the publish command from above with an output directory using the builtin Build.ArtifactStagingDirectory variable. The following is the full task with the changes.

- task: [email protected]
  inputs:
    command: 'publish'
    publishWebProjects: true
    arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

With the file we need in the artifact staging directory we need to publish those files using the Publish Pipeline Artifact task. The following is the full task that publishes the artifact staging directory to the pipeline.

- task: [email protected]
  inputs:
    targetPath: '$(Build.ArtifactStagingDirectory)'
    publishLocation: 'pipeline'

For reference, the following is the full YAML for the pipeline with all the above changes.

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: [email protected]
  inputs:
    packageType: 'sdk'
    version: '3.1.x'

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'dotnet build $(buildConfiguration)'
  
- task: [email protected]
  inputs:
    command: 'publish'
    publishWebProjects: true
    arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)'

- task: [email protected]
  inputs:
    targetPath: '$(Build.ArtifactStagingDirectory)'
    publishLocation: 'pipeline'

Save and run the pipeline. When the pipeline is complete on the result page you will see 1 published for artifacts.

Click on 1 published and it will take you to a page that lists the artifacts. If you mouse over any of the rows you will see the option to download the associated file(s).

Quick Tip

As I stated above getting a handle on what directories are where can be a pain. If you ever need to see what files are where you can use the following publish task to output the full set of files the pipeline is using by publishing the pipeline’s entire workspace. This has helped me in the past to orient myself.

- task: [email protected]
  inputs:
    targetPath: '$(Pipeline.Workspace)'
    publishLocation: 'pipeline'

Wrapping Up

Our pipeline is now at the point we have files we could deploy. Hopefully, this gives you a good jump start on your own build pipelines. Azure Pipelines is a huge topic and this is a very basic build so keep an eye out for more posts on this topic in the future.

Pipeline Creation in Azure DevOps

This post is going to walk through creating a new build pipeline in Azure DevOps. This post is going to stick with a very simple example which we will build on in future posts. If you are new to this series of post check out the related posts.

Getting Started with Azure DevOps

This post will all happen from the Azure DevOps website so get logged in to your account select the project you will be working with before continuing. The project this sample is using is named Playground.

Pipeline Creation

From the project menu on the right of the site click the Pipelines option.

Since our sample project doesn’t have any pipelines setup we will see a landing page telling us to create a new pipeline. Once you have some pipelines this page is a lot more useful. Click the Create Pipeline button to continue.

The next step is to pick where our code is stored. For this sample, the code is in an Azure Repo Git repository, but as you can see Azure DevOps is pretty open about where your code is stored. As you will see from the screenshot there are a bunch of YAML based options and a very small option to use the classic editor. The classic editor is much easier to get started with, but the YAML options are getting the most attention from Microsoft and have the advantage of being stored in Git with your source so I recommend going with a YAML option even though there is more of a learning curve.

Next, select the repo this pipeline is for.

In the next step, Configure, you are given a list of templates to pick from which really helps when your new to yaml. Our sample applications are ASP.NET Core so click the Show More button and click ASP.NET Core. As you can see from the screenshot Azure DevOps can build just about anything and isn’t restricted to Microsoft based tech.

The result is the following YAML file. At this point, we aren’t doing to dive into the particulars of what the YAML is doing and go with the default. To continue to click the Save and run button.

Since the YAML is stored in the repo the save process is actually making a commit to a branch. Click the Save and run button on the commit dialog and it will save the YAML file to your branch and run the pipeline.

Pipeline Results

The following are the results from the pipeline run and it turns out to have failed. If you click the highlighted error it will take you to the detailed logs of the pipeline which will normally give you a good indication of why the pipeline failed.

The following screenshot is the result of clicking on the error. As you can see it provides a the output of the build command.

In this case line, 43 provides us with the reason the build failed. The following is the full line since the screenshot cuts it off. In this case, the issue is the agent running the build doesn’t have .NET Core 3.1 installed.

usr/share/dotnet/sdk/3.0.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(127,5): error NETSDK1045: The current .NET SDK does not support targeting .NET Core 3.1. Either target .NET Core 3.0 or lower, or use a version of the .NET SDK that supports .NET Core 3.1. [/home/vsts/work/1/s/src/WebApp1/WebApp1.csproj]

Fixing the Pipeline

Click the back button in your browser to return to the pipeline results page. Click the three dots in the top right of the results page and select Edit pipeline. This will open an edit with the YAML for the build open.

Using the Tasks helper on the right side of the screen we are going to select the Use .NET Core task which will allow us to install the version of .NET Core we need to build our applications.

Enter the version of .NET Core your application needs, 3.1 in this case and hit add.

The following is the resulting YAML. Note that the above helper isn’t required and you can hand-edit the YAML if you want.

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
- task: [email protected]
  inputs:
    packageType: 'sdk'
    version: '3.1.x'

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'dotnet build $(buildConfiguration)'

Hit the Save button in the top right of the page, enter a commit message and click the Save button in the bottom right of the page. This will return you back to the edit screen for your YAML. Click the Run button on the top right of the page to start the pipeline.

This round (for the sample application) the pipeline will succeed.

Wrapping Up

Hopefully, this will give you a good jumping-off point to build your first Azure DevOps Pipeline. There is a lot a depth in Pipeline some of which I will explore in some future posts.

Getting Started with Azure DevOps

I have been doing a lot of work lately with our build and release pipelines in Azure DevOps and while it is fresh on my mind I’m going to do a few posts to remind me of some of the things I have leaned and may or may not have gotten to use. This post is going to serve as a jumping-off point for the post that will follow for readers who may not have used Azure DevOps in the past.

Before moving forward make sure and sign up for a free Azure DevOps account.

Create a new Project

When you first log in to Azure DevOps you should end up at something like the following that lists all your organizations on the left, ericlanderson being the sample organization I’m using. In the man section of the page make sure you are on the Projects tab and then click the New project button.

On the next screen, all that is required is a Project name. If you do want to allow public access to the project you will have to tweak an organization policy. Under Advanced you can also tweak how you want to manage work for the project, but that is out the scope of the point of this post. When done click the Create button.

Initialize a Repo

When project creation is complete you will be forwarded to the project landing page as you can see below. Now we want to initialize a new repo for this project. Click Repos button.

Since the sample projects are going to be .NET based I opted to use a .gitignore for Visual Studio. When the options you want are selected click the Initialize button.

When the initialization process is complete you will be taken to a files view of the repo.

Connect to project and clone the repo from Visual Studio

Now that we have the Azure DevOps setup it is time to switch over to Visual Studio and interact with our new project. In Visual Studio all the interactions with Azure DevOps will happen via the Team Explorer window. The first step is to connect to the project. In the Team Explorer window hit the plug icon.

Next, click Manage Connections and then click Connect to a Project.

In the dialog that shows you will need to use the drop-down and select Add an account if you already have an account connected as I do, if not then the process might be slightly different.

After completing the login process with your Azure DevOps credentials you should see your organization listed and under it your project. In this case, the project and repo have the same name. Select the repo and click Clone.

Create Sample Applications

Now that the repo is cloned I’m creating a couple of sample applications using the .NET CLI. Open a command prompt in the root folder of the clone from above and use the following command to create a new Visual Studio solution.

dotnet new sln -n Playground

Next, use the following command to create a new web application.

dotnet new webapp -o src/WebApp1

Then add the new project to the solution file.

dotnet sln add src/WebApp1/WebApp1.csproj

I then repeated the project creation process a second mostly to give us more to work with in future posts. Here are the commands to create the second project and add it to the existing solution.

dotnet new webapp -o src/WebApp2
dotnet sln add src/WebApp2/WebApp2.csproj

Commit Code and Push to Azure DevOps

Back in Visual Studio in the Team Explorer window, we need to switch to the Changes area. There are a couple of ways to get to the changes area. If you are in the home area you can click the changes button.

Another option is to click the current area and select the new area from the drop-down menu. For example, in the following screenshot, I clicked on Branches to show the drop-down and could then click Changes.

Enter a commit message and if you use the dropdown on the Commit All button and select Commit All and Sync it will push all of the changes to the associated Azure DevOps branch.

Wrapping Up

Hopefully if you are new to Azure DevOps this will give you a good jump-off point and will provide a base for some future posts.

Upgrade an ASP.NET Core Application to Bootstrap 4

As part of the work, I did on the application referenced in the web.config transform post from last week I also upgraded the application from Bootstrap 3 to Bootstrap 4 and this post is going to walk through a process to get the new version of Bootstrap up and running in a site.

Create a Sample Application

In case you want a project with Bootstrap 3 installed to play around with before attempting a production site the following command, run in a command prompt, will create an ASP.NET Core 2.1 application that has Bootstrap 3.

dotnet new webapp -f netcoreapp2.1

Upgrade Bootstrap Files

The first step I took was to delete the existing Bootstrap related file which can be found in the wwwroot/lib directory and delete the whole bootstrap directory.

Now open the project in Visual Studio and right-click on the project file and select Add > Client-Side Library.

The dialog that shows will allow you to search for client-side libraries to include in your application. In this case, we are looking for twitter-bootstrap. Your version number may be higher than the one shown in the screenshot below. I change the target location to match where Bootstrap was installed by the template by default, but that isn’t required. Finally, click Install.

The dialog above and the installation of the file is part of LibMan, which provides a lot of functionality for managing client-side libraries. Check out the official LibMan docs for more information.

Dealing with the Bootstrap Changes

This section is going to be really light on details as Bootstrap 4 was a reboot of Bootstrap and a ton of things changed. I will give an example or two here, but really the official Bootstrap Migrating to v4 guide is where you will want to go for all the details.

Here are a couple of examples of the type of changes you will be making.

Before:
<label asp-for="User" class="control-label"></label>

After:
<label asp-for="User" class="col-form-label"></label>

The look of buttons has changed so where we had used default before we changed to secondary.

Before:
<input type="submit" value="Load" class="btn btn-default" />

After:
<input type="submit" value="Load" class="btn btn-secondary" />

Wrapping Up

Hopefully, this will help you get a jump start on upgrading sites from Bootstrap 3 to Bootstrap 4. Thankfully after you get the files in the right spot Bootstrap’s migration guide should get you the rest of the way.

ASP.NET Core 3.1 Web.config Transform for Production

I recently upgraded an application from ASP.NET Core 2.2 to 3.1 around the same time the application got moved to a new server. Side note, don’t make two large changes like that at the same time if you can help it as it always makes tracking down the cause of the issue much harder.  The application in question was created using the ASP.NET Core React template initially.

The Error

After the changes above the site started returning the following error.

HTTP Error 500.0 – ANCM In-Process Handler Load Failue

To diagnose the issue I enabled standard out logging by changing the stdoutLogEnabled to true. The following is the line from the application’s web config with the logging enabled.

<aspNetCore processPath="dotnet" arguments=".\YourApplication.dll" stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout">

The Cause

The logging showed that the application was trying to start the React part of the application using the React Development Server, which uses npm. Here is the related code from the Configure function of the Startup class.

app.UseSpa(spa =>
           {
               spa.Options.SourcePath = "ClientApp";

               if (env.IsDevelopment())
               {
                   spa.UseReactDevelopmentServer(npmScript: "start");
               }
           });

As you can see from the highlighted area the Reac Development Server should only be used when the environment is set to development and sure enough, the web.config has environment variable for ASPNETCORE_ENVIRONMENT and the value is Development as you can see in the following example.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <location path="." inheritInChildApplications="false">
    <system.webServer>
      <handlers>
        <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
      </handlers>
      <aspNetCore processPath="dotnet" arguments="YourApplication.dll" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" hostingModel="InProcess">
        <environmentVariables>
          <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Development" />
        </environmentVariables>
      </aspNetCore>
    </system.webServer>
  </location>
</configuration>

A Fix

My knee jerk reaction was to find a way to get the value of that variable changed when the application is published in release mode. This helped me find the Microsoft docs for Transform web.config.

For this fix, I used a build configuration base transform. To do this I added a new file named web.Release.config. With this new file present, the transforms in the file will be executed when a release build is run. The following is the transform I used to get ASPNETCORE_ENVIRONMENT set to Production.

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <location>
    <system.webServer>
      <aspNetCore>
        <environmentVariables>
          <environmentVariable xdt:Transform="Replace" xdt:Locator="Match(name)" name="ASPNETCORE_ENVIRONMENT" value="Production" />
        </environmentVariables>
      </aspNetCore>
    </system.webServer>
  </location>
</configuration>

It is important that the structure of your transforms match what is in your actual web.config or the transform won’t be able to locate the element that needs to be transformed which in this case we are looking for an element under configuration/location/system.webServer/aspNetCore/enviromentVariables.

In our example, we are telling the transform we want to replace the element  (xdt:Transform=”Replace”) that matches the name (xdt:Locator=”Match(name)”) ASPNETCORE_ENVIRONMENT with the value Production.

Wrapping Up

This is just one small example of what can be done with web.config transforms. The official docs gave me a general high-level idea of what transforms can do, but wasn’t super helpful for what I needed to do. If the docs don’t cover your use case get ready to do a fair amount of searching.

Side note in the application I was referencing above it turns out that the web.config in the published for .NET Core 2.1 didn’t contain a value for ASPNETCORE_ENVIRONMENT which was why it wasn’t an issue before.