Azure DevOps Pipelines: Multiple Jobs in YAML
This post is going to show how to run multiple jobs out of a single YAML file from an Azure DevOps Pipeline. This post is going to build on the Azure DevOps project created in previous posts. If you are just joining this series check out the previous posts to find out how the project has progressed.
Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Starting Point and the Plan
As the sample stands now we have a single Pipeline that builds two different ASP.NET Core web applications in a single job using the following YAML.
trigger: - master pool: vmImage: 'ubuntu-latest' variables: buildConfiguration: 'Release' steps: - task: UseDotNet@2 inputs: packageType: 'sdk' version: '3.1.x' - script: dotnet build --configuration $(buildConfiguration) displayName: 'dotnet build $(buildConfiguration)' - task: DotNetCoreCLI@2 inputs: command: 'publish' publishWebProjects: true arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)' - task: PublishPipelineArtifact@1 inputs: targetPath: '$(Build.ArtifactStagingDirectory)' publishLocation: 'pipeline'
This post is going to take this pipeline and split the build and publish of the two web applications and make each application its own job. In Pipelines a job is something that a single agent takes and runs. By splitting into multiple jobs the pipeline can run multiple jobs at the same time if you have enough build agents available. One reason to do this would be to speed up the total Pipeline run if you have parts of your build that are independent. Another example of why you would need jobs is if the different jobs need different agents such as one needing a Windows agent and another a Linux agent.
Creating the Jobs
Having different jobs means we are going to have to move things like what agent pool to use and the steps for the job under a jobs element and then declare a specific job and the details that job needs to run. As you can see in the following example the end goal is the same as the YAML from above (except it is dealing with a specific project), but the details are nested under jobs and defined under a job.
trigger: - master variables: buildConfiguration: 'Release' jobs: - job: WebApp1 displayName: 'Build WebApp1' pool: vmImage: 'ubuntu-latest' steps: - task: UseDotNet@2 displayName: 'Use .NET 3.1.x' inputs: packageType: 'sdk' version: '3.1.x' - task: DotNetCoreCLI@2 displayName: 'Build' inputs: command: 'build' projects: '**/WebApp1.csproj' arguments: '--configuration $(buildConfiguration)' - task: DotNetCoreCLI@2 displayName: 'Publish Application' inputs: command: 'publish' publishWebProjects: false projects: '**/WebApp1.csproj' arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)' - task: PublishPipelineArtifact@1 displayName: 'Publish Artifacts' inputs: targetPath: '$(Build.ArtifactStagingDirectory)' artifact: 'WebApp1' publishLocation: 'pipeline'
Also notice that you can still define variables that can be used across jobs as is done above with the buildConfiguration variable. The following is the full YAML file that builds and publishes the artifacts for both web applications.
trigger: - master variables: buildConfiguration: 'Release' jobs: - job: WebApp1 displayName: 'Build WebApp1' pool: vmImage: 'ubuntu-latest' steps: - task: UseDotNet@2 displayName: 'Use .NET 3.1.x' inputs: packageType: 'sdk' version: '3.1.x' - task: DotNetCoreCLI@2 displayName: 'Build' inputs: command: 'build' projects: '**/WebApp1.csproj' arguments: '--configuration $(buildConfiguration)' - task: DotNetCoreCLI@2 displayName: 'Publish Application' inputs: command: 'publish' publishWebProjects: false projects: '**/WebApp1.csproj' arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)' - task: PublishPipelineArtifact@1 displayName: 'Publish Artifacts' inputs: targetPath: '$(Build.ArtifactStagingDirectory)' artifact: 'WebApp1' publishLocation: 'pipeline' - job: WebApp2 displayName: 'Build WebApp2' pool: vmImage: 'ubuntu-latest' steps: - task: UseDotNet@2 displayName: 'Use .NET 3.1.x' inputs: packageType: 'sdk' version: '3.1.x' - task: DotNetCoreCLI@2 displayName: 'Build' inputs: command: 'build' projects: '**/WebApp2.csproj' arguments: '--configuration $(buildConfiguration)' - task: DotNetCoreCLI@2 displayName: 'Publish Application' inputs: command: 'publish' publishWebProjects: false projects: '**/WebApp2.csproj' arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)' - task: PublishPipelineArtifact@1 displayName: 'Publish Artifacts' inputs: targetPath: '$(Build.ArtifactStagingDirectory)' artifact: 'WebApp2' publishLocation: 'pipeline'
After all your edits are done commit the changes to your YAML file and then run the pipeline. As you can see from the following screenshot of my sample pipeline run the pipeline has two jobs instead of one that the original YAML resulted in. Also, note that the pipeline results in two published artifacts (one per job in our case) instead of the one with the original.
Wrapping Up
As mentioned above there are a lot of reasons you might want to split up your pipeline into multiple jobs and hopefully, you now have a good idea of how that is done. Make sure and check back in the future for a post on how to take repeated tasks and make them reusable.
Azure DevOps Pipelines: Multiple Jobs in YAML Read More »