In yesterday’s post I mentioned that I had been unsuccessful in using Iterative Insertion to simplify building for multiple Delphi versions. Well, as so often happens, I had a flash of insight overnight and found the solution this morning!
In the foot note of that previous post, I mentioned having used a different trick to preserve per-Delphi version specific build steps in my build jobs (to ease the process of identifying build failures in that eventuality).
The ‘trick’ was to create a local template for my builds. This template is specific to my project build needs so doesn’t form part of the azure-pipeline-templates library but sits in my repo alongside my main build pipeline:
parameters:
delphiVersion: ''
platform: 'x86'
steps:
- template: delphi-build.yml@templates
parameters:
delphiVersion: ${{ parameters.delphiVersion }}
platform: ${{ parameters.platform }}
project: tests\Tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\Tests -f=.results\Delphi.${{ parameters.delphiVersion }}.xml
This template in turn references the main delphi-build.yml template in GitHub. It provides most of the parameters I need that do not vary regardless of the different Delphi versions I am building for. It achieves variability in parameter values where needed by using template substitution.
This is most obvious in the delphiVersion parameter, which is passed through from this template to the delphi-build template. But it is also used to ensure that test results produced after a successful build are named for that delphiVersion also.
This simple template already would allow me to greatly simplify my main build.yml for this project:
trigger:
- develop
- master
resources:
repositories:
- repository: templates
type: github
name: deltics/azure-pipeline-templates
ref: refs/heads/develop
endpoint: GitHubTemplates
pool:
name: 'The Den'
steps:
- template: gitversion.yml@templates
- script: |
duget init
duget restore --path tests
displayName: 'Initialise DuGet'
# x86 builds ---------------------------------
- template: build-with-version.yml
parameters:
delphiVersion: 7
- template: build-with-version.yml
parameters:
delphiVersion: 2010
- template: build-with-version.yml
parameters:
delphiVersion: xe4
- template: build-with-version.yml
parameters:
delphiVersion: 10.3
# x64 builds ---------------------------------
- template: build-with-version.yml
parameters:
delphiVersion: xe4
platform: x64
- template: build-with-version.yml
parameters:
delphiVersion: 10.3
platform: x64Still pretty verbose, but a vast improvement on the original and much easier to maintain.
You may note that my reference to the azure-pipeline-templates GitHub repository in this pipeline includes a ref property which results in this pipeline using the latest version from the develop branch, as I am developing this pipeline alongside making refinements to that GitHub template. Once finalised (i.e. develop merged to master), this ref will be removed from this pipeline.
x86 and x64 builds still have to be specified separately as there is no assumption by the downstream templates that I wish to build for both architectures even if the specified Delphi compiler version supports x64.
The next step is to introduce a further template which can ‘stamp out’ these individual build steps for me.
It turns out that the required template is itself very simply. I had made two mistakes yesterday when first attempting this. The first was not double-checking my (still developing) knowledge of YAML and ensuring that my array syntax was correct. The second was over-looking a trivial but significant element in my iterative insertion syntax.
In my defence, the errors these mistakes resulted in (e.g. mapping not allowed here) were again not especially helpful.
Here’s the template I ended up with:
parameters:
delphiVersions: []
platform: 'x86'
steps:
- ${{ each version in parameters.delphiVersions }}:
- template: build-with-version.yml
parameters:
delphiVersion: ${{ version }}
platform: ${{ parameters.platform }}This template assumes that I am building for x86 but allows me to specify some other platform in a parameter. It also expects that I will provide an array of delphiVersions that I wish to build.
It then uses the iterative insertion feature of Azure DevOps pipelines to add a step for each of those delphiVersions. Each of these steps uses the build-with-version.yml local template.
I could have ‘bypassed’ the build-with-version template and instead directly referenced the delphi-build.yml template, but this would only make sense if I were to then git rid of the build-with-version.yml entirely, otherwise I would have to keep both templates in sync if I ever needed to change the build properties for this project. This way, I know that build-with-version.yml is the one place where my project build is essentially configured.
With that done, here’s what my final build pipeline looks like for this project:
trigger:
- develop
- master
resources:
repositories:
- repository: templates
type: github
name: deltics/azure-pipeline-templates
ref: refs/heads/develop
endpoint: GitHubTemplates
pool:
name: 'The Den'
steps:
- template: gitversion.yml@templates
- script: |
duget init
duget restore --path tests
displayName: 'Initialise DuGet'
- template: build-all.yml
parameters:
delphiVersions: [ 7, 2006, 2007, 2009, 2010, xe, xe2, xe3, xe4, xe5, xe6, xe7, xe8, 10, 10.1, 10.2, 10.3 ]
- template: build-all.yml
parameters:
delphiVersions: [ xe2, xe3, xe4, xe5, xe6, xe7, xe8, 10, 10.1, 10.2, 10.3 ]
platform: x64
I still need to specify x86 and x64 builds separately but can now specify all Delphi versions for each platform in a single step.
You may be wondering why the <strong>resources</strong> section is not also needed in the build-with-version template. The answer is that resources can only be declared in the initial pipeline, and not in a template. i.e. a pipeline must declare up-front any resources that any referenced templates need and that’s still the case even if the pipeline itself does not directly reference those resources itself!
When the build actually runs, those two steps are transformed into multiple, individual build steps, one for each Delphi version. Each step is helpfully named in the build logs for the Delphi version and target platform, so if a change breaks this project only for specific versions of Delphi, I can see at a glance which version is affected!
As you can see, currently the build will abort at the first failure with any given Delphi version; subsequent build steps are not performed, unless they have a condition applied to force them.
I started looking at how to add such a condition and things quickly become quite messy. In particular, Delphi builds should only be forced if any current failure was a previously failed Delphi build. If there was some other, earlier failure in the build pipeline then I would not wish to force in that case.
A tricky problem and one I may return to at some point in the future. But not today.