If Mark Twain was hired to write error messages for the Azure DevOps team, he might have deployed his own, suitably paraphrased quote, completing it with… ‘Perhaps you should take some time to make it shorter ?‘
I’ve been wrestling with a problem today with my Delphi build template that is a classic case of misdirection and inadequate error handling (or rather, unhelpful, unadorned error messages being passed to the end-user without further explanation or suggestion).
A bit of background…
As noted before, much of the Delphi work I have done (and am returning to) involves supporting many different versions of Delphi, from Delphi 7 to current. As a result, my CI/CD pipelines need to build code with all of those Delphi versions and (increasingly) run tests to exercise it.
So I was making changes to my build template to enable me to pass in an array of Delphi versions and have it perform builds for all those versions.
This comes with some downsides, which we’ll perhaps come back to. But my initial problem was how to achieve it.
I first decided to try Iterative Insertion, as this seemed to be suited to the job. That is, I wanted to create a further template which would iteratively insert into itself the steps needed to in-turn invoke my build template for each Delphi version that I specified.
My goal was to be able replace a list of over a dozen individual build steps with just one. To give you some idea, consider a build involving just 3 Delphi versions (let’s say 7, 2007 and 10.3 Rio for example):
- template: delphi-build.yml@templates
parameters:
delphiVersion: 7
project: tests\tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\tests -f=.results\delphi.7.xml
- template: delphi-build.yml@templates
parameters:
delphiVersion: 2007
project: tests\tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\tests -f=.results\delphi.2007.xml
- template: delphi-build.yml@templates
parameters:
delphiVersion: 10.3
project: tests\tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\tests -f=.results\delphi.10.3.xml
- etcThe goal was to be able to achieve the same thing with just this one step:
- template: delphi-builds.yml@templates
parameters:
delphiVersion: 7 2007 10.3
project: tests\tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\tests -f=.results\delphi.$($delphiVersion).xmlNeat, huh ?
Long story short, I could not figure out how to do this with Iterative Insertion. The ‘error messages’ from the YAML processor were less than helpful and I couldn’t find anyone doing anything similar using this approach.
So I tried coming at it from a different angle.
Instead of trying to dynamically generate a template to call another template, I used the dynamic nature of Powershell itself. After an initial struggle, trying to figure out how to pass an array of Delphi versions as a parameter to a template and then have that picked up as an array of strings by Powershell, I realised that it was far simpler than that.
The simple string syntax already gave me everything I wanted and needed. Since YAML does not require string values to be quoted, simply passing a space-separated list of values did the job and was actually cleaner anyway.
This meant that my target syntax could actually be supported directly by my existing template:
- template: delphi-build.yml@templates
parameters:
delphiVersion: 7 2007 10.3
project: tests\tests
preBuildInline: duget restore --noUnpack
postBuildInline: .bin\tests -f=.results\delphi.$($delphiVersion).xmlAll I needed to do was refactor the Powershell script in that template to create a function to perform the build with the script itself becoming a processor which turned the delphiVersion parameter into an array and then called that build function iteratively for each version in that array (i.e. each version space separated in the parameter).
Neat.
A few short moments later I had an almost working script. The only problem I had was that I forgot to deal with one important side effect of now calling this script effectively multiple times.
The script set the working location for the build to the folder containing the project being compiled. When building for multiple Delphi versions this worked fine for the very first version compiled and then failed miserably for all subsequent compilations because the subsequent invocations of the build function were starting in that location rather than the root of the repo.
No problem, one of the great things about Powershell is that it features many of the capabilities that we’re used to in other programming languages. In this case the familiar try/finally was the tool I reached for.
Before calling Set-Location to change the current directory, simply stash the result of Get-Location, wrap the important parts of the build process in a try / finally and call Set-Location again in the finally to restore the previous location.
What could be simpler ?
This simple change completely broke my script. And I mean: Completely.
The pipeline started complaining that my try<code> block was missing a catch or finally (it obviously wasn’t). I tidied up some of my formatting inconsistencies, just on the off-chance that there was something in there throwing things off with my bracketing (though there were no complaints on that score).
Ooops. Spoke too soon. Although I had not introduced or removed any bracketing in my tidy ups, suddenly my pipeline started complaining that my {}‘s were mis-matched.
After painstakingly re-assuring myself that this wasn’t the case (it wasn’t) I turned my suspicious gaze on YAML whitespace indentation – this had caught me out before.
All in vain.
But I had undertaken more tidying up along the way and now Azure DevOps itself was complaining. Up to now the errors had occurred when the build job reached my step on the build machine and executed the Powershell. Now Azure DevOps was point-blank refusing to even start a run. Now there was some issue right at the top, line 17 to be precise, the very beginning of the script step. And now the problem was something about not being able to ‘convert from an array to a string‘.
I was clearly getting nowhere trying to fix the script as it stood. In fact, going backwards!
Not being able to make sense of the errors, I decided to re-construct the entire script piece by piece. Stripping it back to nothing then adding chunks of functionality a few lines at a time until it broke. The idea was that this would help me identify where exactly in the script the problem was and so hopefully make it easier to spot.
This started well, with my script instantly back to glowing health, even if not doing very much. In fact, I got all the way to the final dozen or so lines of code before it broke. These final few lines were simple stuff so figuring out where the problem was should be simple.
The bad news: Yes, these lines were indeed very simple. There was clearly and self-evidently nothing wrong with them at all.
Yet, without those 12 lines my script compiled and ran just fine. With them, it would not even run.
On a nervous hunch, I added them back in, but commented them out.
And still the pipeline would not run.
Conclusion: The problem was not caused by anything the code was doing – commented out code doesn’t do anything. The problem was simply the code being there at all!
There was one thing that might explain both the errors I was seeing before and this new discovery: An upper limit on the length of a script that can be included in an Azure Pipeline step!
And sure enough, there appears to be such a limit.
The different error messages were presumably the result of this limit being run into at different stages in the process.
If the “raw” script itself was too big, then the problem of not being able to convert an array to a string presumably comes from the pipeline trying to parse the template itself.
But if the raw script was just within the limit, the problem might then have been caused when substitutions and insertions were applied (e.g. the preBuildInline and postBuildInline script fragments that my consuming pipeline inserted). This may have pushed the script over the bounds of whatever buffer was being used to prep it. This truncated buffer was then being passed on to the build agent where Powershell itself would then choke on the syntax errors caused by that truncation. Those errors would vary according to where the truncation had occurred, and none of them would make sense when considered in the context of the whole script.
It all started to make sense, but why oh why could there not have been a straightforward error message to tell me that this length limit had been exceeded!? It would have saved me hours!
From subsequent digging through forums and the like, it seems that the limit is 5,000 characters. This seems more or less consistent with my own observations. It’s difficult to be very precise as the use of substitutions and insertions in a template seem to be a contributing factor. i.e. it’s not only a question of how many characters there are in the script source in a template or step, but the total characters after any substitutions have taken place.
Be that as it may, all I have to do now is figure out how to scrunch this script down without making it impenetrable and retaining the functionality I want and need.
Let this be a lesson in the utility of helpful error messages (and the unnecessary pain that can be caused by their omission).
Footnote: I mentioned that building with multiple versions of Delphi in a single step has a downside. Well, it is this: Since all of the compilations with the various Delphi versions specified are performed in a single step, the resulting output from the build pipeline can only tell me that Delphi compilation failed. I then need to scour through the logs of that one step to figure out which Delphi versions failed. Iterative Insertion would avoid that by creating separate steps for each Delphi version, so it was a real shame that I couldn’t get that to work. So for now, I’m going to keep the multiple Delphi version support for some perhaps limited occasions when it may be useful, but use a different trick to keep my main pipelines trimmed to a minimum.