So far we’ve look at creating and publishing packages using duget. This is of course an essential part of the process, but the real power comes when using those packages in your projects and ensuring that any dependencies you take also bring in their required dependencies and so on.
This last post in this short series shows how that currently works in duget.
As with the previous post, this was written before Liev arrived. 🙂
Which brings us to the final file of interest: the project configuration file. This identifies the packages that a particular project takes dependencies on. Every project has potentially two such files that determine the dependencies for that project. The first is the path configuration. This is a .duget file in the same folder as the project. The second is a <<em>project</em>>.duget file, sharing the same filename as the dpr but with the .duget extension.
In complex projects with multiple dprs involved having common dependencies, this allows those dependencies to be specified just once at the path level, with only project specific dependencies then added if/as needed in individual project files. If no project configuration is provided then the path configuration applies and vice versa.
Project and Path Configuration files provide facilities to identify dependencies that are specific to particular build configurations or platforms. That is, your debug build configuration may take additional dependencies, e.g. on debugging tools/frameworks, that your release builds do not.
Project Group configurations are specifically not supported. This is because of the complication that a single project may conceivably exist in multiple project groups leading to the possibility that the same project could have different dependencies depending on the project group context. This simply does not make sense for project dependencies.
Build Config and Platform specific dependencies are already supported by duget but is the area perhaps most likely to change as part of the evolution of the configuration files so I won’t go into the current implementation. Instead I’ll look at a simple scenario where a project simply takes dependencies that apply to all platforms and all build configurations:
{
references:[
"&..\\.duspec",
"deltics.smoketest:2.0.2"
]
}This example is taken from the configuration for the tests project of another library that I am currently packaging. This project implements the tests that are run to verify that a library may be packaged, only if it passes these tests.
For this reason, and intended only for use in such scenarios, this configuration includes a dependency on a package specification. This is denoted by the & prefix on the reference, indicating that what follows is a path to a .duspec file, rather than a package. This tells duget that this project (the test project) directly references the source for that package, as identified in the package specification.
The second dependency is more straightforward. Since the tests project uses Smoketest 2.0 to implement the tests, it takes a dependency on that required package.
The RESTORE Command
To make sure that dependencies are available to a project for compilation, you simply use the restore command. Again, the simplest version of this command will look for all .duget files in the current folder:
duget restore
It actually looks for dpr files and then for applicable .duget files at the path and project configuration level to be applied to each dpr.
For each project duget restore will “walk the dependency tree”, loading packages from the cache where possible, fetching them from feeds where needed and then repeating that process for any dependencies that those packages bring in. It will also bring in any package dependencies identified in the package specification for any source references (just referencing the source itself isn’t enough).
If any dependency cannot be satisfied from the cache or any feed then the restore process fails.
The restore command will not attempt to update the version of any package to any more current version. Whatever version is specified is the version that will be provided. If multiple versions of a given package are referenced in the dependency tree for a given project then the latest compatible version will be used. If multiple incompatible versions are referenced then the restore command fails. “Compatibility” of versions is determined using Semantic Versioning rules.
Just having the required packages in the cache isn’t enough of course. As well as that, the restore command will unpack the source files from each package into a specially created folder under the projects. Each package is unpacked into a folder named for the package id and version of the package involved. So for two projects in the same folder referencing the same dependency, they share the unpacked folder. e.g. the unpacked folder for deltics.smoketest:2.0.2 would be:
<projectfolder>
\__duget
\deltics.smoketest-2.0.2
If two separate projects in the same folder reference different versions of the same package, then both will be unpacked but each project will reference only it’s required version. e.g. for two projects referencing different versions of deltics.smoketest:
<projectfolder>
\__duget
\deltics.smoketest-2.0.1
\deltics.smoketest-2.0.2
But the restore command doesn’t stop there. It also updates the -I, -R and -U settings of any <project>.cfg file it finds with the paths to the required package folders in the __duget folder, for that project.
It also updates the search path in any dproj file.
The search path added to either cfg or dproj files also includes the path to the source files for any source referenced package. Source file references (those & prefixed .duspec references) are not copied to the __duget folder but have their actual source locations added to the search path(s).
Any non-duget paths present in the cfg or dproj settings already are preserved as-is.
The UPDATE Command
The last command to mention for now is duget update. Since the restore command does not update package versions, the update command takes care of this. Separating the two is important as you would want a duget restore in a build process to reliably restore the same packages used for all previous builds and the same versions on the dev machines (unless and until someone commits a change to update those dependencies).
duget update is a mechanism to simplify that business of updating dependencies on dev machines. You would never run update on a build machine.
duget update also provides an additional level of forgiveness in your dependency configurations. If you don’t know the current version of a package that you wish to use, but you do know the package id, then you can add a reference to just that package id and duget update will identify the latest version for you. restore requires that a version is identified (except for & source references, obviously).
Again, the simplest possible invocation of the update command is:
duget update
And as with other commands, if you only wish to update a specific project (or projects) then specific projects may be identified as arguments. Similarly if you only wish to update the version of a specific package (or packages) then this can be specified in a —package option, e.g.:
duget update myproject —package:some.package;other.package
So if you’re starting a new test project and want to use the latest version of smoketest and/or other packages but don’t know what the latest versions are, you could create a dependency file similar to:
{
references:[
"deltics.smoketest",
“fastmm”
]
}Run duget update over that, and watch as the dependencies are updated to the latest versions and your project automatically configured to use them! You could also specify a version of course, and this may be useful if you specifically don’t want the absolute latest but do want the latest compatible version of some earlier major version.
The only problem arises if a version is specified that doesn’t exist and there are not any versions later than that one available, in which case you’re left with an invalid version in the dependency file. So if you don’t know, you’re better off leaving it out.
By default duget update will only update to a later, compatible release version of a package. If you are willing to accept pre-release packages or breaking changes then as long as you have a feed configured which provides such releases, you can force duget update to take these with the addition of options on the command line:
duget update --acceptPreRelease duget update --acceptBreaking duget update --acceptPreRelease --acceptBreaking
Whatever your appetite for change when updating, make sure you include a duget restore in your automated builds and you are good to go!
What Next
The duget system is being exercised extensively in my current efforts to package my own code (and itself) for consumption using it but I will soon be looking for beta testers. If anyone would like to help with IDE tooling I’d be particularly interested to hear from you.
I am also starting to look at extending the coverage of this tool to FPC and Elements compiler technologies.
Developers using Elements for Android/Java or .net projects of course have the vast wealth of packages served up by the existing package ecosystems for those platforms, but Elements also supports native code development and a package management solution could have a place there.
If you’re interested to get involved either as a tester or even as a contributor, let me know in the comments.