In a previous post I demonstrated how the default “pretty name” for a Smoketest test case (derived from the test case classname) can be over-ridden by a test developer by implementing a specific interface (INameCase) on the test case class itself. There are some other interfaces that can be implemented on a test case, including interfaces that allow a test case to implement housekeeping tasks for the tests it provides.
DUnit uses virtual methods to provide support for setting up and tearing down test cases. This is all well and good, but since they have to be visible to the test case class in order to be over-ridden it means that those methods “pollute” the class. For example, they appear in code completion suggestions for all test cases, even where a test case has no interest in implementing overrides for them.
To minimise this “pollution”, Smoketest uses interfaces.
For example, if a test case wishes to instantiate some object to be shared by all tests in that test case, then the test developer declares that the test case class implements the ISetupTestCase interface.
The ISetupTestCase interface comprises a single method: procedure SetupTestCase;
If the test case also wishes to do cleanup after the test case has been performed it must separately implement the ICleanupTestCase method.
There are interfaces and corresponding methods for performing setup and cleanup operations at various points in the lifecycle of a test project. First the setup interfaces and methods:
| Interface | Method | When The Method Is Called |
|---|---|---|
| ISetupProject | SetupProject | After the test project has been initialised but before any tests are run (only called once) |
| ISetupTestRun | SetupTestRun | Immediately before any test run that includes the test case (called once per test run) |
| ISetupTestCase | Setup | Immediately before the test case itself is performed (called once per test run) |
| ISetupTest | SetupTest(aTest: ITestMethod) | Immediately before each test method in the test case is performed (called once per method per test run) |
And the corresponding cleanup interfaces and methods:
| Interface | Method | When The Method Is Called |
|---|---|---|
| ICleanupProject | CleanupProject | When the test project is shutting down whether or not any tests were run (only called once) |
| ICleanupTestRun | CleanupTestRun | Immediately after any test run that included the test case (called once per test run) |
| ICleanupTestCase | Cleanup | Immediately after the test case itself has been performed (called once per test run) |
| ICleanupTest | CleanupTest(aTest: ITestMethod) | Immediately after each test method in the test case has been performed (called once per method per test run) |
To see an example of these you can look at the test case for JSON recently added to the RTL test project. The implementation of the interfaces is declared as normal on the test case class along with the corresponding methods:
TUnitTest_JSON = class(TTestCase, INameCase,
ISetupTestCase,
ICleanupTestCase,
ICleanupTest)
private
fJSON: TJSONObject;
private
function NameForCase: UnicodeString;
procedure Setup;
procedure Cleanup;
procedure CleanupTest(const aTest: ITestMethod);
In addition to one setup method, the JSON test case implements two cleanup methods and also provides a specific name for the test case.
The setup and cleanup methods all provide housekeeping for the fJSON member.
The Setup method (from ISetupTestCase) creates an instance of TJSONObject to which the fJSON member is a reference.
The Cleanup method (from ICleanupTestCase) destroys the TJSONObject and resets fJSON to NIL.
This fJSON member is used by various test methods in the test case which all need to start with a clean slate, so the CleanupTest method (from ICleanupTest) ensures that the JSON object is cleared after each test method (in this case you could equally do this initialisation in a SetupTest method before each test).
{ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - }
function TUnitTest_JSON.NameForCase: UnicodeString;
begin
result := 'Deltics.JSON';
end;
{ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - }
procedure TUnitTest_JSON.Setup;
begin
fJSON := TJSONObject.Create;
end;
{ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - }
procedure TUnitTest_JSON.Cleanup;
begin
FreeAndNIL(fJSON);
end;
{ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - }
procedure TUnitTest_JSON.CleanupTest(const aTest: ITestMethod);
begin
fJSON.Clear;
end;
The ITestMethod reference passed to the CleanupTest method (and SetupTest, where implemented) can be used to perform specific housekeep for certain test methods by testing the Name of the ITestMethod in each invocation if required.
So far all of the Smoketest examples have consisted of very simple lists of of test methods organised into test cases.
In my next Smoketest post I shall look at how tests can be organised in more sophisticated relationships, establishing a hierarchy of test cases and dynamically creating test cases.
I still does not understand why you use interfaces here…
Some virtual methods, with a void begin..end body for the ancestor, is just fine, faster, and more readable.
Interface are mostly about abstraction, not about object behavior description, unless necessary.
Mostly to remove them from the code completion suggestions when writing tests that do not implement setup/cleanup methods. It cannot be avoided entirely since when you do implement the interfaces then the necessary methods become part of the test case, obviously. But at least you don’t have them cluttering up the list when you are specifically not implementing them.
And also simply because to my mind it is the cleaner approach. 🙂
If all test cases were required to implement setup and cleanup methods than having a virtual abstract method on the base class is appropriate. But setup/cleanup is not an intrinsic behaviour of all test cases but an entirely optional one. Some test cases do it, others don’t.
Using void virtual methods is actually a very crude and clumsy way of implementing this sort of ‘opt-in’ behaviour and the difference in speed is not significant either in terms of the magnitude or the relevance in this case.
It also means that you have to inspect the class method list to discover whether the class implements a specific framework capability. With interfaces this becomes declarative, though the method declaration of course is still required it is not the only indicator that you can/have to look for.
And I don’t think your “rule” about what interfaces are for is actually true. Interfaces absolutely can be used to describe behaviour since they represent a contract independent of any other OO or abstractions that may be involved. This isn’t something to be done only when “necessary”, but rather when “appropriate”. 🙂
About performance, we both agree. It should make a very small difference only at class allocation. Execution should be almost the same in most applications, unless the LOCKed reference count of the interface may slow down multi-threaded process. But since you define parameters e.g. like “(const aTest: ITestMethod)”, it should be fine (AFAIK “const” bypass interface reference counting). Since your test case class is just a container to internal process, not worth measuring. But if all your classes (including the short-living ones) use such interface-based duck-typing pattern, you would start to notify a performance hit.
You are right: it is a matter of taste.
If you think it is “appropriate”, it is does make sense to use it.
But if you want, say, to add a setup/cleanup method to your test case class, you can use code completion to locate the ancestor method in two cases, then copy the method signature to your class, and implement it. No need to remember the interface name. Matter of taste, and habit.
BTW in mORMot we relied on “un-camelcasing” the test case method name instead of providing an interface to retrieve a name. We believe much more in “coding-by-convention” instead of “coding-by-behavior”, as you like. But it is a matter of test…. sorry, a matter of taste!
🙂
Multi-threading isn’t an issue in this case since these interfaces are part of the framework code. They run either in the main thread during project initialisation or in the test run thread during execution of the tests themselves, but there is no thread-contention on these interfaces.
And yes, “const” suppresses some ref counting on interfaces (and strings and other reference types), which is why I habitually use it except where I specifically do not want that behaviour. 🙂
There aren’t any “short-lived” classes in the test framework that lie on the test execution path.
There are some used in the GUI console where the GUI requests information from the test framework which is packaged up inside a lightweight class and passed back by interface reference so that the GUI can extract the necessary information and then let the instance of that particular packet of data die (because the data it carried is potentially already out of date, so if I ask for the same data again I get a new short-lived class to carry the current data).
Using code completion to identify inherited methods to be over-ridden is one way, but those methods are hidden among a list of all the methods inherited by that class. There are other reasons I preferred not to use this approach which I explain in my answer to Eric’s most recent comment so won’t repeat here.
But seriously, anyone offering this as a supposed advantage of this approach (as per DUnit) I suspect does not actually use it very often for the purpose they recommend it. Finding the Setup and Teardown methods in the list of offered methods and properties of a DUnit test case is like the proverbial needle in a haystack. Apart from anything else, you have to already know the name you are looking for in order to find them deeply buried down the long, LONG list of alphabetised names.
Add a framework interface to a test case in Smoketest and the method(s) that interface requires will be top of the list. And a very much shorter list at that. 🙂
>It also means that you have to inspect the class method list to discover
>whether the class implements a specific framework capability.
But in that case, the class declaration becomes an obvious and unambiguous documentation of the features that are supported.
When features are implemented through interfaces, there is a need for either a textual list of which interfaces are supported by what, or a need to looking at the source code implementation to see which interfaces are supported (and how they are named).
For instance if you don’t know how Setup/TearDown is implemented in a class-oriented test framework, looking at the base test class will tell you that.
With interfaces, you will need to find the ISetupTestCase & ITeardownTestCase, which… oops! doesn’t exist because it’s named ICleanupTestCase… which won’t be easily discoverable if you’re looking for the classic “teardown” terminology.
If your Cleanup was a virtual method, discovering it would be trivial.
And even if you find some interface, the only thing that can tell you if it’s really an interface for a Test Case will be the postfix of the interface name or the unit where it’s defined, and even then, it could just as well be an interface intended for features at the the individual test case / check level or at the test case suite level, or maybe it’s just completely unrelated and some internal framework cog…
So while interfaces introduce flexibility, they introduce complexity at the usability level. They also transform a statically typed, compile-time checked problem into a runtime one, as the compiler will happily accept you implementing any interface.
(side note, if the interfaces were prefixed rather than postfixed, they would be slightly more discoverable, ie. ITestCaseSetup & ITestCaseCleanup are more suggestion friendly, since if you type the beginning, the IDE can suggest all the related interfaces, this btw, is also why in an era of IDEs, Hungarian notation is a bad notation)
Not if the test involves a test method that happens to also be called SetupProject or ProjectSetup or something similar.
Your discovery argument is utter nonsense and is predicated a) on a complete absence of documentation and b) that every framework follows some notional “standard” nomenclature (defined by whom ?).
It also ignores the fact that the IDE will offer any incomplete interface methods for completion, so there is no need to lookup the documentation for the interface method prototype, just add the interface to the class and the IDE will give you the help you so desperately need.
You draw out a very long bow of complaints about the implications of using interfaces w.r.t discovery, and all of those complaints apply equally to virtual methods when starting from the same point – of not knowing. The only difference is that if you assume that those virtual methods follow a pattern arbitrarily established and which you now prescribe should be adhered to only because you are already familiar with it, then you, personally, would not have to follow the discovery process.
Anybody else will just use the documentation.
As for the naming of the interfaces, this deliberately reflects using them as a description of a behaviour:
TMyTestClass = class (TTestCase, (oh and by the way) I Setup (this) Test Case, I Name (this) Case, I Cleanup (the) Project, I Fail Early) // etcThat last one is a new one by the way, which doesn’t even have a method to implement. The presence of the interface simply identifies a behaviour that is implemented on it’s behalf by the framework. You no doubt will have some deeply ingrained philosophical objection to that also, but I would point out that it is no different from decorating a class with an ‘attribute’ (which, being thoroughly ‘modern’, I am sure you would whole-heartedly endorse) but meets the requirement of being supported in older versions of Delphi where attributes are not supported.
Using interface also makes that sure intentions are stated but also that the obligations of that intention are then carried through.
When implementing a simple override of a Setup method, it is easy to overlook the (potential) need to implement a Cleanup method. When there is no method to override you are reminded that you have to make your intentions clear and having stated that intention you are then obligated to complete the contract represented by that intention. The compiler will make sure that you do.