A couple of commenters on my previous post have taken issue with my use of interfaces to form contracts between test cases and the test framework, rather than using simple virtual methods and inheritance as found in DUnit. I thought it would be interesting to illustrate why I went down this route.
It is suggested that using inheritance means that discovering the methods required to be implemented to support Setup/TearDown in DUnit is simpler than having to know the required interface name and then having to lookup the method signature involved, as in the case of Smoketest.
This of course assumes that documentation either does not exist or is being wilfully ignored.
First of all, the interfaces in Smoketest are designed to be highly memorable. They read as a natural description of a behaviour that a test case class follows.
Where a test case has some project setup to perform it states “I Setup (the) Project“. If it has some setup to perform specific to the test case “I Setup (this) Test Case“. And so on.
So remembering the interface name really shouldn’t be hard, since it describes what you wish to achieve. Whether you intuit the interface name, select it from the fairly short list of interfaces offered by IDE Code Completion, the chances are you will only have to look it up once.
The compiler then ensures that the contract is fulfilled.
So let’s assume you at least know the interface involved and look at the impact that using the interface has on finding the method(s) required, as compared to finding the method from the list of inherited methods.
First, let’s look at what the IDE offers when we invoke Code Completion in the class declaration of a DUnit test case.
I made my suggestion list popup as big as I possibly could, but they aren’t on even the first page of this greatly enlarged list so scrolling down to the next complete page of suggestions ….
Aha – there they are. Setup and Teardown. Look carefully, you will see them eventually.
Of course, to find them you have to already know what you are looking for since they are simply listed alphabetically among all the other methods that you have no intention of implementing yourself.
I don’t see any methods for performing per-test or project-wide setup/teardown though, so I am not entirely sure of the context in which Setup and Teardown operate. I guess I’ll have to read the documentation after all. 🙂
Now let’s look at the same Code Completion offering for a Smoketest test case class declaration:
Well, would you look at that. The methods introduced by my specified interface contracts (in this case, that I will provide a name for the case and that I perform some test case setup), are put front and centre.
Now, without the interfaces declared on the class, I would see no such methods at all. But is this really a problem ? I don’t think so. The list of available methods is itself much shorter and the complete lack of any setup/cleanup methods will itself remind me that I have to form my contract with the framework first.
And I then get to choose which contract – setting up specific test methods, the test case or project wide setup (and/or attendant cleanup).
On that note, issue has also been taken with the use of “Cleanup” rather than “TearDown” in the method and interface names. This was deliberate. TearDown has a finality about it that seemed to be most often not appropriate.
Take for example the JSON test case which performs some cleanup after every test method by clearing a JSON object in preparation for the next test method. This really isn’t “tearing anything down”. It really is just “cleaning up”.
Sometimes your cleanup will involve completely destroying some artefacts used/created during testing, but this is just one extreme form of cleanup.
When the house-work needs doing, do you tear down your house or do you clean it up ?
Even tearing down the house, if/when it ever becomes necessary, is only what you do in order to “clean” the site, in readiness for a new house to be built.
TearDown may appeal to some people’s sense of the dramatic, but other than that it is actually a very poor choice of name imho.
I didn’t create Smoketest in order to simply create a plug-compatible alternative to DUnit. I didn’t even create it as a “competitor” for DUnit. I created it for myself precisely because I didn’t and don’t like the way DUnit works.
And because DUnit didn’t and doesn’t do some things that I wanted and needed (see my next post).
I simply choose now to share it.
If you do like the way DUnit works you are of course more than welcome to continue using it, but don’t expect Smoketest to conform to any expectations you might bring from that other framework. 🙂
It may be OK if you have only one or two classes, with each one one or two interfaces to remember. It may be OK if you want to write one or two blog articles, for a
But sooner or later, if you code style is using this pattern everywhere, you won’t be able to remember which interface is supported by which class. Or you would use the class name within the interface name (like ISmokeTestSetup), or use name spaces (SmokeTestUnit.ISetup)…
Here you have just a few interfaces, but your class may become as feature-rich as DUnit. And then, you won’t be able to remember your interface name. You will have to browse the code, find out which interfaces are possible for which class, then add the IBehaviorName to the top of the class definition, then go to the interface definition and copy the expected method signature, go back to the class protected section then paste the method signature, then implement it…
I’m pretty skeptical about the scalability and usability of such a pattern.
In good OOP – the one I want to /should do, not the one I’m doing:) – you should better create small classes, with a small number of methods per classes. Each class should have one scope and only one. Then you would aggregate your classes into bigger classes. As a result, code completion popup menu won’t be polluted, and you will be able to find out which behavior to implement. Then create class instances fields, so that your nested small classes will be created with the expected class type – e.g. fMyBehaviorInstance := fMyBehaviorClass.Create()
Smoketest is already more feature rich than DUnit in some respects and yet the number of interfaces required to be remembered for decoration on a test case class is still just a handful. How many classes do you expect to have to invoke in a testing framework ? There is TTestCase and – in Smoketest – TPerformanceCase. That’s it.
I am not advocating this as a general approach solution for all types of application or project, only as an appropriate mechanism for a framework such as Smoketest where the number of classes you work with directly can be counted on the fingers of both your thumbs. 😉
I also don’t understand where you get this notion of having to find an interface declaration and copying and pasting methods from. If you add an interface to a class the IDE will offer the methods from that interface for you to directly implement. The IDE will do the looking up and copying/pasting for you, if you want to think of it that way.
That was the whole point of this post. I have to wonder, did you read it ? 😉
And even if you insist on doing this leg work yourself then it is actually still less work than if you insist on doing the equivalent legwork for overriding a virtual method. With an interface you can just copy and paste the declaration. With a virtual method you cannot. You will have to change the “virtual” declaration to an “override” after you have pasted it.
i.e. lookup interface-copy-paste vs lookup base class-copy-paste-modify
And if you aren’t going to insist on doing this additional legwork in the case of virtual methods, why insist on doing it with interface methods when you don’t have to in either case ?
How do you know *which* interfaces are supported by the framework for a given purpose? how to you know it’s ICleanupTestCase and not ITeardown or iWhatever?
The answer is: you don’t, unless you’ve written them yourself, and done so recently, or had a look at the source code.
There is very little discoverability, especially once interfaces are many and scattered across units. In converse, the inheritance approach is very discoverable.
The argument about the IDE adding methods for you sounds a bit weird tbh. Don’t you use a boilerplate snippet? That’s a better way to solve it than compromising the framework simplicity with feature interfaces…
I mean, I never type the declaration of a test case unit and class, I just have it auto-generated as a whole or as a snippet, it’s standard enough, and there is more there than Setup/Teardown (which btw are okay to leave empty if you don’t use them, no point optimizing them away, and there may be a point when a new test case involves them down the road).
As for having all methods in the suggestions, well it’s a good way to know what you can do, and in practice a couple keystroke can filter the most wild suggestion lists. It’s preferable to having few suggestions and wondering from where to start from IMHO.
How do you know which inherited methods provided by a framework are for a given purpose ?
How do you know when the Setup/TearDown/RiptItUp/DemolishIt/Whatever method is called ?
The answer my friend is blowing in the
winddocumentation.Whether it is an interface or an inherited method, the answer is always in the documentation. And you cannot “discover” these things that you raise simply by looking at a declaration. The declaration will tell you what the methods look like. It will not tell you how they are or should be used.
And no, I don’t use boilerplate snippets because the framework isn’t so burdened by overloaded and virtualised behaviours that have to be remembered, that boilerplate generators are necessary to make the framework usable. And also because in Test Driven Development you typically write the tests first so you don’t even have any existing code to point a code generator at in order to generate the test cases!
One of the things that I disliked about DUnit from the outset was the amount of scaffolding needed to get a test up and running in the first place. Having code generators only helps up to a point and is most appropriate when you do your testing as an afterthought rather than as the first step.
I’m happy for you that you always use the auto-generated test case class. But since the code generation in DUnit fails miserably to create anything resembling a usable test case when confronted by such “sophisticated” code as a class function, then realistically I suspect that if you do always generate your code using the wizard then you must also be spending a lot of time needlessly cleaning up your generated test cases to a point where you can start writing your tests, rather than… you know, actually writing the tests. 😉
But either way, I have no intention of making Smoketest into the very thing that I rejected in the first place.
Instead of trying to convince me to re-shape Smoketest so that it is just an imitation of DUnit and thereby completely negate any reason for it to exist, why not just keep using DUnit instead of spending your time picking holes in an alternative you clearly have no interest in, even if your reasons don’t – to my mind – actually stack up ?
We will see how many followers and contributors you will have.
Thanks for sharing the project!
Popularity is not a measure of quality. 😉
Here’s some more cons to using interfaces:
1. If you have a single method in each interface, it’s lots of interfaces to remember (one for each function), double the typing when declaring a function (first its name, then its interface name which can be different – and for what?).
If you have several methods, then you have to implement all, even if you just need one. More typing.
2. With interfaces you cannot pre-implement default behavior and allow to override it.
You cannot call inherited default behavior (or can, but in a clumsy way, calling the similarly named function from another object).
3. If you *require* all your tests to inherit from a single type, then I don’t even see what’s the point of interfaces. What possibilities do they add except more typing?
If you don’t require that, then you’re somewhat encouraging people to add testing behavior to their normal classes. It’s better to design stuff in a way that can’t be abused. Single ancestor requirement seems pretty reasonable to me, never caused any inconveniences with DUnit.
That said, I haven’t used SmokeTest yet so there’s that. Maybe I’m missing something.
This is an artificial comparison – the number of interfaces (and methods) is already more because the framework supports more methods in this area. You don’t need to type both the interface and the method. Type the interface and let code completion fill in the method declaration for you.
With interfaces the only pre-impemented default behaviour that you have no control over in this case is the base test case class in the framework itself and you would not implement any default behaviour there anyway unless you were mad.
You can implement default behaviours in any test case classes you derive since any classes derived from your test case class will itself inherit the interface and the default implementation you provide.
So if you really have to have a DUnit style “override a method” approach, simply implement a “DUnitCase” class, implement all the interfaces as virtual methods, then derive your test cases from that “DUnit compatible” class. You can even implement a CheckEquals() method while you’re at it. 😉
Your logic is moot. When you start with DUnit you won’t just go and do that without any guidance (some person or some kind of documentation or tutorial) that will explain the Setup/TearDown things.
When starting with Smoketest you won’t do that either. You would have read this blog or something else.
In both cases you know what to do (either override virtual methods or implement some interface).
Forget about the DUnit Wizard – it’s a joke. But writing a simple DUnit test case is as easy as writing your first hello world in OOP.
I’m not sure who’s logic is moot. I’m not the one ignoring the existence of documentation so I can only think it was someone else’s logic that was moot. The only logic involved in my implementation decisions was “I don’t like the way DUnit does it”. You might not like the way I did it, but I don’t ask you to. If you prefer the DUnit way it is very simple to graft a DUnit compatability layer onto Smoketest. If you don’t like the way DUnit works it is actually very much more difficult to do anything about it.
Haven’t checked it out but it looks like a good project.
I don’t like interfaces (hate debugging the darned things) but the rest of the project is very appealing to me. Given that it is not likely I’ll need to debug the test framework itself, I think it doesn’t matter much you used them.
I love the fluid approach for testing.
Cheers!
Thanks for the comment, I hope you find the framework useful. It is relatively easy to avoid the interfaces if you really don’t like them, as I will show in a post soon. 🙂