TDD and tomatoes

Tomatoes are like tests, it is easier to make a tomato go from green to red, than from red to green.

TDD : Can Shared Fixture test unexpected statefulness ?

When praticing TDD, the impact of the different types of fixture that can be used to setup the tests must be known in order to avoid to create Obscure Tests

The usual default fixture type is the Fresh Fixture

When the fresh fixture takes too much time a Shared Fixture can be used. Now, that being said, the shared fixture should usually be avoided (there's a risk of Erratic Test)

A few days ago, during a discussion about test fixture, some colleagues of mine and I were wondering if shared fixture could be used in order to test the statelessness of the objects used in the fixture as a side effect of the sharing.

Here is the initial reasoning : when an object is supposed to be stateless, but is buggy and not actually stateless , the use of this object will lead to some failing test, since the state left by some tests will make some subsequent tests fail. The statelessness is tested "for free" with a lot of different combination. Actually, if the object happens to be stateful it is a case of Interacting Tests and the reason why some tests fail could be quite hard to find.

I must admit that it may help to find some obscure statefulness interaction, but I do not really like this idea :

• I fear Interacting Tests far more than a single bug, as Interacting Tests in a test suite is hell on earth,
• A test is supposed to have One Assertion Per Test. Or, at most a few assertions. In this case, the statefulness test is spread in the whole test class.

Actually, the statefulness should be tested in its own test. Of course it means that only a few method call combination will be tested but it should be enough to prove the statelessness (or else it means that the code of the class under test must be refactored in order to make the stalessness explicit in the code structure)

To try to use the side effect of the shared fixture to test statelessness may seem to be worthwhile at first, but it creates Technical Debt. And in this case I think that the financial interest of the debt is pretty high.

"Unit Integrated Tests" for integrator objects

Whether I use interaction based testing or state based testing I usually write "real" unit tests in the sense that only one real object is involved (the SUT, System Under Test) and only stubs and mocks are used for the fixture. It usually is the only way to easily get a good code coverage (or at least to easily specifically cover some parts of the code )

However, Lately, when testing Facade or Adapter objects he seemed to me that the unit tests for these objects were :
- too complicated
- too...useless

For most Facade or Adapter objects :
- They're stateless, so let's forget state based testing, we must use mocks
- They're highly coupled to the contract of the adapted objects (by definition of a Facade or of an Adapter)

This kind of class does nothing clever, they basically chain one or multiples calls. They may have bug though if they do not respect the contracts of the adapted classes.

So a unit test for this kind of class depends more on the contract of the mocked objects, than on the behavior of the class under test. A very clear test smell.

Even worse, the fixtures for the mocks are going to closely mimick the code of the SUT (if a method of a Facade calls an object A and then an object B, the fixture for the test on mocks and stubs will tell the exact same thing).

In this specific case the unit test does not provide an orthogonal view of the behavior implemented by the code (the usual value of a unit test) ; it is the exact same view. If I make a wrong assumption on the contract of the adapted objects, I'm going to make the same exact mistake both in the test and then in the code (TDDly speaking)

The problem basically is that the job of this kind of objects is to integrate other objects in the system. So he seemed to me that it is very hard to try to test an integration behavior in isolation.

That's why, now, when I must deal with objects that heavily depend on the contract of the collaborating objects but do not do anything complicated, "integrator objects", I use integrated tests.

However when I encounter an object that heavily depend on the contract of the collaborating objects and does many complicated things, it just means that it is too hard to test, I refactor.

TDD : How I was overusing mocks

The path to TDD is hard. When you think that you begin to understand how to apply it, you read this great Jay Fields post, and you understand that you missed the point about stubs...

As explained in his post, I was overusing mocks. I knew that I had to improve a lot in my way of using mocks, but what I really had to do was to use fewer mocks and more stubs. Basically I was missing the point that stubs are still of great use and are not superseded by mocks.

It's been a long time since I learned such an important thing in a post about TDD. Now I'm really going to try to stick to "one assert per test" (including "one mock assert per test")

And trying to do that, I'm also going to try "one test class per test setup". See this presentation about BDD by Dave Astels

EasyMock and methods with side effects

I really like EasyMock, but was disappointed to not see any explanation in the documention about how to simulate the modification of an argument of a method with side effects.

I thought that maybe I could write a custom argument matcher that modifies the argument (even if it seems weird to implement a side effect on the matches method of the IArgumentMatcher)

Actually, it was confirmed by this brilliant article about a Generic Custom Argument Matching in EasyMock

Continuous Integration and test environment availability

When a developper does not have an environment to run the unit tests of a projet, never allows this team member to check in, even if you have a continuous integration server running.

If the build is broken, this programmer will have to perform several check in with the new code and wait for the next build in order to know if the build was fixed. Et voilà, the continuous integration server has become the test environment for this person...

TDD : state-based testings VS behavior-base testing

Using TDD to write my code I'm still trying to find when I should use mock objects or just test against the whole cluster of objects. That's the difference between state-based testing (no mocks) and behavior-based testing (using mocks) explained in this brilliant article by Martin Fowler : Mocks Aren't Stubs

I'm sure that both technics are useful and I switch from one technic to an other depending on the complexity of the objects I'm testing.

• State-based testing (no mock) tend to increase the amount of integration test between objects, so is better suited to tests clusters of object with complex interaction


• Behaviour-based testing (with mocks) tend to increase the test coverage on a given object as it is easier to control the test environment of the object under test

Using mocks objects tend to create more work, but the more I use them the more I detect bugs in my code that would very hard, or impossible to detect with state-based testing.

Last time it happened it allowed me to see that an external object was called and was not supposed to be called (because the mock was called and warned me that it was an unexpected call)

I would have never been able to detect this call with state base testing because :

• It did not change the state of the object under test


• It did not change the state of the external object

However the call to the real external object is very CPU intensive and would have harmed the performance. Behavior based testing allowed me to see this, it would have been undetected with state-based testing.

TDD, red-green-refactor : red as important as green

When I got started with TDD, I tended to often forget the first step : red

At first, it can seem silly to try to run a test when you did not write the matching functional code. You may think "Of course, it is going to be red ! There's nothing in my application for this feature !"

Of course... Unless there's a bug in your test. And it could be green and you're not going to see it. Then, you write the application code, with a bug too, you run your buggy test, and it is green even with your buggy functional code...

Basically the red step of red-green-refactor helps you debug your test before your code. The red step really is some sort of TDD applied to the test itself. When there is no functional code, a test must be red. It does not mean that the test is correct, but if it is green, surely, it must be fixed.

And even if your test is red and buggy, it is not a big deal, because when your functional code will be written, the test will be red too. The problem could come from your functional code, from your test, or both, but at least you will have to fix something to make the test pass.

TDD like all Agile techniques, is all about rapid feedback. Red gives you feedback about your test, green feedback about your functional code.