http://c2.com/cgi/wiki?UnitTestingNonDeterministicCode
Some consider it to be a CodeSmell when they encounter code that is difficult to unit test.
- MockRandomizer -- Trick your random number generator into generating predictable numbers (such as 0,1,2,3...)
MockRandomizer means using a random number generator which you have modified or overridden to make its output predictable. It might always return the same value, or a predictable sequence.
A mock randomizer helps when UnitTestingNonDeterministicCode by making it deterministic. (Caveat: Must be "randomizer-driven", i.e. the non-nondeterminism isn't a consequence of a physical process, OS scheduling, etc). It allows: if (if get this random result) then assert(this state should hold). Accordingly, the test should predictably pass or fail.
An alternative stochastic approach is to repeat a test many times, and verify that the statistically expected result occurs on average within a stated tolerance. Naturally, sometimes such a test will fail or pass when it shouldn't have. The advantages of MockRandomizers over stochastic approaches are that they're less work to setup, and don't ever false positives or negatives.
A simple Java MockRandomizer example:
Deck = new Deck(); deck.setRandom(new Random() { public void nextDouble() { return 0.0; } }); deck.shuffle();We can mechanically work out what state a shuffled deck should be in, when all random call are zero, and assert it in the usual fashion. In my experience this does usefully exercise the code I want to test, though you may need several different MockRandomizers? to cover your cases.
- DontChangeTheCodeTest -- If you absolutely can't solve the problem, there's always this resort.
- Pare down the amount of untestable code as much as possible.
- Take that small bit, and manually verify its results.
- Create a test assertion that has:
- Some mechanism to detect if anyone changes the code (string comparison, datestamp comparison, MD5Sum comparison, etc.)
- Any expectations you have for the code inside the "test failed" message.
- SQL - For some evil database interactions, our DBAs create SQL appropriate for doing a task, and the programmers are generally told to avoid tinkering with it (sometimes the SQL has optimizations, other times it's just plain fragile). To keep anything from breaking, I paste the SQL in two places, one: the actual source, and two: the unit test assertion that fails with a message, "You should talk to so-and-so if you're going to change this SQL".
- HTML - A web-designer will sometimes craft a piece of HTML that looks right under all supported browsers, but can break in subtle ways. Instead of codifying those subtle ways into subtle unit tests, it might be better to just add a DontChangeTheCodeTest.
- JavaScript - Same deal. Just make sure you follow step #1 (pare down the amount of JavaScript as much as possible).
- NoTestsYetTest - For untested legacy code (I suppose "untested" and "legacy" are redundant).
- GUIs, other graphics - Though, these might be better if you take a snapshot of the results of the code rather than the code itself.
- Security - If you have some code that has to be a certain way or else it's insecure, first try to figure out a SourceTest. If you can't figure that out, I guess a DontChangeTheCodeTest would be a reasonable stopgap.