Tuesday, May 10, 2016

Unit Testing Non Deterministic Code



http://c2.com/cgi/wiki?UnitTestingNonDeterministicCode


Some consider it to be a CodeSmell when they encounter code that is difficult to unit test.
I agree that under most circumstances, code that's difficult to UnitTest is a smell.
  • MockRandomizer -- Trick your random number generator into generating predictable numbers (such as 0,1,2,3...)
http://c2.com/cgi/wiki?MockRandomizer
MockRandomizer means using a random number generator which you have modified or overridden to make its output predictable. It might always return the same value, or a predictable sequence.
A mock randomizer helps when UnitTestingNonDeterministicCode by making it deterministic. (Caveat: Must be "randomizer-driven", i.e. the non-nondeterminism isn't a consequence of a physical process, OS scheduling, etc). It allows: if (if get this random result) then assert(this state should hold). Accordingly, the test should predictably pass or fail.
An alternative stochastic approach is to repeat a test many times, and verify that the statistically expected result occurs on average within a stated tolerance. Naturally, sometimes such a test will fail or pass when it shouldn't have. The advantages of MockRandomizers over stochastic approaches are that they're less work to setup, and don't ever false positives or negatives.
A simple Java MockRandomizer example:
 Deck = new Deck();
 deck.setRandom(new Random() {
 public void nextDouble() {
  return 0.0;
 }
 });
 deck.shuffle();



We can mechanically work out what state a shuffled deck should be in, when all random call are zero, and assert it in the usual fashion. In my experience this does usefully exercise the code I want to test, though you may need several different MockRandomizers? to cover your cases.
  1. Pare down the amount of untestable code as much as possible.
  2. Take that small bit, and manually verify its results.
  3. Create a test assertion that has:
    1. Some mechanism to detect if anyone changes the code (string comparison, datestamp comparison, MD5Sum comparison, etc.)
    2. Any expectations you have for the code inside the "test failed" message.
Situations where this has successfully been used:
  • SQL - For some evil database interactions, our DBAs create SQL appropriate for doing a task, and the programmers are generally told to avoid tinkering with it (sometimes the SQL has optimizations, other times it's just plain fragile). To keep anything from breaking, I paste the SQL in two places, one: the actual source, and two: the unit test assertion that fails with a message, "You should talk to so-and-so if you're going to change this SQL".
  • HTML - A web-designer will sometimes craft a piece of HTML that looks right under all supported browsers, but can break in subtle ways. Instead of codifying those subtle ways into subtle unit tests, it might be better to just add a DontChangeTheCodeTest.
  • JavaScript - Same deal. Just make sure you follow step #1 (pare down the amount of JavaScript as much as possible).
  • NoTestsYetTest - For untested legacy code (I suppose "untested" and "legacy" are redundant).
Situations where this would probably be successful (if you've actually done this, and liked it, please move the items you have experience up to the previous list):
  • GUIs, other graphics - Though, these might be better if you take a snapshot of the results of the code rather than the code itself.
  • Security - If you have some code that has to be a certain way or else it's insecure, first try to figure out a SourceTest. If you can't figure that out, I guess a DontChangeTheCodeTest would be a reasonable stopgap.

Labels

Review (572) System Design (334) System Design - Review (198) Java (189) Coding (75) Interview-System Design (65) Interview (63) Book Notes (59) Coding - Review (59) to-do (45) Linux (43) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (31) Big Data (29) Product Architecture (28) MultiThread (27) Soft Skills (27) Concurrency (26) Cracking Code Interview (26) Miscs (25) Distributed (24) OOD Design (24) Google (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) System Design - Practice (20) Tips (19) Algorithm (17) Company - Facebook (17) Security (17) How to Ace Interview (16) Brain Teaser (14) Linux - Shell (14) Redis (14) Testing (14) Tools (14) Code Quality (13) Search (13) Spark (13) Spring (13) Company - LinkedIn (12) How to (12) Interview-Database (12) Interview-Operating System (12) Solr (12) Architecture Principles (11) Resource (10) Amazon (9) Cache (9) Git (9) Interview - MultiThread (9) Scalability (9) Trouble Shooting (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cassandra (8) Company - Uber (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Design (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Mac (7) Machine Learning (7) NoSQL (7) C++ (6) Chrome (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Python (5)

Popular Posts